gty111 / gLLM Star 53 Code Issues Pull requests Discussions gLLM: Global Balanced Pipeline Parallelism System for Distributed LLM Serving with Token Throttling pipeline-parallelism tensor-parallelism llm-serving llm-inference pagedattention continuous-batching qwen3 token-throttling chunked-prefill Updated Jan 12, 2026 Python
psmarter / mini-infer Star 36 Code Issues Pull requests A high-performance LLM inference engine with PagedAttention | 基于PagedAttention的高性能大模型推理引擎 python machine-learning ai deep-learning gpu cuda inference pytorch transformer triton language-model llm pagedattention Updated Dec 31, 2025 Python
Rianbajukendari / mini-infer Star 0 Code Issues Pull requests 🚀 Accelerate LLM inference with Mini-Infer, a high-performance engine designed for efficiency and power in AI model deployment. python machine-learning ai deep-learning gpu cuda inference pytorch transformer triton language-model llm pagedattention Updated Feb 1, 2026 Python