Pinned Loading

1. dash-infer dash-infer Public

   Forked from modelscope/dash-infer

   DashInfer is a native LLM inference engine aiming to deliver industry-leading performance atop various hardware architectures, including x86 and ARMv9.

   C

2. sglang sglang Public

   Forked from sgl-project/sglang

   SGLang is a fast serving framework for large language models and vision language models.

   Python

3. vllm vllm Public

   Forked from vllm-project/vllm

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python

4. ggml-org/llama.cpp ggml-org/llama.cpp Public

   LLM inference in C/C++

   C++ 85.7k 12.9k

5. modelscope/dash-infer modelscope/dash-infer Public

   DashInfer is a native LLM inference engine aiming to deliver industry-leading performance atop various hardware architectures, including CUDA, x86 and ARMv9.

   C 262 28