vllm-project

A high-throughput and memory-efficient inference and serving engine for LLMs

A framework for efficient model inference with omni-modality models

An open, programmable decision layer for models and compute.

Cost-efficient and pluggable Infrastructure components for GenAI inference

State-of-the-art LLM compression, built for production inference with vLLM

Community maintained hardware plugin for vLLM on Huawei Ascend

vLLM’s reference system for K8S-native cluster-wide deployment with community-driven performance optimization

Community maintained hardware plugin for vLLM on Apple Silicon

Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs

Common recipes to run vLLM

A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM

A high-performance and light-weight router for vLLM large scale deployment

An LLM post-training framework with vLLM for RL Scaling

Moving to vllm-project/vllm-jax. TPU inference for vLLM, with unified JAX and PyTorch support.

Humming is a high-performance, lightweight, and highly flexible JIT (Just-In-Time) compiled GEMM kernel library specifically designed for quantized inference.

Stateful API logic for agentic applications using vLLM

A safetensors extension to efficiently store sparse quantized tensors on disk

vLLM plugin for attention-ffn disaggregation support

Agent skills for vLLM

The vLLM XPU kernels for Intel GPU

Community maintained hardware plugin for vLLM on Intel Gaudi

This repo hosts code for vLLM CI & Performance Benchmark infrastructure.

Community maintained hardware plugin for vLLM on AWS Neuron

vLLM Daily Summarization of Merged PRs

vLLM Quantization plugin for GGUF

High-performance Rust benchmark client for vLLM serving endpoints.