triton-inference-server

Triton Inference Server Joined Jul 2020 View on GitHub

Stars: 18,621; across 35 public repos
Rank: #2,023; of 95,338 organizations, top 3%; Down 4 places this week

Repositories with 50+ stars

triton-inference-server's repositories with their overall rank
Rank	Repository	Language	Stars
5,035	server The Triton Inference Server provides an optimized cloud and edge inferencing solution.	Python	11,136
70,127	tensorrtllm_backend The Triton TensorRT-LLM Backend		944
75,663	tutorials This repository contains tutorials and examples for Triton Inference Server	Python	870
77,319	pytriton PyTriton is a Flask/FastAPI-like interface that simplifies Triton's deployment in Python environments.	Python	850
92,186	client Triton Python, C++ and Java client libraries, and GRPC-generated client examples for go, java and scala.	C++	704
94,461	python_backend Triton backend that enables pre-process, post-processing and other logic to be implemented in Python.	C++	685
119,315	model_analyzer Triton Model Analyzer is a CLI tool to help with better understanding of the compute and memory requirements of the Triton Inference Server models.	Python	531
148,723	fastertransformer_backend	Python	414
159,280	backend Common source, scripts and utilities for creating Triton backends.	C++	383
166,690	vllm_backend	Python	364
250,458	model_navigator Triton Model Navigator is an inference toolkit designed for optimizing and deploying Deep Learning models with a focus on NVIDIA GPUs.	Python	226
291,652	onnxruntime_backend The Triton backend for the ONNX Runtime.	C++	188
296,852	core The core library and APIs implementing the Triton Inference Server.	C++	184
296,852	pytorch_backend The Triton backend for the PyTorch TorchScript models.	C++	184
326,751	perf_analyzer	Python	164
355,727	dali_backend The Triton backend that allows running GPU-accelerated data pre-processing pipelines implemented in DALI's python API.	C++	148
498,345	tensorrt_backend The Triton backend for TensorRT.	C++	97
510,716	fil_backend FIL backend for the Triton Inference Server	C++	94
574,662	common Common source, scripts and utilities shared across all Triton repositories.	C++	81
623,409	triton_cli Triton CLI is an open source command line interface that enables users to create, deploy, and profile models served by the Triton Inference Server.	Python	73
746,519	tensorflow_backend The Triton backend for TensorFlow.	C++	58
746,519	hugectr_backend	Jupyter Notebook	58

Among 95,338 organizations

16k–25k stars: 926 organizations, including triton-inference-server

top 50%top 10%top 1%top 0.1%

Stars along the bottom, and organizations in each band on a log scale

Where the stars are

server 11.1k, 60%tensorrtllm_backend 944, 5%tutorials 870, 5%pytriton 850, 5%client 704, 4%python_backend 685, 4%model_analyzer 531, 3%fastertransformer_backend 414, 2%backend 383, 2%vllm_backend 364, 2%model_navigator 226, 1%

3 more repos 189, 1%

onnxruntime_backend 188, 1%

Smaller repos 185, 1%

core 184, 1%pytorch_backend 184, 1%perf_analyzer 164, 1%dali_backend 148, 1%tensorrt_backend 97, 1%fil_backend 94, 1%common 81, <1%

Languages

Python 14.6k stars, 79% · 9 repos
C++ 2.8k stars, 15% · 11 repos
Jupyter Notebook 58 stars, <1% · 1 repo

When they were made

22 repos with 50+ stars, made between 2018 and 2023

101001k10k100k tutorialstensorrtllm_backendserver

Quick stats

Next rank: +13 stars to reach #2,022
Next milestone: +16,349 stars to reach the top 1%
Public repos: 35, 22 with 50+ stars
Popular repos: 1 over 1k stars, 1 over 10k stars
Top repo: server, #5,035 of all repos, 60% of stars
Smaller repos: 185 stars from repos under 50