IST-DASLab/
FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.
by IST Austria Distributed Algorithms and Systems Lab Python Apache-2.0 licence 96 forks Created Jan 2024 View on GitHub
- Stars
- 1,174
- 16% of IST-DASLab's stars
- Rank
- #57,577
- of 847,120 repos, top 7%
- In Python
- #10,385
- of 172,217 repos, top 7%
Among 847,120 repos
1k–1.6k stars: 23,533 repos, including marlin
top 50%top 10%top 1%top 0.1% 
Stars along the bottom, and repos in each band on a log scale
Quick stats
- Next rank
- +1 star to reach #57,539
- Next milestone
- +6,058 stars to reach the top 1%
- Popular
- One of 66,327 repos with 1k+ stars
- Yearly average
- 417 stars a year since Jan 2024