microsoft/
[EMNLP'23, ACL'24] To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss.
by Microsoft Python MIT licence 451 forks Created Jul 2023 View on GitHub
- Stars
- 6,812
- <1% of microsoft's stars
- Rank
- #9,063
- of 847,070 repos, top 2%
- In Python
- #1,668
- of 172,206 repos, top 1%
Among 847,070 repos
6.3k–10k stars: 4,190 repos, including LLMLingua
top 50%top 10%top 1%top 0.1% 
Stars along the bottom, and repos in each band on a log scale
Quick stats
- Next rank
- +1 star to reach #9,059
- Next milestone
- +418 stars to reach the top 1%
- Popular
- One of 66,291 repos with 1k+ stars
- Yearly average
- 2,035 stars a year since Jul 2023