karpathy/ minbpe
Minimal, clean code for the Byte Pair Encoding (BPE) algorithm commonly used in LLM tokenization.
by Andrej Python MIT licence 1,150 forks Created Feb 2024 View on GitHub
- Stars
- 11,024
- 2% of karpathy's stars
- Rank
- #5,100
- of 847,070 repos, top 0.6%
- In Python
- #952
- of 172,206 repos, top 0.6%
Among 847,070 repos
10k–16k stars: 2,590 repos, including minbpe
top 50%top 10%top 1%top 0.1%
Stars along the bottom, and repos in each band on a log scale
Quick stats
- Next rank
- +3 stars to reach #5,099
- Next milestone
- +27,121 stars to reach the top 0.1%
- Popular
- One of 5,746 repos with 10k+ stars
- Yearly average
- 4,034 stars a year since Feb 2024