coder/ ai-tokenizer
A faster than tiktoken tokenizer with first-class support for Vercel's AI SDK.
by Coder TypeScript Other licence 6 forks Created Oct 2025 View on GitHub
- Stars
- 63
- <1% of coder's stars
- Rank
- #696,118
- of 847,083 repos, top 83%
- In TypeScript
- #51,863
- of 61,833 repos, top 84%
Among 847,083 repos
63–100 stars: 216,335 repos, including ai-tokenizer
top 50%top 10%top 1%top 0.1%
Stars along the bottom, and repos in each band on a log scale
Quick stats
- Next rank
- +1 star to reach #687,784
- Next milestone
- +3 stars to reach the top 80%
- Yearly average
- 58 stars a year since Oct 2025