huggingface/tokenizers

💥 Fast State-of-the-Art Tokenizers optimized for Research and Production

by Hugging Face Rust Apache-2.0 licence 1,252 forks Created Nov 2019 View on GitHub

Stars
11,234
2% of huggingface's stars
Rank
#4,958
of 847,070 repos, top 0.6%
In Rust
#257
of 23,784 repos, top 2%

Among 847,070 repos

Compare with

10k–16k stars: 2,590 repos, including tokenizers

Stars along the bottom, and repos in each band on a log scale

Quick stats

Next rank
+9 stars to reach #4,956
Next milestone
+26,911 stars to reach the top 0.1%
Popular
One of 5,746 repos with 10k+ stars
Yearly average
1,599 stars a year since Nov 2019

Stars counted Recounted daily Next count around