IBM/
ModuleFormer is a MoE-based architecture that includes two different types of experts: stick-breaking attention heads and feedforward experts. We released a collection of ModuleFormer-based...
by International Business Machines Python Apache-2.0 licence 11 forks Created Aug 2023 View on GitHub
- Stars
- 223
- <1% of IBM's stars
- Rank
- #252,362
- of 847,083 repos, top 30%
- In Python
- #48,991
- of 172,210 repos, top 29%
Among 847,083 repos
160–250 stars: 103,741 repos, including ModuleFormer
top 50%top 10%top 1%top 0.1% 
Stars along the bottom, and repos in each band on a log scale
Quick stats
- Next rank
- +1 star to reach #251,455
- Next milestone
- +133 stars to reach the top 20%
- Popular
- One of 488,395 repos with 100+ stars
- Yearly average
- 69 stars a year since Aug 2023