bigscience-workshop

🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading

Toolkit for creating, sharing and using natural language prompts.

Ongoing research training transformer language models at scale, including: BERT & GPT-2

Central place for the engineering/scaling WG: documentation, SLURM scripts and logs, compute environment and data.

Crosslingual Generalization through Multitask Finetuning

Tools for curating biomedical training data for large-scale language modeling

Reproduce results and replicate training fo T0 (Multitask Prompted Training Enables Zero-Shot Task Generalization)

Code used for sourcing and cleaning the BigScience ROOTS corpus

Tools for managing datasets for governance and training.

Libraries, Archives and Museums (LAM)

BLOOM+1: Adapting BLOOM model to support a new unseen language