EleutherAI

A framework for few-shot evaluation of language models.

An implementation of model parallel GPT-2 and GPT-3-style models using the mesh-tensorflow library.

An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries

The hub for EleutherAI's work on interpretability and learning dynamics

Deep learning for dummies. All the practical details and useful utilities that go into working with real models.

Sparsify transformers with SAEs and transcoders

Polyglot: Large Language Models of Well-balanced Competence in Multi-languages

Open-AI's DALL-E for large scale training in mesh-tensorflow.

Delphi was the home of a temple to Phoebus Apollo, which famously had the inscription, 'Know Thyself.' This library lets language models know themselves through automated interpretability.

Erasing concepts from neural representations with provable guarantees

Keeping language models honest by directly eliciting knowledge encoded in their activations.

OSLO: Open Source for Large-scale Optimization

A library for finding knowledge neurons in pretrained transformer models.

Official repository for the paper: Scaling Self-Supervised Representation Learning for Symbolic Piano Performance (ISMIR 2025)

Mapping out the "memory" of neural nets with data attribution

Python Research Framework

Data processing system for polyglot

Python tools for processing the stackexchange data dumps into a text dataset for Language Models

Efficient and robust implementation of seq-to-seq automatic piano transcription.

Experiments for efforts to train a new and improved t5

See the issue board for the current status of active and prospective projects!

One stop shop for all things carp