meta-pytorch

Simple and efficient pytorch-native transformer text generation in <1000 LOC of python.

PyTorch native post-training library

Model interpretability and understanding for PyTorch

Bayesian optimization in PyTorch

Pytorch domain library for recommendation systems

Training PyTorch models with differential privacy

A lightweight library for PyTorch training tools and utilities

A batched offline inference oriented version of segment-anything

A PyTorch repo for data loading and utilities to be shared by the PyTorch domain libraries.

Helpful tools and examples for working with flex-attention

Media decoding and encoding for PyTorch: videos, images, and audio, on CPU and GPU.

PyTorch Single Controller

PyTorch-native post-training at scale

LeanRL is a fork of CleanRL, where selected PyTorch scripts optimized for performance using compile and cudagraphs.

Autonomous GPU Kernel Generation & Optimization via Deep Agents

Fault tolerance for PyTorch (HSDP, LocalSGD, DiLoCo, Streaming DiLoCo)

TorchX is a universal job launcher for PyTorch applications. TorchX is designed to have fast iteration time for training/research and support for E2E production ML pipelines when you're ready.

(deprecated) torchcomms: a modern PyTorch communications API

Tritonbench is a collection of PyTorch custom operators with example inputs to measure their performance.

Applied AI experiments and examples for PyTorch

A library that contains a rich collection of performant PyTorch model metrics, a simple interface to create new metrics, a toolkit to facilitate metric computation in distributed training and...

This is a repository for all workshop related materials.

Configuration classes enabling type-safe PyTorch configuration for Hydra apps

TritonParse: A Compiler Tracer, Visualizer, and Reproducer for Triton Kernels

This repository contains the experimental PyTorch native float8 training UX

torch::deploy (multipy for non-torch uses) is a system that lets you get around the GIL problem by running multiple Python interpreters in a single C++ process.

A performant, memory-efficient checkpointing library for PyTorch applications, designed with large, complex distributed workloads in mind.

MSLK (Meta Superintelligence Labs Kernels) is a collection of PyTorch GPU operator libraries that are designed and optimized for GenAI training and inference, such as FP8 row-wise quantization and...

Ship correct and fast LLM kernels to PyTorch

TorchFix - a linter for PyTorch-using code with autofix support

Cryptographically secure pseudorandom number generators for PyTorch

Triton-based Symmetric Memory operators and examples

An experimental implementation of compiler-driven automatic sharding of models across a given device mesh.

TORCH_TRACE parser for PT2

A storage solution for PyTorch tensors with distributed tensor support.

Example apps and demos using PyTorch's ExecuTorch framework

Dist-MoE implements Speed-of-Light distributed Mixture-of-Experts forward and backward kernels. They are highly optimized and fused comm-compute kernels for training and inference on SM100+ GPUs

C++ implementations for various tokenizers (sentencepiece, tiktoken etc).