lucidrains

Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch

Implementation of DALL-E 2, OpenAI's updated text-to-image synthesis neural network, in Pytorch

Implementation of Denoising Diffusion Probabilistic Model in Pytorch

Implementation of Imagen, Google's Text-to-Image Neural Network, in Pytorch

Implementation of RLHF (Reinforcement Learning with Human Feedback) on top of the PaLM architecture. Basically ChatGPT but with PaLM

A concise but complete full-attention transformer with a set of promising experimental features from various papers

Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch

Simple command line tool for text to image generation using OpenAI's CLIP and Siren (Implicit neural representation network). Technique was originally created by https://twitter.com/advadnoun

Vector (and Scalar) Quantization, in Pytorch

Simplest working implementation of Stylegan2, state of the art generative adversarial network, in Pytorch. Enabling everyone to experience disentanglement

Implementation of MusicLM, Google's new SOTA model for music generation using attention networks, in Pytorch

Implementation of AudioLM, a SOTA Language Modeling Approach to Audio Generation out of Google Research, in Pytorch

A simple command line tool for text to image generation, using OpenAI's CLIP and a BigGAN. Technique was originally created by https://twitter.com/advadnoun

🦁 Lion, new optimizer discovered by Google Brain using genetic algorithms that is purportedly better than Adam(w), in Pytorch

Reformer, the efficient Transformer, in Pytorch

Implementation of Toolformer, Language Models That Can Use Tools, by MetaAI

Unofficial implementation of Titans, SOTA memory for transformers, in Pytorch

Implementation of Make-A-Video, new SOTA text to video generator from Meta AI, in Pytorch

Implementation of GigaGAN, new SOTA GAN out of Adobe. Culmination of nearly a decade of research into GANs

Usable Implementation of "Bootstrap Your Own Latent" self-supervised learning, from Deepmind, in Pytorch

Implementation of Alphafold 3 from Google Deepmind in Pytorch

Implementation of 'lightweight' GAN, proposed in ICLR 2021, in Pytorch. High resolution image generations that can be trained within a day or two

To eventually become an unofficial Pytorch implementation / replication of Alphafold2, as details of the architecture get released

Implementation of SoundStorm, Efficient Parallel Audio Generation from Google Deepmind, in Pytorch

Implementation of LambdaNetworks, a new approach to image recognition that reaches SOTA with less compute

Implementation of the training framework proposed in Self-Rewarding Language Model, from MetaAI

Pytorch implementation of Transfusion, "Predict the Next Token and Diffuse Images with One Multi-Modal Model", from MetaAI

Implementation of Video Diffusion Models, Jonathan Ho's new paper extending DDPMs to Video Generation - in Pytorch

Implementation of Natural Speech 2, Zero-shot Speech and Singing Synthesizer, in Pytorch

Implementation of 🦩 Flamingo, state-of-the-art few-shot visual question answering attention net out of Deepmind, in Pytorch

Implementation of Perceiver, General Perception with Iterative Attention, in Pytorch

Implementation of CoCa, Contrastive Captioners are Image-Text Foundation Models, in Pytorch

An implementation of Performer, a linear attention-based transformer, in Pytorch

Implementation of TabTransformer, attention network for tabular data, in Pytorch

An All-MLP solution for Vision, from Google AI

Implementation of Band Split Roformer, SOTA Attention network for music source separation out of ByteDance AI Labs

Implementation of Muse: Text-to-Image Generation via Masked Generative Transformers, in Pytorch

Implementation of RETRO, Deepmind's Retrieval based Attention net, in Pytorch

A Pytorch implementation of Sparsely-Gated Mixture of Experts, for massively increasing the parameter count of language models

Implementation of MeshGPT, SOTA Mesh generation using Attention, in Pytorch

Transformer based on a variant of attention that is linear complexity in respect to sequence length

Implementation of Rotary Embeddings, from the Roformer paper, in Pytorch

Implementation of the specific Transformer architecture from PaLM - Scaling Language Modeling with Pathways

Implementation of the sparse attention pattern proposed by the Deepseek team in their "Native Sparse Attention" paper

Implementation of Phenaki Video, which uses Mask GIT to produce text guided videos of up to 2 minutes in length, in Pytorch

Implementation of TimeSformer from Facebook AI, a pure attention-based solution for video classification

A concise but complete implementation of CLIP with various experimental improvements from recent papers

Implementation of Voicebox, new SOTA Text-to-speech network from MetaAI, in Pytorch

Implementation of Bottleneck Transformer in Pytorch

Implementation of MagViT2 Tokenizer in Pytorch

A simple way to keep track of an Exponential Moving Average (EMA) version of your Pytorch model

Implementation of MEGABYTE, Predicting Million-byte Sequences with Multiscale Transformers, in Pytorch

Implementation of Memorizing Transformers (ICLR 2022), attention net augmented with indexing and retrieval of memories using approximate nearest neighbors, in Pytorch

Implementation of the Point Transformer layer, in Pytorch

Implementation of π₀, the robotic foundation model architecture proposed by Physical Intelligence

Implementation of Enformer, Deepmind's attention network for predicting gene expression, in Pytorch

Implementation of a single layer of the MMDiT, proposed in Stable Diffusion 3, in Pytorch

Implementation of 💍 Ring Attention, from Liu et al. at Berkeley AI, in Pytorch

Implementation of NÜWA, state of the art attention network for text to video synthesis, in Pytorch

Implementation of Classifier Free Guidance in Pytorch, with emphasis on text conditioning, and flexibility to include multiple text embedding models

Unofficial implementation of iTransformer - SOTA Time Series Forecasting using Attention networks, out of Tsinghua / Ant group

Implementation of Parti, Google's pure attention-based text-to-image neural network, in Pytorch

Implementation of E(n)-Equivariant Graph Neural Networks, in Pytorch

Implementation of E2-TTS, "Embarrassingly Easy Fully Non-Autoregressive Zero-Shot TTS", in Pytorch

Pytorch implementation of SIREN - Implicit Neural Representations with Periodic Activation Function

An implementation of local windowed attention for language modeling

Implementation of Slot Attention from GoogleAI

Implementation of rectified flow and some of its followup research / improvements in Pytorch

Implementation of RT1 (Robotic Transformer) in Pytorch

Implementation of Autoregressive Diffusion in Pytorch

Implementation of the convolutional module from the Conformer paper, for use in Transformers

Implementation of gMLP, an all-MLP replacement for Transformers, in Pytorch

Implementation of Recurrent Memory Transformer, Neurips 2022 paper, in Pytorch

Implementation of Q-Transformer, Scalable Offline Reinforcement Learning via Autoregressive Q-Functions, out of Google Deepmind

Implementation of Axial attention - attending to multi-dimensional data efficiently

Implementation of a memory efficient multi-head attention as proposed in the paper, "Self-attention Does Not Need O(n²) Memory"

Implementation of ST-Moe, the latest incarnation of MoE after years of research at Brain, in Pytorch

Implementation of the Transformer variant proposed in "Transformer Quality in Linear Time"

Implementation of Segformer, Attention + MLP neural network for segmentation, in Pytorch

Implementation of Deformable Attention in Pytorch from the paper "Vision Transformer with Deformable Attention"

Implementation of Bit Diffusion, Hinton's group's attempt at discrete denoising diffusion, in Pytorch

Implementation of Soft MoE, proposed by Brain's Vision team, in Pytorch

Implementation of SE3-Transformers for Equivariant Self-Attention, in Pytorch. This specific repository is geared towards integration with eventual Alphafold2 replication.

Implementation of the proposed minGRU in Pytorch

Explorations into training LLMs to use clinical calculators from patient history, using open sourced models. Will start with Wells' Criteria

Explorations into some recent techniques surrounding speculative decoding

Implementation of Linformer for Pytorch

Implementation of Transformer in Transformer, pixel level attention paired with patch level attention for image classification, in Pytorch

Fully featured implementation of Routing Transformer

Quick implementation of nGPT, learning entirely on the hypersphere, from NvidiaAI

Implementation of a U-net complete with efficient attention as well as the latest research findings

Implementation of the Equiformer, SE3/E3 equivariant attention network that reaches new SOTA, and adopted for use by EquiFold for protein folding

Implementation of Lumiere, SOTA text-to-video generation from Google Deepmind, in Pytorch

Implementation of a Transformer, but completely in Triton

Implementation of Spear-TTS - multi-speaker text-to-speech attention network, in Pytorch

Sinkhorn Transformer - Practical implementation of Sparse Sinkhorn Attention

Implementation of Pixel-level Contrastive Learning, proposed in the paper "Propagate Yourself", in Pytorch

Use Jax functions in Pytorch

Implementation of the Adan (ADAptive Nesterov momentum algorithm) Optimizer in Pytorch

Implementation of Graph Transformer in Pytorch, for potential use in replicating Alphafold2