OpenGVLab

[CVPR 2024 Oral] InternVL Family: A Pioneering Open-Source Alternative to GPT-4o. 接近GPT-4o表现的开源多模态对话模型

[ICLR 2024] Fine-tuning LLaMA to follow Instructions within 1 Hour and 1.2M Parameters

Unofficial Implementation of DragGAN - "Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifold" （DragGAN 全功能实现，在线Demo，本地部署试用，代码、模型已全部开源，支持Windows, macOS, Linux）

[CVPR2024 Highlight][VideoChatGPT] ChatGPT with video understanding! And many more supported LMs such as miniGPT4, StableLM, and MOSS.

InternGPT (iGPT) is an open source demo platform where you can easily showcase your AI models. Now it supports DragGAN, ChatGPT, ImageBind, multimodal chat like GPT-4, SAM, interactive image...

[CVPR 2023 Highlight] InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutions

[ECCV2024] Video Foundation Models & Data for Multimodal Understanding

VisionLLM Series

[ICLR 2026 Oral] ScaleCUA is the open-sourced computer use agents that can operate on cross-platform environments (Windows, macOS, Ubuntu, Android).

Official implementation of SAM-Med2D

[ECCV2024] VideoMamba: State Space Model for Efficient Video Understanding

[ICLR2024 spotlight] OmniQuant is a simple and powerful quantization technique for LLMs.

[CVPR 2023] VideoMAE V2: Scaling Video Masked Autoencoders with Dual Masking

[CVPR 2024] Deformable Convolution v4

Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memory

Chatbot Arena meets multi-modality! Multi-Modality Arena allows you to benchmark vision-language models side-by-side while providing images as inputs. Supports MiniGPT-4, LLaMA-Adapter V2, LLaVA,...

[ICLR 2025 Spotlight] Vision-RWKV: Efficient and Scalable Visual Perception with RWKV-Like Architectures

[ICLR2026] VideoChat-Flash: Hierarchical Compression for Long-Context Video Modeling

[ICLR 2024 & ECCV 2024] The All-Seeing Projects: Towards Panoptic Visual Recognition&Understanding and General Relation Comprehension of the Open World"

[ICLR 2025 Spotlight] OmniCorpus: A Unified Multimodal Corpus of 10 Billion-Level Images Interleaved with Text

[CVPR 2023] Prompt, Generate, then Cache: Cascade of Foundation Models makes Strong Few-shot Learners

[T-PAMI 2025] PonderV2: Pave the Way for 3D Foundation Model with A Universal Pre-training Paradigm

Instruct2Act: Mapping Multi-modality Instructions to Robotic Actions with Large Language Model

[ACL 2025 Main] EfficientQAT: Efficient Quantization-Aware Training for Large Language Models

[ICCV2023] UniFormerV2: Spatiotemporal Learning by Arming Image ViTs with Video UniFormer

[ICCV2023 Oral] Unmasked Teacher: Towards Training-Efficient Video Foundation Models

[NeurIPS 2023 Datasets and Benchmarks Track] LAMM: Multi-Modal Large Language Models and Applications as AI Agents

InternVL-U is a 4B-parameter unified multimodal model (UMM) that brings multimodal understanding, reasoning, image generation, image editing into a single framework.

The suite of modeling video with Mamba

[NIPS2025] VideoChat-R1 & R1.5: Enhancing Spatio-Temporal Perception and Reasoning via Reinforcement Fine-Tuning

MM-Interleaved: Interleaved Image-Text Generative Modeling via Multi-modal Feature Synchronizer

This repo is official implementation of HumanBench (CVPR2023)

Diffree: Text-Guided Shape Free Object Inpainting with Diffusion Model

ControlLLM: Augment Language Models with Tools by Searching on Graphs

General Vision Benchmark, GV-B, a project from OpenGVLab

[ICML2025] The code and data of Paper: Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generation

Official PyTorch implementation of UniHCP

[ICCV 2025] GUIOdyssey is a comprehensive dataset for training and evaluating cross-app navigation agents. GUIOdyssey consists of 8,834 episodes from 6 mobile devices, spanning 6 types of...

An official implementation of "Hulk: A Universal Knowledge Translator for Human-Centric Tasks"

[CVPR 2024 Champions][ICLR 2025] Solutions for EgoVis Chanllenges in CVPR 2024

[ACL 2024] ChartAssistant is a chart-based vision-language model for universal chart comprehension and reasoning.

[NeurIPS 2024] Needle In A Multimodal Haystack (MM-NIAH): A comprehensive benchmark designed to systematically evaluate the capability of existing MLLMs to comprehend long multimodal documents.

ZeroGUI: Automating Online GUI Learning at Zero Human Cost

[ICML 2024] | MMT-Bench: A Comprehensive Multimodal Benchmark for Evaluating Large Vision-Language Models Towards Multitask AGI

[NeurIPS 2024 Spotlight ⭐️ & TPAMI 2025] Parameter-Inverted Image Pyramid Networks (PIIP)

Train InternViT-6B in MMSegmentation and MMDetection with DeepSpeed

[CVPR 2025] Mono-InternVL: Pushing the Boundaries of Monolithic Multimodal Large Language Models with Endogenous Visual Pre-training

[ICCV 23]An approach to enhance the efficiency of Vision Transformer (ViT) by concurrently employing token pruning and token merging techniques, while incorporating a differentiable compression rate.

[Accepted By EMNLP 2026 Main Conference] Sequential Diffusion Language Model (SDLM) enhances pre-trained autoregressive language models by adaptively determining generation length and maintaining...

[ICLR2025] MMIU: Multimodal Multi-image Understanding for Evaluating Large Vision-Language Models

Vinci: A Real-time Embodied Smart Assistant based on Egocentric Vision-Language Model

[CVPR 2024] Data and benchmark code for the EgoExoLearn dataset

[CVPR 2023] implementation of Towards All-in-one Pre-training via Maximizing Multi-modal Mutual Information.

Visual Embodied Brain: Let Multimodal Large Language Models See, Think, and Control in Spaces

「AAAI 2024」 Referred by Multi-Modality: A Unified Temporal Transformers for Video Object Segmentation

Awesome-DragGAN: A curated list of papers, tutorials, repositories related to DragGAN

Official Implementation of "Denoising Diffusion Semantic Segmentation with Mask Prior Modeling"

[ICML 2026] GenExam: A Multidisciplinary Text-to-Image Exam

[NeurIPS 2024] Vision Model Pre-training on Interleaved Image-Text Data via Latent Compression Learning

A curated list of the papers, repositories, tutorials, and anythings related to the large language models for tools

Task Preference Optimization: Improving Multimodal Large Language Models with Vision Task Alignment

[ICLR 2025] TimeSuite: Improving MLLMs for Long Video Understanding via Grounded Tuning

[ICML2023] Long-Term Rhythmic Video Soundtracker

[ICCV2025] V2PE: Improving Multimodal Long-Context Capability of Vision-Language Models with Variable Visual Position Encoding

[CVPR 2025] PVC: Progressive Visual Token Compression for Unified Image and Video Processing in Large Vision-Language Models