EvolvingLMMs-Lab

One-for-All Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks

🦦 Otter, a multi-modal model based on OpenFlamingo (open-sourced version of DeepMind's Flamingo), trained on MIMIC-IT and showcasing improved instruction-following and in-context learning ability.

A fork to add multimodal model training to open-r1

Fully Open Framework for Democratized Multimodal Training

NEO Series: Native Vision-Language Models from First Principles

A simple, unified multimodal models training engine. Lean, flexible, and built for hacking at scale.

Relate Anything Model is capable of taking an image as input and utilizing SAM to identify the corresponding mask within the image.

[ACL-2026] MMSearch-R1 is an end-to-end RL framework that enables LMMs to perform on-demand, multi-turn search with real-world multimodal search tools.

[CVPR 2025] EgoLife: Towards Egocentric Life Assistant

Long Context Transfer from Language to Vision

Codec-Aligned Sparsity as a Foundational Principle for Multimodal Intelligence

Agentic LaTeX Writer - Local-first editor for AI-assisted academic writing

[CVPR 2026] LongVT: Incentivizing "Thinking with Long Videos" via Native Tool Calling

SkillOpt-Lite and HarnessOpt: Optimize your skill or harness with one line of vibe

[ICCV 2025] Auto Interpretation Pipeline and many other functionalities for Multimodal SAE Analysis.

A simple video streaming baseline that outperforms SOTAs.

[CVPR 2026] OpenMMReasoner: Pushing the Frontiers for Multimodal Reasoning with an Open and General Recipe

[Roadmap] Visual Generation in the New Era: An Evolution from Atomic Mapping to Agentic World Modeling

Holistic Evaluation of Multimodal LLMs on Spatial Intelligence

Evaluating Knowledge Acquisition from Multi-Discipline Professional Videos

[NeurIPS 2026] ParaVT: Taming the Tool Prior Paradox for Parallel Tool Use in Agentic Video Reinforcement Learning

High-Resolution Visual Reasoning via Multi-Turn Grounding-Based Reinforcement Learning

Fully Open Framework for Democratized Multimodal Reinforcement Learning.

A framework that allows you to apply Sparse AutoEncoder on any models