| Mar 01, 2025 | EgoLife👓 is accepted by CVPR 2025. |
| Jan 23, 2025 | LMMs-Eval⚖️ is accepted by NAACL2025 Findings. |
| Aug 13, 2024 | Join MMLab@NTU as a master student! 🚀🚀🚀 |
| Jul 17, 2024 | We introduce LMMs-Eval, a comprehensive and efficient benchmark for evaluating Large Multimodal Models, alongside LMMs-Eval Lite and Multimodal Livebench, which ensure low-cost and contamination-free evaluations in dynamic environments. |
| Jul 01, 2024 | Octopus🐙 is accepted by ECCV-2024. |
| Jun 12, 2024 | We introduce lmms-eval/v0.2.0 to support video evaluations for video models like LLaVA-NeXT Video and Gemini 1.5 Pro across tasks such as EgoSchema, PerceptionTest, VideoMME, and more. |
| Oct 12, 2023 | We introduce Octopus, an embodied vision language programmer that plays GTA-V. |
| Aug 20, 2023 | PSG4D🤖 is accepted as NeurIPS-23 Spotlight. |