MMMU（大规模多学科多模态理解与推理基准）是一个用于评估多模态模型在大规模多学科任务上表现的基准测试。MMMU包含从大学考试、测验和教材中精心收集的11.5K个多模态问题，涵盖六大核心学科：艺术与设计、商业、科学、健康与医学、人文与社会科学以及技术与工程。

MMMU
│
├── Basic_Medical_Science
│   ├── dev-00000-of-00001.parquet
│   ├── test-00000-of-00001.parquet
│   └── validation-00000-of-00001.parquet
│
├── Clinical_Medicine
│   ├── dev-00000-of-00001.parquet
│   ├── test-00000-of-00001.parquet
│   └── validation-00000-of-00001.parquet
│
├── Diagnostics_and_Laboratory_Medicine
│   ├── dev-00000-of-00001.parquet
│   ├── test-00000-of-00001.parquet
│   └── validation-00000-of-00001.parquet
│
├── Pharmacy
│   ├── dev-00000-of-00001.parquet
│   ├── test-00000-of-00001.parquet
│   └── validation-00000-of-00001.parquet
│
└── Public_Health
    ├── dev-00000-of-00001.parquet
    ├── test-00000-of-00001.parquet
    └── validation-00000-of-00001.parquet

@article{yue2023mmmu,
    title={MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI},
    author={Xiang Yue and Yuansheng Ni and Kai Zhang and Tianyu Zheng and Ruoqi Liu and Ge Zhang and Samuel Stevens and Dongfu Jiang and Weiming Ren and Yuxuan Sun and Cong Wei and Botao Yu and Ruibin Yuan and Renliang Sun and Ming Yin and Boyuan Zheng and Zhenzhu Yang and Yibo Liu and Wenhao Huang and Huan Sun and Yu Su and Wenhu Chen},
    journal={arXiv preprint arXiv:2311.16502},
    year={2023},
}

官方网站：
访问官网

下载链接：

相关论文：
查看论文

发布日期： 2023-11