MULTILINGUAL · MULTIMODAL · STRUCTURED

Read the structure.
Across languages.

MInfoVisQA: A Multilingual Benchmark for Information-Rich Visual Question Answering

MBZUAI Georgia Tech The University of Tokyo University of Illinois Urbana-Champaign

128 seed questions across 24 languages. How well do multimodal models understand the same charts and tables when the visual language changes?

Open data & code / Image-based evaluation / 24 languages
THE PAPER IN 25 SECONDS
GPT-5.6-Luna answers the same English question about two language versions of a chart. The reference answer is Winter in both.
24languages
32observed visual categories
3,072localized images
8,960QA configurations
01 / THE BENCHMARK

Visual reasoning, without
the language shortcut.

Parallel questions and visuals preserve the underlying quantities and structure, making language a deliberate part of the evaluation.

01

Charts + visual tables

From grouped bars and multi-series lines to heatmaps and diagrams. Tables distinguish simple, row-spanning, column-spanning and mixed-spanning structures.

Explore visual categories ↗
02

Aligned + cross-language QA

Evaluate each language with its own visual, question and answer. Then ask in Chinese or English about visuals in each of the other 23 languages.

Understand the protocol ↓
03

Traceable construction

Source-linked questions, labels and QA translated with Google Translate API, reproducible renderers and image hashes. Charts and tables enter the model as images, preserving the visual task.

See the pipeline ↗
24 LANGUAGES

One shared evaluation framework.

02 / MODEL PERFORMANCE

Model performance

Updated 19 November 2026
26 models · Semantic accuracy (%) · higher is better
Download CSV ↓
Loading results…
MInfoVisQA model accuracy, sortable by each metric. Click a model for provenance and evaluation notes.

XQA-ZH / EN: 23 cross-language settings each. LQA mean: mean of the 24 displayed same-language scores. AVG: macro-average of all 70 configurations.

Results JSON ↗
Score provenance & how to read the leaderboard

All 26 models and 702 score cells are synchronized with the current manuscript main table. The table uses exact matching followed by text-only GPT-6 Astra judging for non-matches. Scores are author-reported; this website does not independently rerun the evaluations. View the frozen source table.

Reported scores and the original AVG are preserved at one decimal. AVG weights all 70 configurations; it is not the mean of the 26 visible XQA/LQA columns. LQA mean is computed from the 24 displayed language scores. Model grouping follows the paper; open weights does not imply these runs were self-hosted. Official model links are shown where previously documented.

Ranks use the reported overall AVG; tied scores share a rank. Language variants share the same seed question, so 8,960 configurations do not represent 8,960 independent questions.

03 / INSIDE THE DATA

Same evidence.
A new perspective.

Explore actual localized examples from the released dataset. Switch the visual language and question setting to see what the model receives.

English localized grouped bar chart from ChartQAReleased image · click to view at full resolution ↗
LQA · EN → ENQUESTION

Reveal reference answer

Examples are copied without modification from the current release. Source attribution and original identifiers are retained; source-specific dataset terms apply.

04 / EVALUATION DESIGN

70 configurations.
One consistent protocol.

For each base QA, the visual language changes while the task stays linked to the same source. Question and answer always share a language.

The language configuration map

24 × 24

Visual language →

Question & answer language →
LQA · 24XQA-ZH · 23XQA-EN · 23
1

Read the visual

Each input contains a localized image and question. Original external prose, when required, is translated into the question language using Google Translate API. Tables are never replaced by text transcriptions.

2

Match, then judge

Exact answer matches are accepted. Other completed answers go to a text-only equivalence judge. The judge receives no image; unconfirmed equivalence counts as incorrect.

3

Aggregate every configuration

Each XQA column averages 23 visual languages. Each LQA column evaluates a single aligned language. AVG gives equal weight to all 70 configurations.

AVG = (23 × XQAZH + 23 × XQAEN + Σ LQAℓ) / 70
Source QA→Recover structure→Google Translate API→Render→Review & export
BUILT FROM SIX DATASETS
CharXiv 48ChartQAPro 27TableVQA-Bench 22ChartQA 12Visual-TableQA 10MMTU 9

128 base questions × 70 configurations = 8,960 QA instances. Language variants share their base identity and are not independent source observations. Automated construction and review do not constitute complete human verification. See the dataset card for scope, quality flags and limitations.

05 / BUILD ON MINFOVISQA

Start with the data.
Make results reproducible.

Download the frozen benchmark, evaluate your model and share scores with the configuration needed to reproduce them.

CONTRIBUTE A RESULT

Bring your model to the leaderboard.

Include the exact model identifier, dataset revision, inference settings, judge configuration, coverage and failure counts, plus all 70 configuration scores.

Submit results on GitHub Leaderboard maintenance guide ↗

Citation

If you use MInfoVisQA in your research, please cite our preprint.

@article{jiang2026minfovisqamultilingualbenchmarkinformationrich,
  title={MInfoVisQA: A Multilingual Benchmark for Information-Rich Visual Question Answering},
  author={Jiang, Changjiang and Zhou, Jiang and Yuan, Xinbin and Zhang, Fan and Feng, Tao and Xie, Zhuohan},
  journal={arXiv preprint arXiv:2611.21644},
  year={2026}
}
EVALUATION RECORD