A comprehensive benchmark for ophthalmological knowledge and reasoning
What is BELO?
BEnchmarking LLMs in Ophthalmology
Ophthalmology
900 MCQs
Expert Reasoning
BELO is a benchmark for evaluating LLMs in ophthalmology. It features 900 carefully curated multiple-choice questions, each paired with answers and expert reasoning—sourced from board exams, ophthalmic literature, and biomedical research—to ensure both scientific depth and clinical relevance.
BELO Template
| Field | Value |
|---|---|
| Context | You are an assistant specializing in ophthalmology. |
| Task | Please select the correct option from the given choices for the following multiple-choice question related to ophthalmology. The number of options may vary, with some questions having two options (A/B), three options (A/B/C), or four options (A/B/C/D). After selecting the correct option, provide a detailed explanation that includes the reasoning behind your choice, supported by relevant ophthalmological knowledge and information. |
| Input | Question and options |
| Output format | Please format your response in JSON as shown below: { "Correct answer": "A", "Reasoning": "[Provide reasoning that clearly explains why option A is the correct choice, including relevant ophthalmological details and knowledge.]" } Question: Options: Please provide your response in the JSON format shown above: |
Macro F1 Over Time
Best Scores Per Metric
Leaderboard
Rank | Model | Macro F1 | Accuracy | ROUGE-L | BERT Score | BART Score | METEOR | Align |
|---|---|---|---|---|---|---|---|---|
 GPT-5Aug 7th 2025 OpenAI | 90.80 | 90.40 | 19.20 | 72.00 | -3.21 | 23.70 | 22.90 | |
| OpenAI o1 Sep 12th 2024 OpenAI | 89.00 | 88.20 | 18.60 | 71.20 | -3.29 | 24.70 | 23.60 | |
 DeepSeek-R1Jan 22nd 2025 DeepSeek | 88.40 | 87.60 | 18.60 | 71.10 | -3.31 | 22.50 | 20.90 | |
#4 | OpenAI o3-mini Jan 31st 2025 OpenAI | 85.80 | 85.60 | 20.30 | 71.80 | -3.36 | 22.80 | 25.20 |
#5 | OpenAI GPT-4o May 13th 2024 OpenAI | 83.50 | 83.10 | 20.40 | 71.30 | -3.61 | 22.70 | 25.50 |
#6 |  LLaMA 3 8B InstructJul 31st 2024 Meta | 74.40 | 74.10 | 18.70 | 70.30 | -3.68 | 22.40 | 20.10 |
#7 |  Gemini 1.5Feb 15th 2024 Google | 63.90 | 59.60 | 16.30 | 69.00 | -3.63 | 22.50 | 19.70 |
#8 | MedGemma May 20th 2025 Google | 54.60 | 55.00 | 18.80 | 71.10 | -3.41 | 23.20 | 21.20 |