Colorblind Bench

Can LLMs read a color vision test?

By Jonathan Chang

This post and benchmark were written and built with AI assistance using Codex and Claude Code.

Example

Each plate contains 23. The dots have matched brightness. Only their color defines the number. Color separation increases from left to right.

Color-dot plate with the number 23 at separation 0.040
Separation 0.040Open image ↗
Color-dot plate with the number 23 at separation 0.080
Separation 0.080Open image ↗
Color-dot plate with the number 23 at separation 0.120
Separation 0.120Open image ↗

Select an image to open the original 1024 × 1024 PNG.

How the images are made

  1. Place 900 dots in a circle. Set their positions, sizes, and brightness before applying a number.
  2. Use a two-digit mask to assign one color to dots inside the number and another color to the other dots.
  3. Match the brightness of the two colors. Change their separation across seven levels in two red–green color directions.
  4. Add grayscale copies and color blanks with no number. These controls should produce null.

The set has 140 color plates (10 numbers × 2 color directions × 7 separations), 100 grayscale copies, and 20 color blanks.

Control examples
Grayscale twin of a color plate, with no number visible
Grayscale twinSame dot geometry; color removed.
Colored dots with no intended number
Color blankColored dots; no number assigned.

Results

* Model released after this benchmark first went live on 10 September 2026. The mark follows the model release date, not the date its results were added.

Each result set has one saved response for every image. Both digits must match for a correct read. A number reported on a control is a false read.

Gemini 4 Argon * read 102/140 color plates correctly. Gemini 4 Argon * scored highest at the strongest color separation: 16/20.

Compare model scores

35 result sets

Correct number reads. Each level has 20 color plates; “All color” combines all 140. Color separation increases from left to right.

Ranked by all 140 color plates. Ties share a rank.

RankModel
11022121818181816
2816111413141112
374481215121112
455478871011
5470127111214
6460025111315
74202598810
840013581013
93610249812
103100058711
11260125666
12250034378
13200001478
14130003334
15120000156
1680021122
1770002212
=1860012120
=1862012010
2040000022
=2130002100
=2130001011
=2320000002
=2320000011
=2320001010
=2320000002
=2710000001
=2710100000
=2900000000
=2900000000
=2900000000
=2900000000
=2900000000
=2900000000
=2900000000

Bold marks the best score in each column. Equal scores share a rank. * Released after the first publication.

Inspect individual image answers

GPT-6 Astra

Saved results · low reasoning · original image detail

Color 81 / 140
Controls 120 / 120
All inputs 201 / 260

✓ Correct · × Incorrect · Cell labels show the selected model’s answer.

Open a thumbnail to inspect the exact image and saved answer. “none” means no number was reported.

Color plates · Protan

Rows show the expected number. Columns show color separation.

Number0.0060.0100.0160.0260.0400.0800.120
18Preview of plate color_protan_s006_n1818Preview of plate color_protan_s010_n1816Preview of plate color_protan_s016_n1818Preview of plate color_protan_s026_n1816Preview of plate color_protan_s040_n1818Preview of plate color_protan_s080_n1816Preview of plate color_protan_s120_n1818
23Preview of plate color_protan_s006_n2323Preview of plate color_protan_s010_n2323Preview of plate color_protan_s016_n2323Preview of plate color_protan_s026_n2323Preview of plate color_protan_s040_n2323Preview of plate color_protan_s080_n2323Preview of plate color_protan_s120_n2323
30Preview of plate color_protan_s006_n3020Preview of plate color_protan_s010_n3038Preview of plate color_protan_s016_n3038Preview of plate color_protan_s026_n3030Preview of plate color_protan_s040_n3030Preview of plate color_protan_s080_n3038Preview of plate color_protan_s120_n3038
36Preview of plate color_protan_s006_n3626Preview of plate color_protan_s010_n3638Preview of plate color_protan_s016_n3626Preview of plate color_protan_s026_n3638Preview of plate color_protan_s040_n3629Preview of plate color_protan_s080_n3636Preview of plate color_protan_s120_n3626
41Preview of plate color_protan_s006_n4141Preview of plate color_protan_s010_n4147Preview of plate color_protan_s016_n4141Preview of plate color_protan_s026_n4141Preview of plate color_protan_s040_n4141Preview of plate color_protan_s080_n4141Preview of plate color_protan_s120_n4141
54Preview of plate color_protan_s006_n5456Preview of plate color_protan_s010_n5456Preview of plate color_protan_s016_n5454Preview of plate color_protan_s026_n5454Preview of plate color_protan_s040_n5454Preview of plate color_protan_s080_n5454Preview of plate color_protan_s120_n5434
67Preview of plate color_protan_s006_n6767Preview of plate color_protan_s010_n6767Preview of plate color_protan_s016_n6767Preview of plate color_protan_s026_n6767Preview of plate color_protan_s040_n6767Preview of plate color_protan_s080_n6767Preview of plate color_protan_s120_n6767
79Preview of plate color_protan_s006_n7972Preview of plate color_protan_s010_n7979Preview of plate color_protan_s016_n7974Preview of plate color_protan_s026_n7979Preview of plate color_protan_s040_n7979Preview of plate color_protan_s080_n7979Preview of plate color_protan_s120_n7979
82Preview of plate color_protan_s006_n8242Preview of plate color_protan_s010_n8262Preview of plate color_protan_s016_n8262Preview of plate color_protan_s026_n8282Preview of plate color_protan_s040_n8262Preview of plate color_protan_s080_n8242Preview of plate color_protan_s120_n8282
95Preview of plate color_protan_s006_n95nonePreview of plate color_protan_s010_n9548Preview of plate color_protan_s016_n9595Preview of plate color_protan_s026_n9596Preview of plate color_protan_s040_n9596Preview of plate color_protan_s080_n9598Preview of plate color_protan_s120_n9593

Color plates · Deutan

Rows show the expected number. Columns show color separation.

Number0.0060.0100.0160.0260.0400.0800.120
18Preview of plate color_deutan_s006_n1832Preview of plate color_deutan_s010_n1818Preview of plate color_deutan_s016_n1818Preview of plate color_deutan_s026_n1816Preview of plate color_deutan_s040_n1818Preview of plate color_deutan_s080_n1818Preview of plate color_deutan_s120_n1816
23Preview of plate color_deutan_s006_n2323Preview of plate color_deutan_s010_n2323Preview of plate color_deutan_s016_n2323Preview of plate color_deutan_s026_n2323Preview of plate color_deutan_s040_n2323Preview of plate color_deutan_s080_n2323Preview of plate color_deutan_s120_n2323
30Preview of plate color_deutan_s006_n3038Preview of plate color_deutan_s010_n3030Preview of plate color_deutan_s016_n3038Preview of plate color_deutan_s026_n3038Preview of plate color_deutan_s040_n3030Preview of plate color_deutan_s080_n3038Preview of plate color_deutan_s120_n3030
36Preview of plate color_deutan_s006_n3626Preview of plate color_deutan_s010_n3636Preview of plate color_deutan_s016_n3636Preview of plate color_deutan_s026_n3626Preview of plate color_deutan_s040_n3636Preview of plate color_deutan_s080_n3629Preview of plate color_deutan_s120_n3629
41Preview of plate color_deutan_s006_n4147Preview of plate color_deutan_s010_n4141Preview of plate color_deutan_s016_n4141Preview of plate color_deutan_s026_n4141Preview of plate color_deutan_s040_n4141Preview of plate color_deutan_s080_n4141Preview of plate color_deutan_s120_n4141
54Preview of plate color_deutan_s006_n5458Preview of plate color_deutan_s010_n5424Preview of plate color_deutan_s016_n5454Preview of plate color_deutan_s026_n5454Preview of plate color_deutan_s040_n5434Preview of plate color_deutan_s080_n5458Preview of plate color_deutan_s120_n5454
67Preview of plate color_deutan_s006_n6727Preview of plate color_deutan_s010_n6767Preview of plate color_deutan_s016_n6767Preview of plate color_deutan_s026_n6767Preview of plate color_deutan_s040_n6767Preview of plate color_deutan_s080_n6767Preview of plate color_deutan_s120_n6767
79Preview of plate color_deutan_s006_n7979Preview of plate color_deutan_s010_n7979Preview of plate color_deutan_s016_n7979Preview of plate color_deutan_s026_n7979Preview of plate color_deutan_s040_n7979Preview of plate color_deutan_s080_n7979Preview of plate color_deutan_s120_n7979
82Preview of plate color_deutan_s006_n8262Preview of plate color_deutan_s010_n8282Preview of plate color_deutan_s016_n8282Preview of plate color_deutan_s026_n8282Preview of plate color_deutan_s040_n8262Preview of plate color_deutan_s080_n8222Preview of plate color_deutan_s120_n8262
95Preview of plate color_deutan_s006_n95nonePreview of plate color_deutan_s010_n9596Preview of plate color_deutan_s016_n9598Preview of plate color_deutan_s026_n9526Preview of plate color_deutan_s040_n9548Preview of plate color_deutan_s080_n9593Preview of plate color_deutan_s120_n9596

Grayscale controls · Protan

No number is expected. Row labels identify the source mask.

Source0.0060.0100.0160.0260.040
Mask 18Preview of plate gray_protan_s006_n18nonePreview of plate gray_protan_s010_n18nonePreview of plate gray_protan_s016_n18nonePreview of plate gray_protan_s026_n18nonePreview of plate gray_protan_s040_n18none
Mask 23Preview of plate gray_protan_s006_n23nonePreview of plate gray_protan_s010_n23nonePreview of plate gray_protan_s016_n23nonePreview of plate gray_protan_s026_n23nonePreview of plate gray_protan_s040_n23none
Mask 30Preview of plate gray_protan_s006_n30nonePreview of plate gray_protan_s010_n30nonePreview of plate gray_protan_s016_n30nonePreview of plate gray_protan_s026_n30nonePreview of plate gray_protan_s040_n30none
Mask 36Preview of plate gray_protan_s006_n36nonePreview of plate gray_protan_s010_n36nonePreview of plate gray_protan_s016_n36nonePreview of plate gray_protan_s026_n36nonePreview of plate gray_protan_s040_n36none
Mask 41Preview of plate gray_protan_s006_n41nonePreview of plate gray_protan_s010_n41nonePreview of plate gray_protan_s016_n41nonePreview of plate gray_protan_s026_n41nonePreview of plate gray_protan_s040_n41none
Mask 54Preview of plate gray_protan_s006_n54nonePreview of plate gray_protan_s010_n54nonePreview of plate gray_protan_s016_n54nonePreview of plate gray_protan_s026_n54nonePreview of plate gray_protan_s040_n54none
Mask 67Preview of plate gray_protan_s006_n67nonePreview of plate gray_protan_s010_n67nonePreview of plate gray_protan_s016_n67nonePreview of plate gray_protan_s026_n67nonePreview of plate gray_protan_s040_n67none
Mask 79Preview of plate gray_protan_s006_n79nonePreview of plate gray_protan_s010_n79nonePreview of plate gray_protan_s016_n79nonePreview of plate gray_protan_s026_n79nonePreview of plate gray_protan_s040_n79none
Mask 82Preview of plate gray_protan_s006_n82nonePreview of plate gray_protan_s010_n82nonePreview of plate gray_protan_s016_n82nonePreview of plate gray_protan_s026_n82nonePreview of plate gray_protan_s040_n82none
Mask 95Preview of plate gray_protan_s006_n95nonePreview of plate gray_protan_s010_n95nonePreview of plate gray_protan_s016_n95nonePreview of plate gray_protan_s026_n95nonePreview of plate gray_protan_s040_n95none

Grayscale controls · Deutan

No number is expected. Row labels identify the source mask.

Source0.0060.0100.0160.0260.040
Mask 18Preview of plate gray_deutan_s006_n18nonePreview of plate gray_deutan_s010_n18nonePreview of plate gray_deutan_s016_n18nonePreview of plate gray_deutan_s026_n18nonePreview of plate gray_deutan_s040_n18none
Mask 23Preview of plate gray_deutan_s006_n23nonePreview of plate gray_deutan_s010_n23nonePreview of plate gray_deutan_s016_n23nonePreview of plate gray_deutan_s026_n23nonePreview of plate gray_deutan_s040_n23none
Mask 30Preview of plate gray_deutan_s006_n30nonePreview of plate gray_deutan_s010_n30nonePreview of plate gray_deutan_s016_n30nonePreview of plate gray_deutan_s026_n30nonePreview of plate gray_deutan_s040_n30none
Mask 36Preview of plate gray_deutan_s006_n36nonePreview of plate gray_deutan_s010_n36nonePreview of plate gray_deutan_s016_n36nonePreview of plate gray_deutan_s026_n36nonePreview of plate gray_deutan_s040_n36none
Mask 41Preview of plate gray_deutan_s006_n41nonePreview of plate gray_deutan_s010_n41nonePreview of plate gray_deutan_s016_n41nonePreview of plate gray_deutan_s026_n41nonePreview of plate gray_deutan_s040_n41none
Mask 54Preview of plate gray_deutan_s006_n54nonePreview of plate gray_deutan_s010_n54nonePreview of plate gray_deutan_s016_n54nonePreview of plate gray_deutan_s026_n54nonePreview of plate gray_deutan_s040_n54none
Mask 67Preview of plate gray_deutan_s006_n67nonePreview of plate gray_deutan_s010_n67nonePreview of plate gray_deutan_s016_n67nonePreview of plate gray_deutan_s026_n67nonePreview of plate gray_deutan_s040_n67none
Mask 79Preview of plate gray_deutan_s006_n79nonePreview of plate gray_deutan_s010_n79nonePreview of plate gray_deutan_s016_n79nonePreview of plate gray_deutan_s026_n79nonePreview of plate gray_deutan_s040_n79none
Mask 82Preview of plate gray_deutan_s006_n82nonePreview of plate gray_deutan_s010_n82nonePreview of plate gray_deutan_s016_n82nonePreview of plate gray_deutan_s026_n82nonePreview of plate gray_deutan_s040_n82none
Mask 95Preview of plate gray_deutan_s006_n95nonePreview of plate gray_deutan_s010_n95nonePreview of plate gray_deutan_s016_n95nonePreview of plate gray_deutan_s026_n95nonePreview of plate gray_deutan_s040_n95none

Color blanks

No number is expected in any blank.

Direction12345678910
ProtanPreview of plate blank_01nonePreview of plate blank_03nonePreview of plate blank_05nonePreview of plate blank_07nonePreview of plate blank_09nonePreview of plate blank_11nonePreview of plate blank_13nonePreview of plate blank_15nonePreview of plate blank_17nonePreview of plate blank_19none
DeutanPreview of plate blank_02nonePreview of plate blank_04nonePreview of plate blank_06nonePreview of plate blank_08nonePreview of plate blank_10nonePreview of plate blank_12nonePreview of plate blank_14nonePreview of plate blank_16nonePreview of plate blank_18nonePreview of plate blank_20none

Original plate

Original PNG · 1024 × 1024. Arrow keys change plates. At 1:1, focus the image area to pan with arrow keys.

Download PNG
Model
Expected answer
Model answer
Correctness
Input ID
Family
Axis / separation

Saved final response









      

      Model settings and secondary metrics
      
Expand a model for request settings, digit accuracy, and control errors.

      
      

1Gemini 4 Argon *All color plates102/140Separation 0.12016/20False reads0/120
102/140 exact · 25 wrong numbers · 13 null answers · 0 invalid
Model ID: gemini-4-argon
Thinking level: low · Image detail: native · Output limit: 2048 tokens · Output format: JSON schema
All seven color separations
Score0.0060.0100.0160.0260.0400.0800.120
Exact10%60%90%90%90%90%80%
Digits15%70%95%95%95%95%90%
Numeric35%100%100%100%100%100%100%
Correct control rejections: 100% grayscale; 100% color blanks.


2GPT-6 AstraAll color plates81/140Separation 0.12012/20False reads0/120
81/140 exact · 57 wrong numbers · 2 null answers · 0 invalid
Model ID: gpt-6-astra
Reasoning: low · Image detail: original · Output limit: default
All seven color separations
Score0.0060.0100.0160.0260.0400.0800.120
Exact30%55%70%65%70%55%60%
Digits57.5%75%85%80%80%75%77.5%
Numeric90%100%100%100%100%100%100%
Correct control rejections: 100% grayscale; 100% color blanks.


3Claude Opus 5.5 *All color plates74/140Separation 0.12012/20False reads0/120
74/140 exact · 66 wrong numbers · 0 null answers · 0 invalid
Model ID: claude-opus-5-5
Thinking: adaptive · Effort: low · Image detail: native · Output limit: 8192 tokens
All seven color separations
Score0.0060.0100.0160.0260.0400.0800.120
Exact20%40%60%75%60%55%60%
Digits35%57.5%72.5%82.5%70%65%75%
Numeric100%100%100%100%100%100%100%
Correct control rejections: 100% grayscale; 100% color blanks.


4GPT-6.1 Sol *All color plates55/140Separation 0.12011/20False reads0/120
55/140 exact · 85 wrong numbers · 0 null answers · 0 invalid
Model ID: gpt-6.1-sol
Reasoning: low · Image detail: original · Output limit: default
All seven color separations
Score0.0060.0100.0160.0260.0400.0800.120
Exact20%35%40%40%35%50%55%
Digits37.5%50%62.5%60%55%57.5%75%
Numeric100%100%100%100%100%100%100%
Correct control rejections: 100% grayscale; 100% color blanks.


5GPT-4.1All color plates47/140Separation 0.12014/20False reads0/120
47/140 exact · 63 wrong numbers · 30 null answers · 0 invalid
Model ID: gpt-4.1-2025-04-14
Reasoning: not set · Image detail: high · Temperature: 0 · Output limit: 60 tokens
All seven color separations
Score0.0060.0100.0160.0260.0400.0800.120
Exact0%5%10%35%55%60%70%
Digits0%12.5%50%67.5%77.5%80%85%
Numeric5%45%100%100%100%100%100%
Correct control rejections: 100% grayscale; 100% color blanks.


6GPT-4.1-miniAll color plates46/140Separation 0.12015/20False reads0/120
46/140 exact · 90 wrong numbers · 4 null answers · 0 invalid
Model ID: gpt-4.1-mini-2025-04-14
Core plate settings: Reasoning: not set · Image detail: high · Temperature: 0 · Output limit: 60 tokens
All seven color separations
Score0.0060.0100.0160.0260.0400.0800.120
Exact0%0%10%25%55%65%75%
Digits12.5%32.5%40%62.5%75%77.5%82.5%
Numeric80%100%100%100%100%100%100%
Correct control rejections: 100% grayscale; 100% color blanks.


7Gemini 3.8 FlashAll color plates42/140Separation 0.12010/20False reads0/120
42/140 exact · 79 wrong numbers · 19 null answers · 0 invalid
Model ID: gemini-3.8-flash
Thinking level: low · Image detail: native · Output limit: 2048 tokens · Output format: JSON schema
All seven color separations
Score0.0060.0100.0160.0260.0400.0800.120
Exact0%10%25%45%40%40%50%
Digits0%30%52.5%70%67.5%67.5%72.5%
Numeric5%100%100%100%100%100%100%
Correct control rejections: 100% grayscale; 100% color blanks.


8Gemini 3.6 FlashAll color plates40/140Separation 0.12013/20False reads1/120
40/140 exact · 82 wrong numbers · 18 null answers · 0 invalid
Model ID: gemini-3.6-flash
Thinking level: minimal · Image detail: native · Output limit: 2048 tokens · Output format: JSON schema
All seven color separations
Score0.0060.0100.0160.0260.0400.0800.120
Exact0%5%15%25%40%50%65%
Digits0%27.5%42.5%62.5%67.5%75%82.5%
Numeric15%95%100%100%100%100%100%
Correct control rejections: 100% grayscale; 95% color blanks.
False numbers on controls
AnswerColor blanksGrayscale
8110


9Gemini 3.7 FlashAll color plates36/140Separation 0.12012/20False reads0/120
36/140 exact · 90 wrong numbers · 14 null answers · 0 invalid
Model ID: gemini-3.7-flash
Thinking level: low · Image detail: native · Output limit: 2048 tokens · Output format: JSON schema
All seven color separations
Score0.0060.0100.0160.0260.0400.0800.120
Exact5%0%10%20%45%40%60%
Digits10%30%42.5%55%72.5%65%77.5%
Numeric30%100%100%100%100%100%100%
Correct control rejections: 100% grayscale; 100% color blanks.


10Gemini 3.1 Pro PreviewAll color plates31/140Separation 0.12011/20False reads1/120
31/140 exact · 88 wrong numbers · 17 null answers · 4 invalid
Model ID: gemini-3.1-pro-preview
Core plate settings: Thinking level: low · Image detail: native · Output limit: 2048 tokens
All seven color separations
Score0.0060.0100.0160.0260.0400.0800.120
Exact0%0%0%25%40%35%55%
Digits0%25%45%57.5%70%57.5%77.5%
Numeric15%100%100%95%100%85%100%
Correct control rejections: 100% grayscale; 95% color blanks.
False numbers on controls
AnswerColor blanksGrayscale
410


11Gemini 3.5 FlashAll color plates26/140Separation 0.1206/20False reads15/120
26/140 exact · 104 wrong numbers · 10 null answers · 0 invalid
Model ID: gemini-3.5-flash
Core plate settings: Thinking level: low · Image detail: native · Output limit: 2048 tokens
All seven color separations
Score0.0060.0100.0160.0260.0400.0800.120
Exact0%5%10%25%30%30%30%
Digits5%22.5%37.5%55%57.5%62.5%65%
Numeric50%100%100%100%100%100%100%
Correct control rejections: 100% grayscale; 25% color blanks.
False numbers on controls
AnswerColor blanksGrayscale
5250
5350
530
2520


12Gemini 3 Flash PreviewAll color plates25/140Separation 0.1208/20False reads22/120
25/140 exact · 103 wrong numbers · 12 null answers · 0 invalid
Model ID: gemini-3-flash-preview
Core plate settings: Thinking level: low · Image detail: native · Output limit: 2048 tokens
All seven color separations
Score0.0060.0100.0160.0260.0400.0800.120
Exact0%0%15%20%15%35%40%
Digits7.5%27.5%42.5%55%55%65%70%
Numeric40%100%100%100%100%100%100%
Correct control rejections: 92% grayscale; 30% color blanks.
False numbers on controls
AnswerColor blanksGrayscale
4712
5221
1520
301
510
1201
2010
2110
2410
2610
2801
3810
4201
6010
7401
8110
8610


13GPT-5 miniAll color plates20/140Separation 0.1208/20False reads2/120
20/140 exact · 99 wrong numbers · 21 null answers · 0 invalid
Model ID: gpt-5-mini
Core plate settings: Reasoning: minimal · Image detail: high · Output limit: 1000 tokens
All seven color separations
Score0.0060.0100.0160.0260.0400.0800.120
Exact0%0%0%5%20%35%40%
Digits0%22.5%25%32.5%45%62.5%57.5%
Numeric5%90%100%100%100%100%100%
Correct control rejections: 99% grayscale; 95% color blanks.
False numbers on controls
AnswerColor blanksGrayscale
1211


14Gemini 3.1 Flash-LiteAll color plates13/140Separation 0.1204/20False reads0/120
13/140 exact · 105 wrong numbers · 22 null answers · 0 invalid
Model ID: gemini-3.1-flash-lite
Core plate settings: Thinking level: low · Image detail: native · Output limit: 2048 tokens
All seven color separations
Score0.0060.0100.0160.0260.0400.0800.120
Exact0%0%0%15%15%15%20%
Digits0%17.5%25%52.5%47.5%52.5%47.5%
Numeric0%90%100%100%100%100%100%
Correct control rejections: 100% grayscale; 100% color blanks.


15GPT-5.1-Codex miniAll color plates12/140Separation 0.1206/20False reads0/120
12/140 exact · 76 wrong numbers · 52 null answers · 0 invalid
Model ID: gpt-5.1-codex-mini
Core plate settings: Reasoning: low · Image detail: high · Output limit: 4000 tokens
All seven color separations
Score0.0060.0100.0160.0260.0400.0800.120
Exact0%0%0%0%5%25%30%
Digits0%2.5%15%25%25%47.5%57.5%
Numeric0%20%70%70%85%100%95%
Correct control rejections: 100% grayscale; 100% color blanks.


16Claude Opus 5All color plates8/140Separation 0.1202/20False reads0/120
8/140 exact · 93 wrong numbers · 39 null answers · 0 invalid
Model ID: claude-opus-5
Thinking: disabled · Effort: low · Image detail: native · Output limit: 8192 tokens
All seven color separations
Score0.0060.0100.0160.0260.0400.0800.120
Exact0%0%10%5%5%10%10%
Digits0%0%17.5%20%35%25%25%
Numeric0%15%90%100%100%100%100%
Correct control rejections: 100% grayscale; 100% color blanks.


17Gemini 3.5 Flash-LiteAll color plates7/140Separation 0.1202/20False reads0/120
7/140 exact · 107 wrong numbers · 26 null answers · 0 invalid
Model ID: gemini-3.5-flash-lite
Thinking level: minimal · Image detail: native · Output limit: 2048 tokens · Output format: JSON schema
All seven color separations
Score0.0060.0100.0160.0260.0400.0800.120
Exact0%0%0%10%10%5%10%
Digits0%17.5%25%45%45%37.5%50%
Numeric0%70%100%100%100%100%100%
Correct control rejections: 100% grayscale; 100% color blanks.


18GPT-5.6 SolAll color plates6/140Separation 0.1200/20False reads0/120
6/140 exact · 134 wrong numbers · 0 null answers · 0 invalid
Model ID: gpt-5.6-sol
Reasoning: low · Image detail: original · Output limit: default
All seven color separations
Score0.0060.0100.0160.0260.0400.0800.120
Exact0%0%5%10%5%10%0%
Digits12.5%12.5%17.5%22.5%25%35%15%
Numeric100%100%100%100%100%100%100%
Correct control rejections: 100% grayscale; 100% color blanks.
Other saved results
Daybreak → Sol · 6/140
6/140 exact · 134 wrong numbers · 0 null answers · 0 invalid
Model ID: gpt-daybreak-blue-latest
Reasoning: low · Image detail: original · Output limit: default
Returned model: gpt-5.6-sol
All seven color separations
Score0.0060.0100.0160.0260.0400.0800.120
Exact10%0%5%10%0%5%0%
Digits25%17.5%17.5%17.5%20%25%25%
Numeric100%100%100%100%100%100%100%
Correct control rejections: 100% grayscale; 95% color blanks.
False numbers on controls
AnswerColor blanksGrayscale
5610


19GPT-4.1 nanoAll color plates4/140Separation 0.1202/20False reads0/120
4/140 exact · 65 wrong numbers · 71 null answers · 0 invalid
Model ID: gpt-4.1-nano-2025-04-14
Reasoning: not set · Image detail: high · Temperature: 0 · Output limit: 60 tokens
All seven color separations
Score0.0060.0100.0160.0260.0400.0800.120
Exact0%0%0%0%0%10%10%
Digits0%0%0%10%12.5%25%20%
Numeric0%0%35%75%70%75%90%
Correct control rejections: 100% grayscale; 100% color blanks.


20Claude Sonnet 5.5 *All color plates3/140Separation 0.1201/20False reads0/120
3/140 exact · 70 wrong numbers · 67 null answers · 0 invalid
Model ID: claude-sonnet-5-5
Thinking: between_tools · Effort: low · Image detail: native · Output limit: 8192 tokens
All seven color separations
Score0.0060.0100.0160.0260.0400.0800.120
Exact0%0%0%5%0%5%5%
Digits0%7.5%10%22.5%20%20%27.5%
Numeric0%40%50%65%60%65%85%
Correct control rejections: 100% grayscale; 100% color blanks.


20Claude Fable 5.1All color plates3/140Separation 0.1200/20False reads0/120
3/140 exact · 116 wrong numbers · 21 null answers · 0 invalid
Model ID: claude-fable-5-1
Thinking: adaptive · Effort: low · Image detail: native · Output limit: 8192 tokens
All seven color separations
Score0.0060.0100.0160.0260.0400.0800.120
Exact0%0%0%10%5%0%0%
Digits0%15%15%40%35%20%27.5%
Numeric0%95%100%100%100%100%100%
Correct control rejections: 100% grayscale; 100% color blanks.


22GPT-5.4All color plates2/140Separation 0.1202/20False reads0/120
2/140 exact · 134 wrong numbers · 4 null answers · 0 invalid
Model ID: gpt-5.4
Core plate settings: Reasoning: not set · Image detail: original · Output limit: 1000 tokens
All seven color separations
Score0.0060.0100.0160.0260.0400.0800.120
Exact0%0%0%0%0%0%10%
Digits5%17.5%17.5%12.5%15%17.5%20%
Numeric80%100%100%100%100%100%100%
Correct control rejections: 100% grayscale; 100% color blanks.


22Muse Wonder *All color plates2/140Separation 0.1202/20False reads0/120
2/140 exact · 115 wrong numbers · 23 null answers · 0 invalid
Model ID: meta/muse-wonder
Reasoning: minimal · Image detail: high · Output limit: 2048 tokens · Output format: JSON schema
Returned model: meta/muse-wonder-20261012, meta/muse-wonder
All seven color separations
Score0.0060.0100.0160.0260.0400.0800.120
Exact0%0%0%0%0%0%10%
Digits0%0%12.5%15%12.5%20%20%
Numeric0%85%100%100%100%100%100%
Correct control rejections: 100% grayscale; 100% color blanks.


22GPT-5.4 miniAll color plates2/140Separation 0.1201/20False reads0/120
2/140 exact · 134 wrong numbers · 4 null answers · 0 invalid
Model ID: gpt-5.4-mini
Core plate settings: Reasoning: none · Image detail: high · Output limit: 1000 tokens
All seven color separations
Score0.0060.0100.0160.0260.0400.0800.120
Exact0%0%0%0%0%5%5%
Digits12.5%10%12.5%12.5%12.5%35%30%
Numeric90%95%100%95%100%100%100%
Correct control rejections: 100% grayscale; 100% color blanks.


22GPT-5.6 LunaAll color plates2/140Separation 0.1200/20False reads0/120
2/140 exact · 77 wrong numbers · 61 null answers · 0 invalid
Model ID: gpt-5.6-luna
Reasoning: low · Image detail: original · Output limit: default
All seven color separations
Score0.0060.0100.0160.0260.0400.0800.120
Exact0%0%0%5%0%5%0%
Digits0%2.5%5%12.5%5%10%7.5%
Numeric0%30%55%75%60%85%90%
Correct control rejections: 100% grayscale; 100% color blanks.


26DeepSeek V4.1 FlashAll color plates1/140Separation 0.1201/20False reads0/120
1/140 exact · 85 wrong numbers · 54 null answers · 0 invalid
Model ID: deepseek/deepseek-v4.1-flash
Reasoning: disabled · Image detail: high · Output limit: 2048 tokens · Output format: JSON schema
All seven color separations
Score0.0060.0100.0160.0260.0400.0800.120
Exact0%0%0%0%0%0%5%
Digits0%2.5%10%7.5%7.5%17.5%7.5%
Numeric5%5%45%85%95%100%95%
Correct control rejections: 100% grayscale; 100% color blanks.


26GPT-5.6 TerraAll color plates1/140Separation 0.1200/20False reads0/120
1/140 exact · 136 wrong numbers · 3 null answers · 0 invalid
Model ID: gpt-5.6-terra
Reasoning: low · Image detail: original · Output limit: default
All seven color separations
Score0.0060.0100.0160.0260.0400.0800.120
Exact0%5%0%0%0%0%0%
Digits10%12.5%2.5%10%7.5%15%15%
Numeric85%100%100%100%100%100%100%
Correct control rejections: 100% grayscale; 100% color blanks.


28Claude Haiku 5.5 *All color plates0/140Separation 0.1200/20False reads0/120
0/140 exact · 8 wrong numbers · 132 null answers · 0 invalid
Model ID: claude-haiku-5-5
Thinking: disabled · Effort: low · Image detail: native · Output limit: 8192 tokens
All seven color separations
Score0.0060.0100.0160.0260.0400.0800.120
Exact0%0%0%0%0%0%0%
Digits0%0%0%0%0%2.5%0%
Numeric0%0%0%10%10%10%10%
Correct control rejections: 100% grayscale; 100% color blanks.


28Claude Sonnet 4.5All color plates0/140Separation 0.1200/20False reads0/120
0/140 exact · 54 wrong numbers · 86 null answers · 0 invalid
Model ID: claude-sonnet-4-5-20250929
Core plate settings: Reasoning: not set · Image detail: native · Output limit: 64 tokens
All seven color separations
Score0.0060.0100.0160.0260.0400.0800.120
Exact0%0%0%0%0%0%0%
Digits0%0%0%2.5%2.5%10%15%
Numeric0%0%5%35%60%90%80%
Correct control rejections: 100% grayscale; 100% color blanks.


28Claude Sonnet 5All color plates0/140Separation 0.1200/20False reads0/120
0/140 exact · 9 wrong numbers · 131 null answers · 0 invalid
Model ID: claude-sonnet-5
Core plate settings: Reasoning: disabled · Image detail: native · Output limit: 64 tokens
All seven color separations
Score0.0060.0100.0160.0260.0400.0800.120
Exact0%0%0%0%0%0%0%
Digits0%0%0%0%5%12.5%2.5%
Numeric0%0%0%0%10%25%10%
Correct control rejections: 100% grayscale; 100% color blanks.


28GPT-6 Luna *All color plates0/140Separation 0.1200/20False reads0/120
0/140 exact · 55 wrong numbers · 85 null answers · 0 invalid
Model ID: gpt-6-luna
Reasoning: none · Image detail: original · Output limit: default
All seven color separations
Score0.0060.0100.0160.0260.0400.0800.120
Exact0%0%0%0%0%0%0%
Digits0%2.5%5%5%0%10%7.5%
Numeric0%25%30%30%45%70%75%
Correct control rejections: 100% grayscale; 100% color blanks.


28GPT-6 Sol *All color plates0/140Separation 0.1200/20False reads0/120
0/140 exact · 101 wrong numbers · 39 null answers · 0 invalid
Model ID: gpt-6-sol
Reasoning: none · Image detail: original · Output limit: default
All seven color separations
Score0.0060.0100.0160.0260.0400.0800.120
Exact0%0%0%0%0%0%0%
Digits0%5%2.5%10%15%7.5%10%
Numeric10%70%80%95%90%80%80%
Correct control rejections: 100% grayscale; 100% color blanks.


28Gemini 2.5 FlashAll color plates0/140Separation 0.1200/20False reads0/120
0/140 exact · 67 wrong numbers · 73 null answers · 0 invalid
Model ID: gemini-2.5-flash
Core plate settings: Thinking budget: 0 · Image detail: native · Output limit: 2048 tokens
All seven color separations
Score0.0060.0100.0160.0260.0400.0800.120
Exact0%0%0%0%0%0%0%
Digits0%0%2.5%0%2.5%17.5%22.5%
Numeric0%0%30%40%80%90%95%
Correct control rejections: 100% grayscale; 100% color blanks.


28Kimi K3All color plates0/140Separation 0.1200/20False reads0/120
0/140 exact · 59 wrong numbers · 81 null answers · 0 invalid
Model ID: moonshotai/kimi-k3
Reasoning: disabled · Image detail: high · Output limit: 2048 tokens · Output format: JSON schema
All seven color separations
Score0.0060.0100.0160.0260.0400.0800.120
Exact0%0%0%0%0%0%0%
Digits0%0%0%5%5%5%10%
Numeric0%0%15%45%65%70%100%
Correct control rejections: 100% grayscale; 100% color blanks.


      
Each row shows one result set. Other saved results are inside that model’s details.

      

    


    

      

        

          Prompt and scoring
          


Inspect this newly generated pseudoisochromatic color-dot plate. Report the one- or two-digit Arabic number only if it is actually visible. Do not infer an answer from familiar clinical color-vision plates and do not force a guess. Return only JSON as {"number":"NN"}; if no numeral is visible, return {"number":null}.

            
In the score details, Exact requires both digits to match. Digits counts correct digit positions. Numeric counts number responses. Invalid answers count as wrong.

            
Color separation is measured in CIE u′v′ coordinates. The two directions are called protan and deutan. The number mask uses Atkinson Hyperlegible.