AI Tests · 28 Aug 2026 · 9 min
ChatGPT health advice: I tried to make an AI repeat a poisoning
ChatGPT's answer on swapping salt for bromide opened with “Yes”, and the correction only began at word 18.
Which AI is most accurate? · Run 2
Claude aced run 2
Six AI apps, same questions, every answer marked against the official source.
Run 2 · Sun 11 Oct 2026 · free plans · right, of 30 tries each
Six apps, head to head
Claudefree30/30
Copilotfree29/30
Geminifree28/30
Perplexityfree25/30
ChatGPTfree23/30
Grokfree6/30
141 right21 wrong18 no answerof 180 tries
Claude got all 30 right. Claude, Copilot and Gemini are within two answers of the top. The tracker is not a ranking. Grok’s dashed stretch is 18 tries that came back empty, not wrong answers. Each empty try was sent again; 31 of the 39 attempts that failed ended on Grok’s own error after about ten minutes of working.
Free letter
Confidently Wrong, every fortnight
Which AI told the truth this month, with the receipts, plus an AI story worth retelling in between.
Free. Leave whenever you like.
Guess first · question 5 · open question
Which planet has the most moons?
Guess first, then see what the six apps said.
Show the answer
Saturn, with 293 confirmed moons (JPL and NASA, Aug 2026)
NASA’s own page summary still said 274. An app had to read past it. 15 of 18 tries got it right in run 2.
Claude
Copilot
Gemini
Perplexity
ChatGPT
Grok
The run grid · every try, 180 of them
Question 8 (Clocks) was the hardest
RightWrongNo answer
| Question | Claude | Copilot | Gemini | Perplexity | ChatGPT | Grok | Right |
|---|---|---|---|---|---|---|---|
| 1Holidays | 15/18 | ||||||
| 2E-scooter | 16/18 | ||||||
| 3Eviction | 16/18 | ||||||
| 4Paternity | 15/18 | ||||||
| 5Moons | 15/18 | ||||||
| 6ETIAS | 15/18 | ||||||
| 7Footwear | 9/18 | ||||||
| 8Clocks | 7/18hardest | ||||||
| 9Towing | 16/18 | ||||||
| 10Shakespeare | 17/18easiest | ||||||
| Of 30 | 30 | 29 | 28 | 25 | 23 | 6 | 141/180 |
Hardest: question 8, 7 of 18 tries right. Easiest: question 10, 17 of 18. Seven of the ten questions are sealed: they show by a short name, and their wording stays private until they retire. Every question’s one-line description and every answer are on the answers page.
Question 7 · sealed · A Highway Code question about footwear
Same question, two answers
Wrong · went along with a false premise
97
The whole answer, verbatim: ChatGPT, try 1. The same mark on all three of its tries at this question. ChatGPT: 23 of 30 right in run 2.
Right
Rule 97. It says clothing and footwear must not prevent you using the vehicle’s controls correctly; it does not explicitly ban flip-flops. Source: fullfact +1
The whole answer, verbatim: Perplexity, try 3. One of its three tries at this question got this mark. Perplexity: 25 of 30 right in run 2.
Every answer is on the answers page, word for word.
Run on run
First baseline run, so no change yet.
Run 1 ran on different plans, so this run is not set against it. Most runs should read “No change detected”. Boring is the goal.
When a change can be called
Runs 2 and 3, every app on its free plan, are the baseline: together they show how much each app's score moves on its own. From run 4, each report sets the run beside the ones before it, back to run 2, question by question, and can call a change.