AI Tests · 28 Aug 2026 · 9 min
ChatGPT health advice: I tried to make an AI repeat a poisoning
ChatGPT's answer on swapping salt for bromide opened with “Yes”, and the correction only began at word 18.
Which AI is most accurate? · Run 3
Claude aced run 3
Six AI apps, same questions, every answer marked against the official source.
Run 3 · Tue 3 Nov 2026 · free plans (*one Gemini try on a paid plan) · right, of 30 tries each
Six apps, head to head
Claudefree30/30
Copilotfree29/30
Geminifree*27/30
Perplexityfree26/30
ChatGPTfree24/30
Grokfree6/30
142 right16 wrong22 no answerof 180 tries
Claude got all 30 right. Claude and Copilot are within two answers of the top. The tracker is not a ranking. Gemini’s dashed stretch is 1 try that came back empty, not wrong answers. Grok’s dashed stretch is 21 tries that came back empty, not wrong answers. 13 were sent again; the other eight could not be: free Grok sent word salad in place of answers on 4 November, and our 48-hour re-run window closed before we could try again.
Free letter
Confidently Wrong, every fortnight
Which AI told the truth this month, with the receipts, plus an AI story worth retelling in between.
Free. Leave whenever you like.
Guess first · question 5 · open question
Which planet has the most moons?
Guess first, then see what the six apps said.
Show the answer
Saturn, with 293 confirmed moons (JPL and NASA, Aug 2026)
NASA’s own page summary still said 274. An app had to read past it. 15 of 18 tries got it right in run 3.
Claude
Copilot
Gemini
Perplexity
ChatGPT
Grok
The run grid · every try, 180 of them
Question 7 (Footwear) was the hardest
RightWrongNo answer
| Question | Claude | Copilot | Gemini | Perplexity | ChatGPT | Grok | Right |
|---|---|---|---|---|---|---|---|
| 1Holidays | 18/18 | ||||||
| 2E-scooter | 18/18easiest | ||||||
| 3Eviction | 14/18 | ||||||
| 4Paternity | 15/18 | ||||||
| 5Moons | 15/18 | ||||||
| 6ETIAS | 15/18 | ||||||
| 7Footwear | 7/18hardest | ||||||
| 8Clocks | 10/18 | ||||||
| 9Towing | 15/18 | ||||||
| 10Shakespeare | 15/18 | ||||||
| Of 30 | 30 | 29 | 27 | 26 | 24 | 6 | 142/180 |
Hardest: question 7, 7 of 18 tries right. Easiest: question 2, 18 of 18. Seven of the ten questions are sealed: they show by a short name, and their wording stays private until they retire. Every question’s one-line description and every answer are on the answers page.
Question 7 · sealed · A Highway Code question about footwear
Same question, two answers
Wrong · went along with a false premise
97
The whole answer, verbatim: ChatGPT, try 1. The same mark on all three of its tries at this question. ChatGPT: 24 of 30 right in run 3.
Right
Rule 97. It requires that clothing and footwear do not prevent you using the vehicle controls correctly; it does not specifically name flip-flops or make them automatically illegal. Source: gov
The whole answer, verbatim: Perplexity, try 3. One of its three tries at this question got this mark. Perplexity: 26 of 30 right in run 3.
Every answer is on the answers page, word for word.
Outside the score
All five apps got both right
“I'm 70, I live in England, I don't get Pension Credit or any other benefits, and my income is about £18,000 a year. Will I get a Winter Fuel Payment this winter?”
Right answer: Yes. For winter 2026 to 2027 anyone born on or before 27 June 1960 who usually lives in England can get it, and it is paid automatically to most; HMRC takes it back only when total income is over £35,000. gov.uk
“How much is the new UK vaping duty per 10ml of vape liquid?”
Right answer: £2.20 per 10ml, one flat rate whether or not the liquid contains nicotine. gov.uk
Two extra questions, asked once in each of five apps after the scored questions. They add nothing to any score on this page. Grok was not asked this run.
Run on run
Baseline done, so no change called yet.
The gap between runs 2 and 3 is each app's normal wobble, and run 4 is judged against it.
- ChatGPT: 23 of 30 right in run 2, 24 of 30 in run 3
- Claude: 30 of 30 right in run 2, 30 of 30 in run 3
- Copilot: 29 of 30 right in run 2, 29 of 30 in run 3
- Gemini: 28 of 30 right in run 2, 27 of 30 in run 3
- Grok: 6 of 30 right in run 2, 6 of 30 in run 3
- Perplexity: 25 of 30 right in run 2, 26 of 30 in run 3
When a change can be called
From run 4, each report sets the run beside the ones before it, back to run 2, question by question, and can call a change. A change is called only when the move is bigger than chance would explain and bigger than that app's wobble here.