How often is ChatGPT wrong? 20 tests in 2026, one clear pattern

The free app first. Run 2 of my tracker went out on Sunday 11 October 2026: ten everyday questions, three tries each, every answer marked against the official source.

Bank holidays in 2027, Saturn’s moons, whether your own e-scooter can go on the pavement and towing speed limits all held. Free Grok got the same ten questions, and how that went is its own story.

Then OpenAI’s models, on a different test with different questions. Edition 2 of my Truth Index put 100 questions to 12 models on 2 November 2026, one try each, through the API (the route developers use) with no web search.

GPT-6.1 Sol got a Trust Score of 97 out of 100 (likely range 94 to 100), the top score of the twelve.

Put the 30 app answers and Sol’s 100 side by side and, apart from one try that gave Shakespeare a birthday nobody wrote down, I count three kinds of mistake.

A wrong idea built into the question. Ask which rule bans something and it finds you a rule, as the footwear question showed.

A sum the calendar changes. A night shift across the October clock change went wrong three times out of three. Every answer named the clock change and did the sum both ways, then put the wrong total first, like a pupil who gets it right in the margin and copies the other line into the box. The clocks post has the full sum.

A fee that has just gone up. The old figure was right once, which is why it still sounds right. Sol’s three mistakes were all fees that went up in 2026.

For two of the three it named GOV.UK as its source. On the dental charge it even said it couldn’t check whether the price had changed, and sent me to the NHS page. It had, by 50p. Its probate fee was right until July.

An earlier tally from mid-2026, about 20 documented cases (17 structured tests plus failures I logged in real use), found the same split, with fixed facts holding and live numbers made up.

I haven’t stopped using ChatGPT. Ask it the bank-holiday kind of question and it’s quick and mostly right.