AI ISA advice: I tested four tools on the questions people get wrong

I run these tests so I know what AI ISA advice I can trust. ISA rules looked like a good thing to push on. They’re UK-only, they’re specific, and they change on a known date (6 April is when the new tax year and most rule changes land). That last part matters: a fixed reform date makes an answer’s age checkable. An ISA, for anyone arriving from the AI side of this site, is a UK tax-free savings account: the 2026/27 allowance is £20,000, and interest, income and capital gains inside the wrapper are tax-free.

So I started with the basics. Total allowance, holding more than one ISA of the same type, the Lifetime ISA limit, flexible ISAs, tax inside the wrapper. Four tools, same questions. Every one came back correct.

Four AI tools sat the same HMRC quiz and all four passed without dropping a mark, not the result I was hoping for from a test designed to catch something out.

So I moved to the questions people get wrong.

The basics: nobody put a foot wrong

I put five standard questions to ChatGPT (Free), Claude (Opus 4.8), Gemini (Pro) and Perplexity (“Best”), and watched the same correct answers come back four times over. How much you can put in across all ISA types in a year (£20,000). Whether you can now hold more than one cash ISA in the same year (yes, since April 2024). The Lifetime ISA limit and its government bonus (£4,000 a year, 25% on top). How a flexible ISA lets you replace cash withdrawn during the same tax year without the replacement reducing that year’s allowance, subject to the provider’s terms. Whether you pay tax on gains inside an ISA (you don’t).

All four got all five right. Three of them named the April 2024 change to the multiple-ISA rule without being asked. That clean result matters because it rules out the lazy conclusion that every ISA answer in these sessions was bad. It does not prove that these tools are generally dependable on stable financial rules; it proves only that this five-question basics set did not separate them.

This is the failure mode I’ve named elsewhere as stale data, fresh confidence: a model reporting a rule from the last time it was stable, with no sign it knows the rule has moved on. The basics don’t trigger it, because nothing here has changed enough for the old answer to be wrong.

The harder questions

The second round was four questions that trip up real people: the kind you’d actually type into an AI before making a decision, not contrived puzzles. I sent the same four to each tool, framed as a UK resident in the 2025/26 tax year:

1. The shared allowance. I’ve paid £15,000 into a cash ISA this year. Can I add £10,000 to a stocks and shares ISA too? (The £20,000 limit is shared across all ISA types, so the answer is no: only £5,000 left.)
2. An inherited allowance. My husband died holding about £60,000 across his ISAs. Does my own allowance change? (An eligible surviving spouse or civil partner can get an Additional Permitted Subscription, or APS, on top of the normal allowance. On these facts it starts around £60,000, but the final limit, eligibility and timing depend on the continuing-ISA rules.)
3. A partial transfer. I’ve paid into my stocks and shares ISA this year and want to move it to a cheaper provider. Can I transfer just part of it, or must I move all of this year’s money? (This is the one. More below.)
4. An age limit. I’m 17. Can I open an adult stocks and shares ISA? (No, adult ISAs are 18 and over.)

Questions 1, 2 and 4 did not contain the transfer-rule change being tested. Question 3 did. The rule on partial transfers of current-year money changed on 6 April 2024, leaving a clean before-and-after answer to verify.

The scoreboard

I ran all four the same way: same four questions, same framing, UK resident in the 2025/26 tax year. Here’s what came back.

Three clean rows. One split. The split is the interesting one.

Question 3: where two tools gave a rule that died in 2024

Before 6 April 2024, if you wanted to move ISA money you’d paid in during the current tax year, you had to move all of it: the whole year’s contributions, or nothing. The April 2024 reforms scrapped that. Since then you can transfer part of this year’s money and leave the rest where it is. The change is on gov.uk’s transfer guidance, and M&G, MoneyHelper and Royal London all confirm the same 6 April 2024 date.

When I asked, Perplexity gave the old rule. Verbatim:

ChatGPT said the same thing:

Neither attached a date. Neither hedged. ChatGPT answered from memory; Perplexity searched the web and cited its sources, and gave the dead rule anyway.

Both recited a rule superseded more than two years earlier, with no sign inside either answer that the rule had changed.

(A free-tier notice appeared mid-answer saying ChatGPT was “using a less powerful model until your limit resets.” The capture does not establish which part of the answer used which underlying model, so this result cannot support a tier comparison.)

Here’s why this one matters more than a wrong allowance figure would. A reader who trusts that answer believes they’re locked in. If they wanted to move to a cheaper provider and only shift part of this year’s money, they’d conclude they can’t, and either stay put on the higher fees, or move the whole lot when they only meant to move some. The wrong answer doesn’t just misinform; it removes an option that’s open to them.

Now the other two. Claude opened its response, before answering anything, with this:

It then ran four web searches, cited OakNorth, M&G, Unbiased, Shawbrook and MoneyHelper, gave the correct post-2024 answer, and named the April 2024 change explicitly. It also volunteered the government’s announced April 2027 plan to cut the cash ISA limit to £12,000 for under-65s, a policy now covered by draft regulations and technical consultation.

Gemini got there too, and said the quiet part out loud:

That’s the whole finding in one sentence. The rule changed; the answer changed with it. Gemini cited Moneyfacts and Magee Gammon inline.

The result does not support a neat searched-versus-didn’t-search rule. Claude and Gemini searched and got the changed rule right. No search was observed for ChatGPT, which gave the old rule. But Perplexity also searched, showed 15 source citations and still gave the old rule. In this one four-tool cell, retrieval helped twice and failed once. Search firing is not verification; the source and effective date still have to support the answer.

The tell, and the prompt that makes it checkable

What I keep coming back to is that the tell was sitting in plain view: no date attached, and no hedge. Perplexity even searched and cited sources, and still didn’t date its answer, which is the point: the missing date is the tell, not the missing search. When an AI answers a question about something that changes on fixed dates (tax rules, ISA limits, benefit thresholds) and gives you a flat answer with no date attached, the answer might be from whenever that rule was last settled, not from now.

The useful extra line is to make the tool cite the rule and the date it took effect:

That’s not a guarantee. Perplexity’s sourced miss proves why. But it turns a vague claim into a testable one. “Under ISA rules, current-year money must move in full” is hard to audit. “Under the rules as of [date], you can move part of it; this changed on 6 April 2024” gives you a date and source to compare with the current primary guidance. That would have made the error easier to catch, not impossible to make. There’s a version of this idea in the Prompt Stack, the step where you separate what’s an established fact from what the model is assuming.

One more thing worth knowing if you act on a correct answer here: current GOV.UK guidance permits all or part of current-year ISA savings to be transferred, but providers can impose transfer restrictions. Check with both sides and use the formal transfer process rather than withdrawing the money yourself.

For a money question, use AI to locate and explain the rule, then open the current primary source yourself. A search indicator or a row of citation chips is not the same thing as a supported answer; this test contains the counterexample.

Caveats

The ISA battery contains one run per tool, and I’d rather you knew its limits than took it as a league table. Three things to hold:

Claude wasn’t a clean-room run. The captured account had prior ISA conversations in it, and this session did not use a memory-off control. Some of Claude’s strong showing may reflect that context rather than the model alone. I can’t rule it out, so I won’t claim Claude is simply “more careful” on the strength of one run.

It’s one run per tool. N=1 in each cell, not a hit rate. The Question 3 miss is real and verified against gov.uk, so it stands as a dated, point-in-time result, but models get updated, and this specific failure may not reproduce next month. The lesson outlives the result: ask for the date.

ChatGPT showed a reduced-model notice during the answer. The capture does not identify which underlying model produced which passage. The wrong statement remains in the saved response, but the run should not be used as a model-tier comparison.

The honest read isn’t “don’t trust AI on your ISA.” Three of four edge-case questions were clean across every tool. The two wrong transfer answers would have been current only before 6 April 2024, more than two years before these runs, not one year earlier. The useful habit is narrower: ask for the effective date, then check it against GOV.UK or HMRC before acting.