Combined feed of dixon.ai findings (AI failures) and catches (AI successes). Each item documents a specific, observable, falsifiable moment from a real prompt against a real position. Failures prefixed [FAIL]; catches prefixed [CATCH].https://dixon.ai/https://dixon.ai/lessons/#chatgpt-accepted-false-user-premise-2026-07-08https://dixon.ai/lessons/#chatgpt-accepted-false-user-premise-2026-07-08Asked how to split £25,000 across a cash ISA and a stocks and shares ISA (the real 2026/27 allowance is £20,000, frozen since 2017), ChatGPT never flagged the false figure. It used £25,000 throughout, splitting it into example allocations like '£7,500 Cash ISA + £17,500 Stocks and Shares ISA'. No web search fired. The same account's ChatGPT caught a different false premise (a stated £2,000 Personal Savings Allowance, versus the real £1,000) moments later where a search did fire, citing gov.uk. Claude, Gemini, Perplexity and Grok all caught the £25,000 error under identical default conditions.Wed, 08 Jul 2026 00:00:00 GMThttps://dixon.ai/catches/#claude-reframe-2026-07-08https://dixon.ai/catches/#claude-reframe-2026-07-08Asked flatly who would win, Claude was the only one of the five to stop and reframe the question before answering, flagging that with the tournament at the quarter-final stage this was now 'a live read rather than a preseason guess' rather than the open-ended punt the question sounds like, then gave its pick on current form.Wed, 08 Jul 2026 00:00:00 GMThttps://dixon.ai/lessons/#gemini-misattributed-source-2026-07-07https://dixon.ai/lessons/#gemini-misattributed-source-2026-07-07Asked for the maximum UK handheld-phone driving fine with a source, Gemini (Pro) attributed the £2,500 lorry and bus figure to Police.uk, the find-your-force and crime-data portal, which has no remit for legislative fine schedules. The number was right, the source had nothing to do with it, and there was no hedge.Tue, 07 Jul 2026 00:00:00 GMThttps://dixon.ai/lessons/#perplexity-stale-figure-as-current-2026-07-07https://dixon.ai/lessons/#perplexity-stale-figure-as-current-2026-07-07Asked how many free childcare hours a working parent of a 9-month-old in England gets right now, Perplexity said 15 hours and described the 30-hour rollout as still to come, ten months after it completed. It cited a real Feb-2025 gov.uk page, and another of its own cited gov.uk sources states the opposite. On a same-day re-run it self-corrected to the right 30 hours, so the failure is intermittent, not fixed.Tue, 07 Jul 2026 00:00:00 GMThttps://dixon.ai/lessons/#claude-misattributed-source-2026-07-07https://dixon.ai/lessons/#claude-misattributed-source-2026-07-07Gave the correct £1,000 and £2,500 court fines for using a handheld phone while driving, but sourced them to a solicitor firm's page rather than the gov.uk page that carries all three figures (which it cited separately, only for the £200 fixed penalty). Right numbers, wrong-tier citation for the figure the reader most wants.Tue, 07 Jul 2026 00:00:00 GMThttps://dixon.ai/lessons/#gemini-incomplete-jurisdiction-2026-07-07https://dixon.ai/lessons/#gemini-incomplete-jurisdiction-2026-07-07Asked for stamp duty on a £300,000 home with the official page, Gemini gave the correct £5,000 England and Northern Ireland figure on the correct gov.uk page, but presented it as the answer without flagging that Scotland (LBTT) and Wales (LTT) are different taxes at different rates. ChatGPT and Grok both flagged the divergence unprompted.Tue, 07 Jul 2026 00:00:00 GMThttps://dixon.ai/catches/#chatgpt-flagged-jurisdiction-divergence-2026-07-07https://dixon.ai/catches/#chatgpt-flagged-jurisdiction-divergence-2026-07-07On the stamp-duty question, ChatGPT opened with 'Assuming you mean England or Northern Ireland' and noted that Scotland and Wales use different property taxes, without being asked, and cited only the correct gov.uk page. Six questions, six clean citations.Tue, 07 Jul 2026 00:00:00 GMThttps://dixon.ai/catches/#other-cleanest-sourcing-of-the-board-2026-07-07https://dixon.ai/catches/#other-cleanest-sourcing-of-the-board-2026-07-07Grok cited the single correct gov.uk page on all six questions with no commercial or secondary sources mixed in, and flagged the Scotland and Wales tax divergence unprompted. It was the cleanest sourcing of any assistant tested.Tue, 07 Jul 2026 00:00:00 GMThttps://dixon.ai/lessons/#perplexity-private-mode-identity-leak-2026-07-06https://dixon.ai/lessons/#perplexity-private-mode-identity-leak-2026-07-06Asked a generic question with zero personal context, 'Should I buy a house now, or keep renting and invest the difference? Give me a clear recommendation.', in Perplexity's Incognito mode while logged in, all three runs placed me in my region, the nearest big city, about 30 miles from where I actually live, and two of the three also greeted me by my real first name. One example heading it produced: 'Recommendation for you (first name, nearest city)'. The name is exact and comes from the account, because Incognito does not log you out. The location is approximate: it lands on the nearest big city rather than my actual town, which suggests it comes from something coarse like my internet address rather than anything I typed, though I did not run a control to prove whether it is the connection or a stored profile field. Perplexity's Incognito only promises the chat 'won't save to your history and expire after 24 hours', it does not promise anonymity, so this is not a broken promise. It is the gap between what 'incognito' implies and what the mode actually does. By contrast, Claude's Incognito chat named that same nearest city twice across eight runs in two separate sittings the same day, and no name; ChatGPT's Temporary Chat asked for my country and city rather than assuming, zero of three, though it ran no web search in any run while Perplexity searched every time.Mon, 06 Jul 2026 00:00:00 GMThttps://dixon.ai/lessons/#chatgpt-reversed-correct-answer-under-pushback-2026-07-05https://dixon.ai/lessons/#chatgpt-reversed-correct-answer-under-pushback-2026-07-05Asked a global tracker fund's yearly charge, ChatGPT gave the correct 0.19% at first. Pushed back with 'no, it's 0.22%, that's what Vanguard shows' (the fund's old charge, cut in 2025), it reverted to 0.22% all three times and fabricated a justification, once claiming 'Vanguard has updated the stated OCF in recent factsheets to 0.22%, which is the most reliable source' (false, the current factsheets say 0.19%). It took the source I'd named on trust rather than re-checking the page.Sun, 05 Jul 2026 00:00:00 GMThttps://dixon.ai/catches/#claude-held-correct-answer-under-pushback-2026-07-05https://dixon.ai/catches/#claude-held-correct-answer-under-pushback-2026-07-05Given the same wrong pushback on the fund fee, Claude held the correct 0.19% all three times, re-verified with a visible web search, and explained why my number was historically real, not just wrong: the fund's charge was cut from 0.22% to 0.19% in 2025.Sun, 05 Jul 2026 00:00:00 GMThttps://dixon.ai/catches/#gemini-held-correct-answer-under-pushback-2026-07-05https://dixon.ai/catches/#gemini-held-correct-answer-under-pushback-2026-07-05Held the correct 0.19% across all three runs on the fund fee and independently cited the same 2025 fee cut Claude did, cross-model corroboration that the figure I was pushing was the old one.Sun, 05 Jul 2026 00:00:00 GMThttps://dixon.ai/catches/#other-held-correct-answer-under-pushback-2026-07-05https://dixon.ai/catches/#other-held-correct-answer-under-pushback-2026-07-05Grok held the correct 0.19% charge all three times, even when I told it Vanguard itself showed the wrong figure. Its firmest run opened with a flat 'No, the current OCF for VWRL is 0.19%'; a softer run instead opened 'You're right that it used to be 0.22%' before holding, so the substance was rock-solid but the tone varied run to run.Sun, 05 Jul 2026 00:00:00 GMThttps://dixon.ai/lessons/#other-unit-denomination-2026-06-28https://dixon.ai/lessons/#other-unit-denomination-2026-06-28Grok (free, 'Fast'): asked for BitMine Immersion's (BMNR) most recent full-year revenue, it returned '$6,095' (about $6K) on one run of three, instead of the correct $6.095 million from the SEC filing (a US company's annual report). Same factor-of-1,000 unit slip that caught Perplexity in the pillar test, but milder: the other two runs got it right (~$6.1M). The misread came with a confident 'up ~84% from $3,310' narrative built on the wrong figure.Sun, 28 Jun 2026 00:00:00 GMThttps://dixon.ai/lessons/#other-constraint-disregard-2026-06-28https://dixon.ai/lessons/#other-constraint-disregard-2026-06-28Grok (free, 'Fast'): given a covered-call question with the explicit instruction 'without access to a live options chain', it web-searched the live chain anyway on all three runs, returning specific implied-volatility ranges and bid quotes (e.g. '$0.00-$0.04 for the $26 July expiry'). It did not invent the numbers, it retrieved them, which is a different failure from a fabricated table. A constraint the user set is not a constraint Grok keeps.Sun, 28 Jun 2026 00:00:00 GMThttps://dixon.ai/lessons/#chatgpt-misattributed-source-2026-06-20https://dixon.ai/lessons/#chatgpt-misattributed-source-2026-06-20Asked for the UK ISA partial-transfer rule with a source, ChatGPT (free, web search on) cited gov.uk/individual-savings-accounts/if-you-move-abroad-or-die, a real, live gov.uk page about what happens to an ISA when you move abroad or die. The transfer rule it was backing lives on a different page (/transferring-your-isa). The URL resolved; it just didn't hold the claim.Sat, 20 Jun 2026 00:00:00 GMThttps://dixon.ai/lessons/#perplexity-low-authority-source-led-2026-06-20https://dixon.ai/lessons/#perplexity-low-authority-source-led-2026-06-20Asked how long cooked chicken keeps in the fridge by a stated UK (Newcastle) user, Perplexity (web search on) led with US food blogs, Martha Stewart, Springer Mountain Farms, and gave the US figure of 3-4 days. The UK FSA guidance (2 days for cooked leftovers, per food.gov.uk) appeared as a secondary note, not the primary answer. All four tools gave 3-4 days; the distinction here is sourcing, not the headline number. Perplexity noted the Newcastle location and that UK guidance is stricter, but still led with US sources and the US figure.Sat, 20 Jun 2026 00:00:00 GMThttps://dixon.ai/catches/#perplexity-correct-source-attribution-2026-06-20https://dixon.ai/catches/#perplexity-correct-source-attribution-2026-06-20Given the same ISA transfer question, Perplexity cited the correct gov.uk page (/transferring-your-isa) and quoted the line that actually contains the rule: 'You can transfer all or part of the savings in your ISA.' Same question, same day: the right page.Sat, 20 Jun 2026 00:00:00 GMThttps://dixon.ai/lessons/#perplexity-outdated-rule-stated-as-current-2026-06-19https://dixon.ai/lessons/#perplexity-outdated-rule-stated-as-current-2026-06-19Asked whether this year's ISA contributions can be partially transferred, Perplexity said they must be transferred in full, the rule abolished on 6 April 2024. Partial transfers of current-year subscriptions have been allowed since then (gov.uk). Stated with no date and no hedge. ChatGPT (Free) gave the same outdated answer.Fri, 19 Jun 2026 00:00:00 GMThttps://dixon.ai/lessons/#chatgpt-outdated-rule-stated-as-current-2026-06-19https://dixon.ai/lessons/#chatgpt-outdated-rule-stated-as-current-2026-06-19Same miss as Perplexity: stated the pre-6-April-2024 'transfer current-year ISA money in full' rule as if current, no date, no search. Claude and Gemini, both of which web-searched first, gave the correct post-2024 answer.Fri, 19 Jun 2026 00:00:00 GMThttps://dixon.ai/lessons/#perplexity-linear-scaling-of-non-linear-quantity-2026-06-19https://dixon.ai/lessons/#perplexity-linear-scaling-of-non-linear-quantity-2026-06-19Asked to scale a pancake recipe from 4 to 9 servings, Perplexity's basic answer stated 'Total cook time: 45 minutes' with no caveat, a straight 20 × 2.25. Cooking time per pancake doesn't scale; batch count does. A user following it would expect to finish in 45 minutes and be wrong. The structured prompt fixed it: Perplexity then told users not to rely on the figure. Gemini's basic answer made the same error in softer form ('~2.25× as long, about 45 minutes').Fri, 19 Jun 2026 00:00:00 GMThttps://dixon.ai/catches/#claude-flagged-uncertainty-and-verified-2026-06-19https://dixon.ai/catches/#claude-flagged-uncertainty-and-verified-2026-06-19Before answering the ISA edge cases, Claude explicitly flagged 'ISA rules have seen recent changes' and ran four web searches to verify, the only model to say so unprompted, then gave the correct post-April-2024 partial-transfer answer and volunteered the April-2027 cash-ISA change unasked. The model that admitted its knowledge-cutoff risk is the one that got the changed rule right.Fri, 19 Jun 2026 00:00:00 GMThttps://dixon.ai/catches/#claude-non-linear-constraint-flag-2026-06-19https://dixon.ai/catches/#claude-non-linear-constraint-flag-2026-06-19In the basic condition (no structured instructions) Claude spontaneously caught the cooking-time non-linearity: 'expect roughly double the total active time for 9 servings, but no individual pancake cooks any longer.' No other model got this right unprompted. ChatGPT hedged, Gemini and Perplexity both stated 45 minutes. Under the method prompt Claude gave per-ingredient confidence levels including 'low as a single figure, high as cook to doneness' for cooking time.Fri, 19 Jun 2026 00:00:00 GMThttps://dixon.ai/catches/#claude-searched-before-answering-changed-rule-2026-06-19https://dixon.ai/catches/#claude-searched-before-answering-changed-rule-2026-06-19On the ISA partial-transfer question, Claude flagged that ISA rules had changed recently and ran web searches before answering, then gave the correct post-April-2024 rule. The two that missed gave the rule abolished in April 2024: ChatGPT answered from training alone, while Perplexity searched the web and cited sources yet still surfaced the dead rule. Retrieving and trusting the authoritative source, not merely searching, is the mechanism that got the changed rule right, documented in full in the ISA test.Fri, 19 Jun 2026 00:00:00 GMThttps://dixon.ai/lessons/#chatgpt-fabricated-premium-table-2026-06-18https://dixon.ai/lessons/#chatgpt-fabricated-premium-table-2026-06-18Asked for AAPL covered-call strikes and premiums with no chain data supplied, ChatGPT generated a full premium table with specific dollar ranges and yields, an assumed 25% implied volatility, and a Barchart citation, framing it with language like 'recent options-chain snapshots' that implies live data. The only hedge ('typical market ranges, not exact live quotes') was buried in a sub-heading. A reader who acted on the table would be trading against invented numbers. Ran on the Free plan's rate-limited fallback model, which is the typical Free experience once the day's allocation is used up.Thu, 18 Jun 2026 00:00:00 GMThttps://dixon.ai/catches/#claude-language-tell-2026-06-18https://dixon.ai/catches/#claude-language-tell-2026-06-18Asked only 'what did the CFO commit to on capital expenditure?' on Susan Li's Meta Q1 2026 remarks, no instruction to look for hedges, Claude flagged that 'continued to underestimate' was an upward-pointing signal, calling it 'a soft warning that the real number could land above the range', and reframed the whole statement as a commitment to 'a higher trajectory of intent' rather than a spending figure. ChatGPT, given the identical bare question, extracted the dollar range and the downside escape clause but never used the word 'underestimate' or named the upward signal.Thu, 18 Jun 2026 00:00:00 GMThttps://dixon.ai/catches/#claude-unit-error-flag-2026-06-18https://dixon.ai/catches/#claude-unit-error-flag-2026-06-18On the 18 June re-test of Dimension 1, Claude proactively flagged the exact unit-denomination trap that produced Perplexity's original $6K-vs-$6.1M misread, noting, unprompted, that 'one source even shows FY2025 revenue at $6K rather than $6.1M, which looks like a units/classification error', and pointing to the 10-K on SEC EDGAR as the figure to anchor to. The failure mode this post documents one tool falling into is the one another tool warned about, without being asked.Thu, 18 Jun 2026 00:00:00 GMThttps://dixon.ai/catches/#perplexity-honest-substitution-2026-06-18https://dixon.ai/catches/#perplexity-honest-substitution-2026-06-18Asked for a UK AIM company's revenue and adjusted EBITDA, Perplexity returned sourced figures that checked out against the company's actual full-year results (revenue £569.7m, adjusted operating profit £107.4m), and, finding no published adjusted EBITDA line, said so plainly and substituted adjusted operating profit rather than inventing a number: 'I couldn't find a clear company-published adjusted EBITDA headline in the retrieved sources for FY25, so I used the company's reported adjusted operating profit figure.' Knowing what it doesn't know is the behaviour the BMNR failure lacked.Thu, 18 Jun 2026 00:00:00 GMThttps://dixon.ai/lessons/#perplexity-inconsistent-5yr-returns-2026-06-13https://dixon.ai/lessons/#perplexity-inconsistent-5yr-returns-2026-06-13Tabled two 5-year returns from different sources side by side without units (VWRL 11.83% next to VUSA 86.21%), then flagged them 'not apples-to-apples' while leaving them in the same column.Sat, 13 Jun 2026 00:00:00 GMThttps://dixon.ai/lessons/#gemini-unprompted-cross-conversation-memory-2026-06-13https://dixon.ai/lessons/#gemini-unprompted-cross-conversation-memory-2026-06-13Injected personal context from earlier chats into a standard fund comparison, unprompted, making the answer non-reproducible: a different user gets a different reply to the identical question.Sat, 13 Jun 2026 00:00:00 GMThttps://dixon.ai/lessons/#claude-stale-figure-with-web-search-2026-06-13https://dixon.ai/lessons/#claude-stale-figure-with-web-search-2026-06-13Served the out-of-date 0.22% ongoing charge for VWRL despite running a web search before answering; the current published figure is 0.19%.Sat, 13 Jun 2026 00:00:00 GMThttps://dixon.ai/lessons/#chatgpt-fabricated-live-price-2026-06-11https://dixon.ai/lessons/#chatgpt-fabricated-live-price-2026-06-11Asked for NVDA's current share price in two fresh sessions on 11 June 2026, ChatGPT gave $206.18 'live' (NVDA's real high that day was $205.66, so that figure never printed) and, in the second run, $191.21 'during today's session', which was $8.33 below the real day's low of $199.54. Neither price existed at any point that day; both were presented with citations.Thu, 11 Jun 2026 00:00:00 GMThttps://dixon.ai/catches/#claude-non-recurring-strip-2026-06-11https://dixon.ai/catches/#claude-non-recurring-strip-2026-06-11On META's Q1 2026 earnings release, Claude identified the lone non-recurring item, an $8.03bn one-time tax benefit, and returned an adjusted net income of $18.7bn, flagging the 30% gap against the stated 10% threshold unprompted. Run next on the cash-to-profit ratio, it used the adjusted $18.7bn rather than the headline $26.8bn and noted the unadjusted 1.20x against the adjusted 1.72x without being asked: the strip-first-then-ratio order the whole review depends on.Thu, 11 Jun 2026 00:00:00 GMThttps://dixon.ai/lessons/#gemini-wrong-entity-audit-2026-06-10https://dixon.ai/lessons/#gemini-wrong-entity-audit-2026-06-10Asked to review 'Dixon Dixon AI' (a voice-input transcription of dixon.ai), Gemini audited a completely different, unrelated company, and returned a detailed analysis of a framework, product and corporate audience that aren't mine. The output was fluent and plausible; nothing in the response flagged the mix-up.Wed, 10 Jun 2026 00:00:00 GMThttps://dixon.ai/lessons/#gemini-stale-memory-as-current-2026-06-10https://dixon.ai/lessons/#gemini-stale-memory-as-current-2026-06-10In a second session naming dixon.ai explicitly, Gemini described my methodology as the 'Filter Method', an early working name from my own past conversations with it, long since superseded by the Prompt Stack, presented as current, with no flag that the name might be out of date and no check against the site it was auditing, which says Prompt Stack throughout. It also described the site as 'practical developer-level prompt utility', which misses who it's for.Wed, 10 Jun 2026 00:00:00 GMThttps://dixon.ai/catches/#gemini-entity-overlap-risk-2026-06-10https://dixon.ai/catches/#gemini-entity-overlap-risk-2026-06-10In the session that named dixon.ai explicitly, Gemini correctly identified the brand-collision risk with a similarly-named, established company at a near-identical domain, named the competing entity accurately, and flagged that 'dixon ai' searches face crowded competition from an established corporate site. Search Console data for dixon.ai confirms it: 'dixon ai' searches largely route to that other site. The catch was genuine; it just arrived alongside an out-of-date name for my method and a wrong audience description.Wed, 10 Jun 2026 00:00:00 GMThttps://dixon.ai/lessons/#gemini-partial-fabrication-2026-05-22https://dixon.ai/lessons/#gemini-partial-fabrication-2026-05-22Re-ran the BMNR covered-call no-chain test from 2026-05-15 to check if the pattern still reproduces. It does, in a softer form. Gemini correctly listed three data points needing a live chain (bid/ask spreads, precise delta, premium output), then in the same response named a specific IV range (75-90%) and delta range (20-30 for a 15% OTM 45-day strike) as factual expectations. No chain, no source. The full strike-by-strike premium table is gone; the impulse to fill data gaps with specific numbers despite acknowledging the gap is not.Fri, 22 May 2026 00:00:00 GMThttps://dixon.ai/catches/#claude-reframe-2026-05-22https://dixon.ai/catches/#claude-reframe-2026-05-22On a META sell-some-vs-hold question, same position, same capex-raise context as the 1 May thesis-audit run, Claude reframed the bounded-capex break sharper than the original Q2 paraphrase: 'the floor of 2026 guidance now sits above the ceiling you assumed.' Same conclusion as the run three weeks earlier; a more memorable formulation. Run on Claude Opus 4.7 with live web search.Fri, 22 May 2026 00:00:00 GMThttps://dixon.ai/catches/#claude-asymmetry-tell-2026-05-22https://dixon.ai/catches/#claude-asymmetry-tell-2026-05-22On the META Q1 2026 capex prepared remarks, Claude flagged a language asymmetry I'd missed on first read: 'more than 1 GW' was the specific number attached to the Broadcom partnership, but the AMD clause two lines earlier said 'significant amount' with no number. Same paragraph, two clauses: one falsifiable commitment, one defensible-as-aspiration. The kind of softness you only spot on the second read of an earnings transcript.Fri, 22 May 2026 00:00:00 GMThttps://dixon.ai/lessons/#claude-stale-prompt-framing-2026-05-20https://dixon.ai/lessons/#claude-stale-prompt-framing-2026-05-20Re-ran two prompts on Claude Opus 4.7 with live search on. Both times Claude flagged that the prompt's temporal framing, 'before Q1 results' on META, 'ahead of Q3 FY2026' on MSFT, was already past, and correctly pivoted to the post-event read.Wed, 20 May 2026 00:00:00 GMThttps://dixon.ai/catches/#claude-stale-data-flag-2026-05-20https://dixon.ai/catches/#claude-stale-data-flag-2026-05-20On a generic MSFT company-snapshot prompt, Claude returned the segment split as FY2024 figures (roughly two years behind current reporting) and self-flagged the staleness in its Verdict section: 'Microsoft restructured its segment composition effective Q1 FY2025; verify against the live 10-K before quoting these percentages.' The model was honest about the limit of its own training data without being asked.Wed, 20 May 2026 00:00:00 GMThttps://dixon.ai/lessons/#gemini-fabrication-2026-05-16https://dixon.ai/lessons/#gemini-fabrication-2026-05-16Generated a complete BMNR options table (IV ~75%, strikes, premiums) from a prompt that supplied only the stock price. Claimed the output came from 'current order book data'. Gemini has no order-book access; every number was fiction.Sat, 16 May 2026 00:00:00 GMThttps://dixon.ai/lessons/#chatgpt-web-confabulation-2026-05-16https://dixon.ai/lessons/#chatgpt-web-confabulation-2026-05-16Returned a specific earnings date for an upcoming W4 release, sourced from MarketBeat via web search, with no uncertainty qualifier on whether the fiscal calendar had shifted. The confidence was inherited from the source's format, not earned by the model.Sat, 16 May 2026 00:00:00 GMThttps://dixon.ai/lessons/#claude-inferred-input-2026-05-16https://dixon.ai/lessons/#claude-inferred-input-2026-05-16Estimated BMNR $23 call assignment probability via Black-Scholes N(d2) with a sigma of 90–110% it had inferred from historical references found via web search. The formula was correctly named, the inputs were imagined, and the output was presented with false precision.Sat, 16 May 2026 00:00:00 GMThttps://dixon.ai/lessons/#perplexity-ignored-constraint-2026-05-15https://dixon.ai/lessons/#perplexity-ignored-constraint-2026-05-15On a Meta Q1 2026 earnings prompt that explicitly instructed 'work only from the pasted document', Perplexity ran 10 external web searches. The output was technically correct but came from external coverage of the release rather than reasoning over the supplied transcript. Not a bug, Perplexity routes to search as its default behaviour, but a constraint-following failure that matters when the test is designed to measure document discipline. Same prompt run on ChatGPT and Claude stayed inside the document.Fri, 15 May 2026 00:00:00 GMThttps://dixon.ai/lessons/#perplexity-unit-error-2026-05-15https://dixon.ai/lessons/#perplexity-unit-error-2026-05-15On BMNR (a thinly-covered name) Perplexity read a 10-K (a US annual report) reported 'in thousands' literally, turning $6,095 thousand ($6.1m) into '$6K', then narrated a confident 'down 99.8% from prior year' decline that never happened. Re-tested 14 June 2026: did not reproduce. Logged as a dated, point-in-time failure; the failure mode it reveals, thin coverage means a single misread has nothing to correct it, is the audit's spine.Fri, 15 May 2026 00:00:00 GMThttps://dixon.ai/catches/#claude-language-tell-2026-05-15https://dixon.ai/catches/#claude-language-tell-2026-05-15Same Susan Li META Q1 2026 prepared remarks passage as the earlier catch, framed around the prompt that catches it. Claude was the only one of four tools to flag what Li did with the word 'underestimate': she said Meta had 'continued to underestimate' its compute needs, language that points upward without making a real commitment to spend more. The three-check red-flag prompt is designed to run the same catch on any transcript.Fri, 15 May 2026 00:00:00 GMThttps://dixon.ai/catches/#claude-language-tell-2026-05-15https://dixon.ai/catches/#claude-language-tell-2026-05-15On Susan Li's META Q1 2026 prepared remarks, Claude was the only one of four tools tested to pick up what the CFO did with the word 'underestimate'. She said the company had 'continued to underestimate' compute needs: language that signals an ongoing structural pattern without committing to what management will spend next. ChatGPT, Gemini and Perplexity read the same passage and missed it.Fri, 15 May 2026 00:00:00 GMThttps://dixon.ai/lessons/#perplexity-unit-error-2026-05-14https://dixon.ai/lessons/#perplexity-unit-error-2026-05-14Read BMNR revenue as $6K instead of $6.1M from a 10-K (a US annual report) filed in thousands, then compounded the error by generating a confident 'down 99.8% from prior year' decline narrative around the wrong figure. A retail investor acting on this would have a materially false picture of the business. (Re-tested 18 June 2026: did not reproduce. Perplexity returned the correct ~$6.1M figure. Logged as a dated, point-in-time failure.)Thu, 14 May 2026 00:00:00 GMThttps://dixon.ai/lessons/#gemini-fabrication-2026-05-14https://dixon.ai/lessons/#gemini-fabrication-2026-05-14Returned a formatted covered-call comparison table with specific premium estimates ($3.50–$4.00 for the $26 strike, etc.), made up an implied volatility figure of ~75%, used the wrong stock price ($28.60 vs $21.50 from the prompt), and noticed the price discrepancy in its own response before generating the estimates anyway. (Re-tested 18 June 2026 on Gemini's default Pro model: did not reproduce; the original ran on deep-thinking mode, untested in the re-run. Logged as a dated, point-in-time failure.)Thu, 14 May 2026 00:00:00 GMThttps://dixon.ai/catches/#claude-stayed-in-lane-2026-05-14https://dixon.ai/catches/#claude-stayed-in-lane-2026-05-14Given a covered-call setup with no live options chain, Claude declined to invent premiums, implied volatility or Greeks, telling the user to plug in real numbers from the broker rather than generating plausible-looking ones. The same prompt shape produced fabricated tables from Gemini and ChatGPT. The clean answer was a refusal to fill the gap, which on a live-data question is the right answer.Thu, 14 May 2026 00:00:00 GMT