Skip to content
// Category
Tool Audit 5 posts · updated Jun 2026

Tool Audit.

Does the tool do the task, or just sound like it did? One invented an options table. One misread an annual report by a factor of a thousand. Verdicts per task, with the screenshots.

6 documented AI failures and catches across these posts, each with its dated screenshot. The evidence →

Posts here put named AI tools against tasks I'd run anyway: same prompts, same day, fresh conversations, outputs saved verbatim. Verdicts are per task, not per tool, because no single tool wins a whole research session. In my testing Claude reads documents best, Perplexity is for cited retrieval, ChatGPT is the dependable middle, and Gemini structures research well on big names, and invents things around options.

Each audit names a winner for the task it tested, shows the screenshots behind the verdict, and records what each tool got wrong as carefully as what it got right. The worst failures (an invented options premium table, an annual-report misread off by a factor of a thousand) are kept on /lessons with dates and screenshots.

Tools change fast, so every post carries the model version and the date it ran. A verdict from January isn't a verdict on today's model, which is half the argument for testing rather than trusting.

← All posts