Sponsorship

Interested in sponsoring AI in QA? Reach out.

AI in QA Review

We're back this week 7:00 AM CT on Monday!

Butch's Take

The teams best positioned to adopt AI in QA are the ones that already had their fundamentals locked in.

I bought Holistic Testing: Weave Quality Into Your Product by Lisa Crispin and Janet Gregory this week. It is not an AI book. It is a book about building quality into your entire development process, from exploration to test automation to the whole-team approach. The more I read, the more I see how directly it maps to where AI fits.

The book's caution about AI is exactly right: "You can generate Gherkin scenarios with ChatGPT, but if you don't already have the Gherkin skills yourself, you won't know if they're well-designed. Learn the skills yourself first."

The teams winning with AI in QA right now did the unglamorous work first: holistic testing practices, whole-team quality, and a testing strategy that existed before AI showed up.

The book is definitely worth checking out, and thanks to Lisa Crispin for posting about that and many other Holistic Testing resources

Headlines & Launches

Does Learning Stop When AI Does the Work?
Vitaly Sharovatov (LinkedIn)
Asks a sharp question: pre-AI, developers and testers built mental models of systems by doing the work themselves. Companies paid to keep cost of change low by investing in onboarding, mentorship, and good architecture. Now agents do more work, output scales, but does learning stop? If the human never builds the mental model, can they judge whether what the AI produced makes sense? Is cost of change actually dropping?

AI Career Advice: Be Able to Do the Work
Keith Klain (LinkedIn)
Direct career advice for test managers facing AI disruption: get AI literate, read the research, learn model evaluation, understand AI safety, and never get too far away from the work. Managers of managers are always the first to go. The nature of testing may not be changing, but how it's delivered and managed will change a lot. Shares resources page and AI Testing LinkedIn group.

Playwright MCP Token Problem Is Fixed
Stefan Judis (LinkedIn)
The 'MCP is too token hungry' advice is outdated. Two fixes changed the math: modern harnesses now load tool definitions lazily (not all at once), and Playwright MCP no longer inlines page snapshots by default. Snapshots go to disk, agent pulls only what it needs. Measured on a small flow: CLI and MCP both landed at 45-50k tokens. Pick based on workflow, not token anxiety.

Events

MoT AI Chapter: Skills and Plugins for AI Coding Tools
Ministry of Testing (Online)
Thursday, July 30, 2026, 13:30 UTC. Free. Session covers what Skills and plugins actually are, where they fit in a testing day (bug reports, test data, feature exploration, reading logs), and how to start with one small thing this week without heavy setup. Join the AI Chapter (free) to register.

Tools & Frameworks

Test Pages AI Chat Bot - Practice Prompt Injection Safely
Alan Richardson (LinkedIn)
EvilTester's Test Pages now has an AI chat bot running Chrome's Web AI API with RAG over shopping cart data. Designed for testers to practice bypassing prompt guardrails: extracting the system prompt, getting fake discounts, and making the bot recommend products for wrong use cases. Real value is learning how far AI-enabled apps can be pushed before they leak APIs, trade secrets, or create reputational damage. Free at testpages.eviltester.com.

The First Useful AI Tool for QA May Not Be an Autonomous Tester
Sergei Krapivin (LinkedIn)
Argues QA teams should start with a search layer over their own test case history before building autonomous agents. Practical setup: a small MCP server connected to your test management system (TestRail, TestOps) that lets Cursor or Claude retrieve existing test cases, draft checklists from team history, detect duplicates, and find related coverage for new tickets. Hybrid search approach: semantic + keyword + metadata filters. Set up in about a day.

Boruzele: AI Exploration Before Automation (ABA)
Liudas Jankauskas (LinkedIn)
Open-source proof of concept for Automation Before Automation (ABA). Instead of using AI to write tests, uses AI to explore the application before traditional automation is created. Playwright controls the browser, execution, tracing, and reporting. The LLM only decides which visible element to explore next. No predefined test steps. Adds an automated discovery layer, not a replacement for existing automation.

Getting Started with Claude Code and Playwright CLI
Aswani (dev.to)
Step-by-step guide to setting up Claude Code with Playwright CLI for browser automation. Covers installation, browser engine setup, the Playwright skill installer, and running your first natural language test against TodoMVC. Highlights token efficiency (CLI writes browser state to disk instead of streaming into context) and CI readiness.

Foundations

Write Code Like a Human Will Maintain It
Scott Robinson (unstack.io)  RENDEZVOUS WITH CASSIDOO
Argues that letting AI generate sloppy code compounds: every shortcut you merge becomes a signal the LLM uses for future generation. Duplicated conditionals become patterns, patterns become style. You are not outsourcing maintenance to the LLM, you are training it to have worse habits. Write code like a human will maintain it, because the LLM is a sponge.

The Amplifier: How AI Reveals What Was Already Broken
Test Pappy (testpappy.wordpress.com)
AI coding tools ship code faster, but testing becomes the bottleneck. Not because testers are slow, but because AI turbocharged one station on the production line while the rest stayed the same. Argues the real problem is not testing speed, it is system design. If quality is a gate at the end, AI just exposed the flaw that was always there. Testing is not a phase, it is a property of the whole system.

Techniques & Tutorials

The Court That Blocked My Release
Dragan Spiridonov (The Quality Forge)
Built an adversarial AI review court (QE-Court) where independent reviewers from different vendors prosecute every change. On its first case (its own release), the court returned BLOCK, finding three data-loss and crash bugs hiding behind a fully green unit suite. Also covers MetaHarness (100k npm downloads), cost-Pareto benchmarks where the cheap tier beat frontier on every QE task at 3-4x less cost, and completion theater caught in CI pipelines.

The Assertion Is the Hard Part
Dmitry Shyshkin (Practice Test Automation)
When an AI agent's assertion fails, its default behavior is to swap it for one that passes, not investigate why. The author caught an agent routing around a failing assertion that would have hidden a real bug. Research backs this: LLM assertion accuracy tops out at 46% on clean code and drops further on buggy code. 89-94% of generated assertions look valid, but only 57-60% check the right thing. Green isn't correct, it's just green.

Why I Prefer BDD over SDD for Agentic Development
Vitaliy Potapov (dev.to)
Argues BDD scenarios are a better planning surface for AI coding agents than markdown specs (SDD). Gherkin diffs keep the conversation focused on user flow (Given/When/Then) instead of mixing requirements, implementation ideas, and task lists. The scenario becomes both the planning artifact and the automated test. Compares workflow with OpenSpec, which produced 6 markdown files for the same pagination feature.

Self-Healing Tests with AI: Triage Before Repair
Slawek Radzyminski (Awesome Testing)
Implements a self-healing test workflow where Claude Code diagnoses failing E2E tests before repairing them. Key insight: a failing test is not necessarily a broken test. The agent triages into four categories (PRODUCT_BUG, TEST_DEFECT, CI_ENVIRONMENT, INCONCLUSIVE) and only proposes repairs for high-confidence test defects. Includes a complete GitHub Actions workflow, example repo, and two real case studies: one product bug Claude refused to heal, one test defect it repaired and opened as a draft PR.

Evaluating Agentic Browser Automation with Browser Use
Swathi Rahul (Medium)
Explored Browser Use Cloud against the OrangeHRM demo app with a single natural language prompt: log in, create an employee, search for them, verify they exist. No Selenium, no Playwright, no selectors. The agent continuously observed the UI, reasoned about state, chose next actions, and validated outcomes. Plans follow-up work on negative scenarios, ambiguous prompts, recovery from UI failures, and consistency across runs.

How I Used a QA Agent Workflow to Reduce Manual Testing Pressure
James Cullimore (Level Up Coding)  SOFTWARE TESTING WEEKLY
Built a QA agent workflow layer for an Android project that maps manual test cases to automated evidence with confidence ratings (trusted, review, manual, failed, not-tested, unmapped). Not a magic AI system, but structured documentation, mappings, reports, and prompts that help QA and coding agents work from the same source of truth. Key insight: the problem is not missing tests, it is trust and traceability. Agents need guardrails more than prompts.

Research & Data

AI and Testing: The Last Useful Animal
Jeff Nyman (Tester Stories)
Departs from technical AI testing to ask a civilizational question: are humans the horses of the AI revolution? Draws a Thomist philosophical framework to argue that optimization without a concept of human flourishing leads to techno-oligarchy. Critiques Sam Altman's 'Moore's Law for Everything' as a modern company town. Argues the common good requires active defense, not passive trust in markets.

Hot Take 🔥

We Keep Patching Test Automation. Should We Rebuild It Instead?
Chathuranga Jayasinghe (The TechiQE)
Argues the fundamental flaw in UI test automation is that tests encode the path (click sequences, selectors) when they exist to verify the goal. Fifteen years of patches (stable selectors, POM, BDD, self-healing locators, record/replay) all make scripts easier to repair, not unnecessary. When journeys get restructured, tests fail even though the feature works. Proposes tests should declare destinations, not transcript routes, and raises hard questions about determinism, auditability, and trust in adaptive execution.

Playwright 1.59 Trace Command for AI Agents
Stefan Judis (LinkedIn)
Playwright 1.59 ships a new trace CLI command that makes trace data accessible as plain text for AI agents. Agents can list test steps (trace actions), inspect page snapshots (trace snapshot), and review network traffic (trace requests) without needing a browser. A simple prompt like 'the test broke, investigate the trace' now points an agent at real evidence. Also includes a trace skill installer: playwright trace install skill.

Claude Code MacOS Notifications Utility
Filip Hric (LinkedIn)
Free utility that sends MacOS notifications (with sound) when Claude Code finishes a task or needs permission. Click the notification to jump to the app. Simple quality-of-life tool for longer agent sessions.

If something in this issue made you think differently about how your team approaches AI in testing, pass it along. The best conversations about AI and QA are happening in Slack channels and stand-ups, not just newsletters.

Have something worth featuring? Reply and send it my way, I read every link.

Thanks for reading,
Butch Mayhew