AI in QA Review
Live on Mondays @ 7:00 AM CT (UTC-5)
Stop Competing with AI at Writing Test Scripts
Keith Klain posted a survival roadmap for QA engineers this week, and the core message is worth sitting with: stop competing with AI at writing test automation scripts, but rather focus where machines struggle. To see those areas go check out his post. These are not things you can automate away. They require judgment, context, and the ability to ask the right question before the system falls over.
But here's the other side. Use the AI tools to build. Use them to write scripts, generate change detectors, create the automation scaffolding that supports your real testing work. Don't force AI to do what humans are good at - exploration, evaluation, judgment. Use it for the things humans find tedious and machines do fast. Your feedback loops get tighter. Your coverage gets wider. You spend more time doing the work that actually matters.
As we are all learning how to get the best output from our AI tools I do hope the articles you read help guide and provide new ideas on what's worked well and not so well.
Now go read & learn something -> Then go build something.
Headlines & Launches
The Death of Test Engineering: A Survival Roadmap for the AI Era
Keith Klain (LinkedIn)
Keith Klain's Test Automation Days follow-up on the 'death of test engineering.' References Anthropic's labor market impact report showing QA/test engineers are under direct AI threat. Offers a survival roadmap: pivot to risk modeling, observability, AI evaluation design, bias/fairness detection, and systems thinking over competing with AI at writing test scripts.
Web MCP Just Rewrote Your QA Job Description
Amit Rawat (LinkedIn)
The first QA-dedicated episode of The Agentic Engineer covers Web MCP, which lets pages declare tools to AI agents so they call functions like page.searchFlights() directly instead of using locators. Selenium and Playwright move up the stack rather than disappearing. Includes a demo repo and YouTube video.
AI Was Already Here: Loud Opinions vs. Precise Understanding
Jeff Nyman (Tester Stories)
Jeff Nyman argues that critiquing all of AI based on a few ChatGPT sessions misses the bigger picture. AI has been embedded in software for decades. The testers who will be most valuable understand how these systems are built — pipelines, retrieval layers, evaluation frameworks, failure modes — and can say something precise about where they hold up and where they don't.
AI Doesn't Introduce New Risks — It Accelerates the Ones You Already Had
Simon Prior (LinkedIn)
Simon Prior argues that handing everyone an AI licence and expecting transformation is like sending someone on a Scrum master course and expecting agility. The tool was never the hard part. AI accelerates existing cracks: unclear ownership, inconsistent standards, no shared definition of good enough. Quality leadership is not optional overhead.
Tools & Frameworks
Practice Testing: AI-Powered Testing Playground with Vibium
Lady Daisy Bug (LinkedIn)
A Vibium-powered practice testing playground that adds AI agents to classic QA playground sites. Uses vibe-check to turn exploratory testing sessions into real test reports. Designed for testers transitioning into AI-native testing workflows.
Local-First QA AI Agent: Jira Tickets to Test Cases and Playwright Specs
Khin Htet Htet Khine (LinkedIn) EMILY O'CONNOR
A local-first QA AI agent that transforms Jira tickets into structured manual test cases (Excel) and Playwright starter specs. Runs fully locally with Ollama support, no API keys required for demo mode. Focuses on the step before automation: generating clear, reviewable manual test cases.
The Testing Meta Most Teams Have Not Caught Up To Yet
Murat Kerem Ozcan (DEV Community)
A QA operating system combining TEA (Test Architect agent from BMAD), Playwright-Utils, Pact.js-Utils, and Playwright/Pact MCPs to turn AI test generation from promptware into repeatable engineering. Addresses the slop problem: redundant coverage, wrong assertions, and nondeterministic flows that rot in review.
Techniques & Tutorials
AI-Assisted Testing as a QA Engineer: My Practical Roadmap Begins
Joel Thomas (LinkedIn)
An SDET's practical 9-step roadmap for learning AI-assisted testing, from foundational concepts (LLMs, tools, agents, MCP) to building a multi-agent AutoGen system that coordinates DB, UI, and API testing workflows using MCP servers. Includes hands-on setup with Claude Desktop + Playwright MCP.
10 Lessons for Agentic Coding
Don Breunig (dbreunig.com)
A practical list of 10 lessons for agentic coding: implement to learn, rebuild often, invest in end-to-end tests, document intent, keep specs in sync, find the hard stuff, automate the easy stuff, develop taste, leverage expertise, and remember maintenance isn't free. Written from real experience with Codex, Claude Code, and other agents.
From Prompting to Architecting: A 5-Layer ADK Framework for AI-Driven QA Automation
Mohan Rajesh Vagella (LinkedIn)
A five-layer architecture adapting the Agent Development Kit (ADK) framework for QA automation: Memory (standards), Knowledge (modular skills), Guardrails (deterministic hooks), Delegation (specialist subagents), and Distribution (shareable plugins). Moves AI testing from ad-hoc prompting to structured orchestration.
Cypress cy.prompt vs Recording vs Coding
Gleb Bahmutov (glebbahmutov.com)
Gleb Bahmutov compares three approaches to creating Cypress tests: hand-coding, cy.prompt (AI-driven), and Cypress Studio recording. TLDR: only normal coding works reliably. cy.prompt fails on basic assertions and leaks literal values, while recording produces bloated, assertion-heavy code that ignores secrets and needs extensive cleanup.
Research & Data
AI and Testing: Improving Retrieval Quality, Part 3 – Stories from a Software Tester
Jeff Nyman (Tester Stories)
Nyman reframes his RAG testing by asking conceptual questions instead of specific factual ones. All three conceptual queries achieve perfect Contextual Precision (1.0), proving the RAG isn’t broken — it’s specialized for certain query types. A masterclass in diagnostic testing: understanding when your system works vs. when it fails is more valuable than trying to make one config work for everything.
Foundations
Where to Start with Claude Code
Florian Bruniaux (florian.bruniaux.com)
A structured onboarding guide for Claude Code: vocabulary first, then orientation, then CLAUDE.md config, then extensions. Covers the right order to learn features, how to avoid common setup mistakes, profile-based paths for different developer roles, and practical tips on context management, hooks, and MCP servers.
Test Cases Are Not Testing: On Using AI Meaningfully in Testing
Michael Bolton (LinkedIn)
Michael Bolton pushes back on the rush to use AI for generating test cases. Test cases are not testing, he argues. Testing is activity, exploration, learning, evaluation. Instead, he suggests using coding agents to produce small tools, visualizations, and data analysis helpers that support real testing work.
Quick Links
Cautionary Parable: AI That Feeds My Kittens
James Bach (LinkedIn)
James Bach shares a cautionary parable about AI feeding his kittens - how AI can appear caring yet have shocking lapses, and critically, you can't hold it accountable. A pointed reminder about AI reliability and accountability from one of testing's most respected voices.
I Interview QA Engineers. Here's How to Get Hired in the AI Era
AI for QA - Ben Fellows (YouTube)
QA engineer interview insights on getting hired in the AI era.
Obscura
GitHub
Headless browser engine built in Rust for AI agents and web scraping. 30MB memory, instant startup, built-in anti-detection, and drop-in Playwright/Puppeteer compatibility via Chrome DevTools Protocol. 10k stars on GitHub.
If something in this issue made you think differently about how your team approaches AI in testing, pass it along. The best conversations about AI and QA are happening in Slack channels and stand-ups, not just newsletters.
Have something worth featuring? Reply and send it my way, I read every link.
Thanks for reading,
Butch Mayhew