AI in QA Review
This week and next week our livestream will be on Tuesdays at 7:00 AM CT. Be sure to subscribe to be notified of when we go live on YouTube!
The Framework Wars are Over
The context wars just started.
For a decade we argued about the tool. Selenium or Webdriver.io or Cypress or Playwright. That debate had larger ramifications, because a human wrote every selector, every wait, every assertion by hand, and the framework decided how painful that was.
Then agents started writing the tests, and the chosen tool is no longer the differentiator.
So the question stopped being "which framework." It became "what does the agent actually know about my system". Your CLAUDE.md or AGENTS.md file. Your feature map. The oracle that tells the agent what "correct" even means. That's the challenge that we will deal with now, and it's got nothing to do with the tool. It has everything to do with the context you can provide about the system under test.
Pick your framework, then stop thinking about it. Spend your real energy on what your agent knows before it writes uninformed test code which leads to rework and heavy steering. I've been experimenting with providing context through nested CLAUDE.md files that reference markdown files that contain context. So far I've had some good results, but I'm excited to push things further with local knowledge graphs like Graphify featured in the Tools & Frameworks section.
Headlines & Launches
Playwright with AI: Agents, Context, and Test Automation Patterns
Andrew Knight (LinkedIn Learning)
New LinkedIn Learning course (3h 58m) by Andrew Knight (Automation Panda). Covers building stable Playwright tests with AI-powered test generation, self-healing techniques, predictive test selection, and CI/CD integration. Released July 3, 2026.
The Framework Was Never the Hard Part. AI Changed That.
Dmitry Shyshkin (Practice Test Automation)
Argues the framework debate (Selenium vs Playwright) is a distraction. AI agents are the real shift, but they write tests that pass while checking the wrong thing. The skill that matters now: reading agent output as a tester, not a reader.
Introducing the Safari MCP Server
WebKit
Apple ships a Safari MCP server in Safari Technology Preview 247. Connects AI coding agents (Claude, Codex, etc.) to a live Safari browser window for DOM access, network inspection, screenshots, console logs, JS evaluation, and accessibility checks. Runs locally, no network calls.
Tools & Frameworks
Ocarina
GitHub
Automation framework for MCP servers. Write YAML playbooks that drive tools across servers, pipe values between steps, and assert on results. Run deterministic MCP server tests in CI with no LLM in the loop.
Graphify: Local Knowledge Graph for AI Coding Agents
Pardha Saradhi (LinkedIn)
Graphify builds a local knowledge graph of your codebase offline via tree-sitter, then wires into Cursor, Claude Code, or similar AI agents so they query the graph instead of re-reading files. Cuts token waste from repeated context-gathering on large test suites.
Custom Claude Code Commands for Software Testing
Madhawa Ratnayake (LinkedIn)
Open-source collection of custom Claude Code slash commands and skills for software testing. Designed to be used with Playwright MCP or Chrome DevTools MCP. Includes practical commands for test planning, spec generation, and developer-testing workflows.
Playwright 1.59 Screencast API
Alan Richardson (LinkedIn)
Alan Richardson (EvilTester) on using Playwright 1.59's new screencast API to record defect replication videos with chapter slide overlays. AI agents can record step-by-step evidence videos that make triage faster than screenshots.
Faultsense: Assertion Layer for E2E Testing
GitHub
Faultsense is a zero-dependency browser agent (17.7 KB) that evaluates end-to-end assertions against real user sessions in production. You annotate UI elements with fs-* attributes instead of writing separate test scripts. Assertions run wherever your app runs: staging via AI agents, production via real users.
If you are using Playwright and trying to figure out where AI fits beyond test generation, this is worth checking out.
Foundations
MCP, Explained for Beginners
Shriram Vasudevan (LinkedIn)
Beginner-friendly field guide to the Model Context Protocol. Covers the M+N integration problem, host/client/server architecture, the three primitives (tools, resources, prompts), transports, building a first server with FastMCP, security guidance, and the 2026 roadmap including AAIF governance under the Linux Foundation.
Techniques & Tutorials
A Practical Introduction to Testing LLMs
Demi Van Malcot (Ministry of Testing)
Practical guide to testing LLMs covering unit, regression, stress, behavioral, metamorphic, contextual consistency, adversarial, real-world use case, explainability, and bias testing. Includes examples and challenges for each technique.
How to Write a CLAUDE.md (and AGENTS.md) for Test Automation
Julia Pottinger (juliapottinger.com)
Practical guide to writing CLAUDE.md and AGENTS.md files for test automation suites. Covers the 5-step build loop: talk it out, mine corrections, have the agent draft from real code, prove it with tests, fold fixes back in. Includes a copy-paste template with checkable rules and a review fence.
Make Vibe Coding Safe: How to Test with Playwright
Max Schmitt (LinkedIn)
Max Schmitt's talk on making vibe coding safe with a small, high-signal Playwright suite that AI coding agents run on every change. Three specs for the journeys that matter, not thirty. Given at AI Council SF.
Trying Claude for E2E Testing on an Airbnb Clone
Ankur Tyagi (Medium)
Hands-on deep dive running Claude Code through every stage of E2E testing on an Airbnb clone: test planning, Playwright spec generation, parallel runs, GitHub Actions CI, and updating tests after UI redesigns. Claude excels at scaffolding but produces specs that silently pass for wrong reasons.
Research & Data
AI and Testing: From Ontology to Implementation
Jeff Nyman (Tester Stories)
Jeff Nyman uses a Z-Machine ontology to drive LLM code generation, then tests the output against real game files. The ontology serves as spec, test oracle, and consistency check across the entire pipeline. A practical demonstration of ontology-driven testing where structured knowledge becomes an active participant, not just documentation.
Quick Links
Vibe Coders Everyday Struggle
Akhilesh Mishra (X)
Relatable 1-minute clip capturing the daily reality of vibe coders. 5.5K likes and 432K views on X.
Running Local AI Models: A Practical Guide
Anne Cantera (LinkedIn) SOFTWARE TESTING ROUND-UP
A categorized guide to running local LLMs, covering user-friendly options (LM Studio, Jan AI, GPT4All) and developer tools (Ollama, Open WebUI, llama.cpp, LocalAI). Argues that recent events around AI platform access make local deployment a necessary fallback, not just an enthusiast option.
What’s Missing from This AI-Driven E2E Testing Workflow?
Reddit
Reddit thread in r/Playwright proposing an AI-assisted E2E workflow. A 15-year SDET shares field experience: Claude routinely skips failing tests, flags flaky tests as bugs, and can't think like a frustrated user. Where AI shines: API/contract testing, bug heatmaps from report backlogs, selector collection, and refactoring. Thread consensus: AI is useful for exploration and evidence, but QA judgment and a source of truth beyond the UI are irreplaceable.
If something in this issue made you think differently about how your team approaches AI in testing, pass it along. The best conversations about AI and QA are happening in Slack channels and stand-ups, not just newsletters.
Have something worth featuring? Reply and send it my way, I read every link.
Thanks for reading,
Butch Mayhew