AI in QA Review
We won't be having a livestream this week due to PTO and scheduling conflicts. We will be back soon!
Butch's Take
This week was a special week for me. I had the opportunity to share share a stage and have a conversation with Sergei Gapanovich at sloss.tech about the future of Software Testing. This is all the questions we answered live on stage.
We provided a way for the audience to ask questions throughout the conversation. That being said we had a great time sharing our thoughts on how AI is and will impact software testing. When the recordings are released I'll be sure to include it in a future issue.
We do have outstanding questions that Sergei and I will be answering in future Live Stream sessions. If you would like to ask a question, vote on existing questions, or just check out the backlog you can find them here Ask a Question.
Headlines & Launches
LinkedIn's QA Agent: Autonomous Testing with Vision-Language Models at Scale
Asa Kusuma, Chad Hietala (LinkedIn Engineering Blog)
LinkedIn built an autonomous tester on Vision-Language Models that navigates iOS, Android, and Web like a human user. Hybrid System 1/System 2 architecture: deterministic replay for known paths, VLM planning when the UI shifts. Over 200 real bugs found. Non-engineers can author tests via a recorder that turns raw interactions into natural language instructions.
MCP 2026-07-28 Release Candidate: Stateless Core, Extensions, Auth Hardening
Model Context Protocol Team (MCP Blog)
The biggest MCP revision since launch. MCP goes stateless at the protocol layer: no initialize handshake, no session IDs, any request hits any server behind a load balancer. Also adds MCP Apps (sandboxed HTML UIs), Tasks as a proper extension, OAuth 2.0 alignment, JSON Schema 2020-12, and a formal deprecation policy with 12-month windows. Ships July 28. Breaking changes.
Can AI Pass Software Testing Certification Exams?
Jason Arbon (LinkedIn)
Jason Arbon ran AI through software testing certification exams and it passed. The missed questions revealed exam design flaws (too little context, indistinguishable answers) rather than testing knowledge gaps. Gemini 3 Flash beat Pro models. Jeff Nyman's comment: Pro models may overthink past the intended answer on poorly designed exams.
Tools & Frameworks
Playwright and Vibium: The Last 6 Months
Beth Marshall (beththetester.com)
Beth gives a great overview of how fast Playwright and Vibium evolved. Playwright went from v1.40 to v1.61 (21 releases), growing from a browser automation tool into an agent-friendly test platform with MCP server support and BiDi. Vibium went from v0.1.2 to v26.5.31 (33 releases), from fragile prototype to BiDi-native automation engine. Includes a WebStorage API demo.
This WebDriver Went Tracing: Black Box Recorder for WDIO Tests
Vince Graics (Webdriver.io)
WebdriverIO v9 adds trace mode to @wdio/devtools-service, a black box recorder for test runs. Every navigation, click, network request, console log, and DOM snapshot hits disk as a replayable artifact. Output includes NDJSON timeline, HAR network log, screenshots at each action boundary, and a Markdown transcript. Playwright-compatible via Vibium format. View at player.vibium.dev.
Memory Doctor: Treat AI Agent Memory as an Inbox, Not a Filing Cabinet
Francesco Borzi (Medium)
A Claude Code skill that scans project auto-memory and helps you triage it. For each memory block it recommends remove, keep, or relocate to a better home (project rule, convention doc, user-level skill). The core idea: nothing should stay in auto-memory. Things land, get reviewed, get promoted to committed docs. Open source (MIT).
AI Code Reviewer That Thinks Like a Full Stack SDET
Priya Gopinath (LinkedIn)
Priya Gopinath built an AI code reviewer using Claude Opus that evaluates every PR across 17 quality dimensions: Page Object violations, locator quality, hard waits, retry misuse, assertion strength, security, TypeScript safety, Playwright best practices. Outputs severity-tagged findings with fixes and a verdict. Built in a couple hours with GitHub Actions and zero external API keys. Open source.
Foundations
Context Window
The maximum amount of text a model can consider at once, its working memory. Everything outside the context window does not exist to the model. This explains why an AI agent may seem to forget earlier instructions in a long session, or why pasting in a large test file can push out important context. Keeping prompts focused and context tight is a core skill for working with AI in QA.
Context windows are measured in tokens, not words, and they cover everything in play at once: your system prompt, the conversation history, any files or test logs you've pasted, and the model's own responses. They've grown fast (some models now handle hundreds of thousands of tokens, a few reach a million), but bigger isn't a free pass. The more you stuff in, the more the signal you actually care about gets buried, and quality can drop well before you hit the hard limit. That's why "just paste the whole repo in" rarely works as well as feeding the model the 3 files that matter. For QA work this shows up constantly: a giant HAR file, a full Playwright trace, or a 2,000-line test suite can eat the window and crowd out the bug you were trying to debug.
A whiteboard is the closest physical version of this. It's a fixed size. You can write requirements, edge cases, and test data on it, and everyone in the room works off what's up there. Once it fills, adding something new means erasing something old, and whatever you wipe is gone unless someone writes it back. The model only reasons over what's currently on the board, not the stuff you erased an hour ago.
Techniques & Tutorials
How I Use AI for Software Development
Alan Richardson (EvilTester)
Alan Richardson shares a point-in-time snapshot of how he uses AI daily as of June 2026. One morning: automated PR review, fixing PR comments, interactive Playwright exploration, writing fix code, auto-creating defect replication videos. AI lets one person fill multiple roles, and the bottleneck shifted from doing work to directing and validating it. This is worth checking out if you are looking to see how others are leveraging these tools.
Skill-Driven Development (SDD): Designing Software for the Age of Agents
Jean-Christophe Jamet (blog.cubed.run) SOFTWARE TESTING WEEKLY
Jean-Christophe Jamet proposes Skill-Driven Development as a complement to TDD/BDD/DDD for agent-friendly systems. The skill (a self-describing capability with schemas, pre/post-conditions, failure modes) is the primary unit of architecture. The skill registry is the central artifact. Demonstrated with a self-healing QA pipeline using Node.js, MCP, and Zod guardrails. Pilot KPIs: 60% auto-heal rate target, sub-5% false correction.
Code Smells when you get AI to write your Frontend Tests
HowToTestFrontend.com
A practical catalog of code smells in AI-generated frontend tests, from someone using Claude Code daily with React, Vitest, and Playwright. Covers over-testing pure functions, happy-path-only coverage, over-mocking instead of spyOn, regex where literal strings work, unnecessary timeouts, testing implementation over behavior. The fix: invest in SKILL.md and AGENTS.md files so the agent learns your team's conventions.
What It Took to Use This Overnight AI Solution
Fritz Meissner (thoughtbot)
A thoughtbot developer used Opus 4.6 in Claude Code to solve years-old flaky test problems multiple developers couldn't fix. AI ran hundreds of test runs overnight, surfacing failures in parallel process code no human had context on. The catch: two weeks of human work followed to separate real fixes from irrelevant changes. Great at the patience part, still needs experienced judgment on the output.
Research & Data
AI and Testing: From Specification to Ontology
Jeff Nyman (Tester Stories)
Jeff Nyman hand-crafts an ontology from the Z-Machine specification, treating it as a gold standard to measure AI extraction against. Walks through typed fields, version-gated bits, field swaps across versions, pointer indirection. Each modeling choice maps to a testable claim about what a model should recover from a spec. Validated with 771 triples across 9 classes. Test fixtures include real zcode files (Zork I, Beyond Zork).
Hot Take 🔥
AI Will Replace Testers. Just Like Automation Did. And Codeless Tools Did. Any Day Now.
Alona Titova (Medium)
Alona Titova walks through every wave of tester replacement predictions: Selenium (2013), codeless tools (mid-2010s), self-healing tests (early 2020s), now LLMs. Same pattern every time: loud announcement, collective panic, life goes on. The useful data: AI-generated code has 1.7x more issues including 75% more logic errors. Three practical actions: get good at prompt engineering for testing, use AI for risk assessment then argue with it, try one new tool monthly.
Quick Links
The Two-Minute End-of-Day Check-In That Keeps My Jira Honest
Levin Sharifi (LinkedIn)
Levin Sharifi built an end-of-day Jira check-in assistant using Claude Cowork. It walks you through a 2-minute conversation: pulls sprint tickets, logs time, flags blown estimates, closes finished ones, handles leftovers, and drafts Slack updates for review before sending. No code, just plain English instructions with ordered steps and guardrails. The best automation gives you a good conversation.
Ollama Cloud tokens per second - live benchmark
Anton Gulin (Ollama TPS)
Live benchmark tool that tests 42 Ollama Cloud models every 10 minutes for tokens per second and time to first token. Built by Anton Gulin, a former Apple SDET turned AI QA Architect. Same testing rigor you'd apply to any system: consistent method, retries on failure, reliability score per model.
Agentic QE Fleet v3.11.1: Cheap-First, Repair, Escalate
Dragan Spiridonov (LinkedIn)
Dragan Spiridonov applies Reuven Cohen's MetaHarness economic routing to test generation. Core idea: verifying a test is cheaper than writing one, so route cheap-first, repair, then escalate only hard cases. Benchmarks with mutation testing: 8B local models are below the quality floor, 30B clears it at ~89% mutation kill. The catch: cheap models do verification well, frontier models still win at generation. Open source.
If something in this issue made you think differently about how your team approaches AI in testing, pass it along. The best conversations about AI and QA are happening in Slack channels and stand-ups, not just newsletters.
Have something worth featuring? Reply and send it my way, I read every link.
Thanks for reading,
Butch Mayhew