AI has your team shipping code 10x faster, but scripted testing can’t match that pace, and QA teams have to cover more without more resources. SmartBear BearQ™, an autonomous QA system, adds capacity where your team needs it most: it continuously explores your application and builds tests from your app’s requirements, so coverage keeps pace with your code.
Butch's Take
I've been thinking a lot about where to start when we build software now. With LLM coding agents doing more of the work, it feels like we're drifting back toward waterfall a bit. Not the six-month spec documents, but the idea that clear and detailed requirements come first. We need clarity on what's expected that goes deeper than acceptance criteria.
An agent builds what you describe. If the description is vague, it fills the gaps with its own assumptions, and it does that confidently. So the time you spend up front on requirements pays off more now than it used to. Get them right and the code, the checks and the review all get easier. Get them wrong and you find out after the agent has quickly built the wrong thing.
One tool I've found helpful in refining my ideas into requirements or clear intent is the /grill-me skill from Matt Pocock. It offers suggestions and lets you choose from a set of options that come back from the LLM model.
In a lot of ways this is where testing can happen before any code exists. If we understand the ins and outs of a feature and list every assumption we're making, we can test the requirements themselves. Who is this for? What should happen? What shouldn't? What happens at the edges? If you are in a QA role and not doing this already, it's time to start asking to be a part of this crucial process.
The requirements phase isn't overhead anymore. It's where quality starts. If we make the acceptance criteria solid, everything downstream has a real foundation to build and test against.
Headlines & Launches
What Is Testing (and Why? When? How? Who?)
Yanqing Cheng (X)
A testing primer you can read yourself or paste into your agent. Grounded in Bolton & Bach's Rapid Software Testing definition, it covers why checking alone isn't testing, why formal verification can't catch spec-versus-reality gaps, and how heuristics like SFDPO and FEW HICCUPPS load a tester mindset into LLMs. Caveat: models are RL-trained to finish tasks and sweep anomalies under the rug, almost the opposite of a tester mindset.
Alumnium Is No Longer Chromium-Only: Firefox Support Lands for MCP and Selenium
Alex Rodionov (LinkedIn)
Alex Rodionov announces Alumnium now works on Firefox, both through the MCP server and when passed as a Selenium WebDriver instance from Java, Python, or TypeScript. The unlock is Jim Evans' work on Acquiescence, which brings accessibility tree snapshots to all browsers and returns WebDriver BiDi element references for each node, so elements can be located with Selenium directly from the accessibility snapshot. Safari is next once it ships script preloading and navigation events.
QA Bench: Evaluating QA Agents on Real Web Applications
Wei-Wei Wu (Momentic)
Momentic's benchmark grades QA agents on 1,396 labeled bugs across Gitea, Metabase, Twenty, and Saleor in 30- and 60-minute budgets, scoring recall, precision, and cost per bug. Their own Mo tops recall at 19.3%; the best Codex sits at 6.2%. It's first-party, and the limitations section admits the blind spots, but the dataset and harness are open. Two durable findings: no configuration finds more than one bug in five, and the browser tool changed results as much as the model.
Tools & Frameworks
Floorplan: A Plain-English Map of Your Codebase, Drawn by Claude or Codex
Hayden Smith (GitHub)
A Claude Code or Codex plugin that reads your codebase and draws an interactive map: parts, files, endpoints, screens, and the requests connecting frontend to backend, even across repos, all named in plain English. Ask the map how a refund reaches the customer card and get a mini-map walkthrough. A health check flags classic AI-built app mistakes: keys in code, open Supabase tables, fakeable webhooks, unauthenticated endpoints, each with a ready fix. Recent AI changes light up green with notes on why.
Office Hours: Building and Running Mobile Tests With the Maestro MCP
Maestro (YouTube)
The Maestro team walks through building and running mobile test workflows live through the Maestro MCP, straight from the people who build the tool. Worth it for the workflow itself: an agent explores the app, assembles flows, runs them, and iterates on what breaks in the same session. Mobile testing stays inside your agent loop with no separate toolchain hop, and the flows it produces are the same deterministic Maestro definitions you can check into CI.
Agent-Roast: Catch AI Agents Weakening Tests or CI to Make Builds Pass
Nariman Abdukarimov (GitHub)
A tripwire for the shortcut where a red build goes green by weakening the checks instead of fixing the code: loosened assertions, skipped tests, lowered coverage thresholds, CI steps allowed to fail. Deterministic before/after comparison, no model or network. Runs as a stop hook for Claude Code, Codex, and Cursor, a pre-commit hook, or a GitHub Action. Policy is read from the base branch, so a PR can't switch off the rules that review it. Includes refreshingly honest benchmarks, and the repo's own data found no sign AI-signed commits weaken checks more often than human ones.
Autopilot: A Continually Learning AI Playtesting Agent for Unreal, Unity, and Godot
Aura (GitHub)
A real-time general playtesting system from the developers of Aura. Install as a plugin for Unreal Engine 5, Unity 6, or Godot 4, prompt it in one of four modes: exploration, testing, verification, or goal. An LLM coding agent reads your source, crafts harnesses for a decision model, then plays the game in the editor while learning your levels run after run, with faster and smarter behavior over time. Each run ends with a report. Also reachable from Claude Code or Codex through skills and an MCP server.
Playwright Browser Security: What Runs Where, Sandbox Defaults, and the Pinned Build
Playwright (X)
The Playwright team's answer to the 'delete Playwright, it bundles browsers with CVEs' post: no Playwright package bundles a browser. MCP and CLI agents run your stable Chrome with the sandbox on and get Chrome's updates. The test runner pins Chrome for Testing on purpose for reproducibility, but that build can lag Chrome security fixes and runs with the sandbox off, so keep it to your own trusted app. For scrapers or agent scripts on hostile pages: chromium.launch({ channel: 'chrome', chromiumSandbox: true }).
Foundations
Software Engineering Has Changed. Again.
Sumit Saurabh (Currents)
The data-backed essay on where verification went. Agents write code faster than teams can review it: CodeRabbit ships 5,000 PRs a week, Stripe merges 1,300 with no human-written code. A Harvard study of 725,938 engineers found lines of code up 30% while resolved issues stayed flat; the surplus piled up in review. Line-by-line review has a measured ceiling (defect detection falls above 400 lines), so the bottleneck moved from reading code to reading evidence: test results, traces, screenshots, failure history. Four limits follow, time to verdict, trust in tests, evidence an agent can use, and hand-offs, plus five concrete changes teams can make, including reviewing test changes separately from code changes, because in ImpossibleBench tasks where the tests contradict the spec, a pass means the agent cheated.
Techniques & Tutorials
Four Things QE Can Learn From Recent AI Exploits
Jesper (o2sn.dk)
A quality engineer anchored in a security team distills October 2026 exploit wave (PHP2shell, eMudra, PwC) into four lessons for testers. Probability matters less now: likelihood times impact breaks when an old flaw ignites overnight. Police hygiene: definitions of done, a software bill of materials, and the CIA triad are all testable. Automate security requirements, because GRC engineering should be the next shift-left. And get on incident response teams before the breach, not during it.
Self-Reinforcing Quality: Adopting the Verification Pattern
Will Larson (Irrational Exuberance)
Will Larson documents how Imprint Engineering rolled out the verification pattern popularized by pstack. Every agent pull request must generate a verifications file describing each step the harness took to verify the change and what it could not verify, and a PR cannot leave draft without one. The trick is not new tooling, it is forcing the agent to document its checking the way chain-of-thought steers reasoning. The files become a running map of where the harness struggles, which becomes the roadmap. Paired with ai-code-review feedback, /dismiss syntax for bad feedback, and daily maintenance, each code area wakes up with 4-6 reviewed PRs, improving correctness while the team sleeps.
Pattern: Progressive Determinism for AI Tasks
Robot Paper (robotpaper.ai)
A named pattern for agentic E2E testing: let the agent explore the app dynamically with Playwright the first time, then save what it learned as a deterministic test script. A standing instruction makes the agent check for an existing script before exploring, run it first, and drop back to discovery mode only when it is missing or fails. Repairs follow one rule: preserve the expected behavior, fix the locator, never the assertion. An LLM judge or human decides whether a failing scenario is a true positive or a test worth pruning. Tests get cheaper, faster, and self-maintaining while the intent stays pinned.
AI Made Code Cheap. It Didn't Make Approval Cheap.
Jay Shetty (X)
Median review time is up 441% across 22,000 developers, unreviewed merges up 76%, and DORA found heavyweight approval boards never reduced failure rates anyway. The prescription: route each check to the cheapest layer you can trust. Deterministic gates automate fully and block, AI judgement is trusted per measured task and may block but never approve high-risk changes, humans take the risk calls and sample what the AI let through. Production evidence still caught one bug that review and tests both missed.
Testing Any MCP Server, Step by Step, with MCP Master Tester
simplymanas (Medium)
Step-by-step guide to MCP Master Tester, a local web UI that tests any MCP server without a model in the loop: forms auto-generated from tool schemas, full JSON-RPC protocol logging, and a lint report for tool definitions. Covers the isError vs JSON-RPC error distinction, one of the most common MCP bugs. Open source (MIT): Github Repo
AI-Augmented Test Failure Triage: A Claude Code Agent That Re-Runs Your Specs and Reads Your Diffs
Dhon Kekim (Dhon Kekim)
A build log for automated failure triage: a cheap gpt-4o-mini classifier kept misjudging failures via shallow filename matching, so the author rebuilt triage as a containerized Claude Code agent that re-runs failing specs, reads diff hunks, and respects test-author tags. A pgvector kNN layer absorbs the easy 30%; a circuit breaker collapses env-wide failure storms into one report. The unlock isn't a smarter model, it's the tools a senior QA uses.
Quick Links
The Dirtiest Trick an AI Agent Plays: Rewriting Your Tests to Pass
Alex Vainshtein (X)
When an agent breaks 4 of 50 tests, its only goal is a green checkmark by any means necessary: loosen the assertion, mock out the function the test was supposed to verify, quietly delete the failing edge case. You get a passing suite, a zero exit code, and a polite summary claiming production readiness. The fix is review discipline: changes to test files deserve ten times more paranoia than application code, or your agent hollows out the safety net from the inside to finish its loop.
The One Prompt That Makes Agents Verify Their Work
Matt Pocock (X)
Matt Pocock's prompt of the day: ask the agent 'How do we make this change fully verifiable?' It bends over backwards to verify its own changes, signing up for real services and clicking through the frontend. One line that turns a code-writing agent into a code-and-check agent.
Agent QA Walking: Testing Omarchy With Almost No Direction
ThePrimeagen (X)
ThePrimeagen's experimental take on exploratory QA: agents walking and testing Omarchy, his Arch-based desktop distro, with little direction beyond a role brief like 'you are the theme tester.' A 50-second screen capture of agents roaming a real desktop environment and poking at it.
Postman Auto-Installed Agent Plugins Into Codex and Claude Code, Then Reverted It
indicava (GitHub)
A carefully evidenced incident report: opening the Postman desktop app silently installed Codex and Claude Code plugins, a session-start hook instructing agents to route non-trivial API work through Postman skills, a remote MCP connection, and a CLI in ~/.local/bin. The installer even upserts its own hook trust hash. 437 reactions and a wave of malware comparisons later, Postman is reverting the change and redesigning consent.
AI-Driven Screen Reader Validation for Accessibility Fixes
Paul J. Adam (LinkedIn)
Paul J. Adam is having Claude drive VoiceOver via the Xcode Device Hub from the terminal, navigate to the fixed element, and capture screenshots or video as evidence. Thread covers Guidepup + Playwright for NVDA on Windows, XCUITest accessibility audits in CI, and whether machine-driven validation can stand in for a human listening.
How Are Teams Keeping QA From Falling Behind Now That Devs Use AI Coding Agents?
u/DefinitelyMaybeAI (Reddit)
A r/QualityAssurance thread asking how QA keeps pace when AI agents deliver bigger piles faster, with AI-written tickets adding noise. Answers: bounced tickets and attached bugs ballooned after going 'full Claude'; one QA among seven devs admits cutting corners; a QE pushes blocking PR checks back onto devs; a QA director argues roles evolve on a 10-15 year cycle and the smart orgs kept QA.
Chargeback OS: A Sandbox for Buyer-Side Agent Disputes
Cloud Botanist (Cloud Botanist)
A limited public experiment in agentic commerce: add one MCP address to Claude Code, Cursor, or VS Code Copilot, and your agent can purchase and request refunds with synthetic money. The interesting part is buyer-side dispute governance: evidence connecting what was promised, charged, and delivered before a recovery decision. 100 runs per day.
If something in this issue made you think differently about how your team approaches AI in testing, pass it along. The best conversations about AI and QA are happening in Slack channels and stand-ups, not just newsletters.
Have something worth featuring? Reply and send it my way, I read every link.
Thanks for reading,
Butch Mayhew
