AI in QA - Issue #31
AI agents can now code, test, and fix their own failures. But an agent checking its own work is still grading its own homework. The QA win is deterministic tests that don’t care how confident the agent sounds.
AI agents can now code, test, and fix their own failures. But an agent checking its own work is still grading its own homework. The QA win is deterministic tests that don’t care how confident the agent sounds.
Automated testing hit the main stage this week, now called "verification." Coding agents aren't great at checking their own work, so teams are building trust through tests. That's a real opening for QA: help developers building their first test frameworks decide what's worth testing.
ThePrimeagen is leading Agentic QA for Omarchy, building a harness where agents drive a full Linux desktop in VMs. My question: will these tools capture what testers know, or just run checks? And is this the start of seasoned developers moving into testing?
The temptation with TypeSafe's Jev is to point it at assertions. Resist that. Assertions stay deterministic. The real fit is triage: the stack-trace judgment you do by hand every morning, now cheap enough for every failure.
Skills you never invoke still burn context every session. Claude Code has built-in cleanup commands. /skill-doctor shows the dead weight, /doctor clears CLAUDE.md debt.
The business minded tester already knows which bugs matter to the business. Point that same judgment at your AI tools. Does the token burn bring revenue in, or protect the revenue that's already there?
As I think about the best way to leverage AI in QA, things land into these 3 buckets. The first is generating deterministic automation code, this is especially helpful when working with enterprise software that have large user bases.
Sponsorship Interested in sponsoring AI in QA? Reach out. AI in QA Review Butch's Take This week I had a chance to watch
SPONSORED State of Test Automation 2027 — take the survey Test Guild is gathering data for the State of Test Automation 2027 report, and your input
Sponsorship Interested in sponsoring AI in QA? Reach out. AI in QA Review Butch's Take This week a peer asked how to get
Sponsorship Interested in sponsoring AI in QA? Reach out. AI in QA Review We're live again this week with Issue #21, if you
Sponsorship Interested in sponsoring AI in QA? Reach out. AI in QA Review We're back this week 7:00 AM CT on Monday!
Sponsorship Interested in sponsoring AI in QA? Reach out. Butch's Take "Confidence is the new coverage" runs through a few items
How QA Teams Are Making AI Work in Production SPONSORED The hard part of AI isn't getting a demo to work, it'
AI in QA Review This week and next week our livestream will be on Tuesdays at 7:00 AM CT. Be sure to subscribe to
AI in QA Review We won't be having a livestream this week due to PTO and scheduling conflicts. We will be back soon!
SPONSORED I'm excited to attend a webinar this Thursday, June 25th, 9:00-11:30 AM EDT, on using Cursor for QA work.
Sponsorship Interested in sponsoring AI in QA? Reach out. AI in QA Review Butch's Take If you haven't taken the poll,