Complete AI Training
Sign inGet my AI kit

Your job's AI kit

Get your AI kit

Tell us who you are and what you do. We show you your kit right away and email you the link: skills, prompts, AI agents, MCP servers and courses for your job.

500+ jobs ready, and we make a kit for any other job. No payment needed to look.

Share

AI agent for qa managers

Device and OS Test Matrix Agent

Cover the devices and OS versions that matter and confirm the app works on each

Quality controlIT and developmentProduct development
See how it worksGet access
Device and OS Test Matrix Agent: what goes in, what the agent does and what you get

What it does

The app works on the developer's phone and fails for 6% of users on an older Android version or a small screen. This agent builds a test matrix from real usage data: the devices, OS versions, screen sizes and languages that cover the most users, plus a few risky ones such as the oldest supported OS. It runs the smoke tests on a device cloud or a pool of real devices, collecting screenshots and logs. It lists failures by device and groups them by cause. After the developer fixes them, it reruns the failed ones and a sample of passing ones. It also updates the matrix each month from new usage data. The developer approves the release. Edge case: a failure happens on one device only because of low storage, so the agent marks it as a device issue.

How it works

Follow the arrows from top to bottom. The orange dashed arrow is the loop: when a check fails, the agent goes back and tries again.

Start and resultWhat it doesA check on its own workWaits for your OKGoes back and retries
Yes, continueYes, continueApprovedNoNo 1 STARTS WHEN Release candidate or monthly refresh 2 USES A TOOL Read usage data and rank devices and OS versions 3 DOES Build the matrix to cover 90% of users plus riskdevices 4 USES A TOOL Run smoke tests on each matrix device 5 DOES Collect failures with screenshots and logs 6 CHECKS THE RESULT Does each failure repeat on a second run on thatdevice? If not: mark it as a device issue and rerun on a similardevice. Back to step 4. 7 DOES Group real failures by cause and send to thedeveloper 8 USES A TOOL Rerun failed tests and a sample of passing onesafter the fix 9 CHECKS THE RESULT Do all matrix devices now pass? If not: send remaining failures back with new logs. Backto step 7. 10 YOU APPROVE Developer approves the release 11 RESULT Matrix coverage and result report
Read the steps as a list
  1. Release candidate or monthly refresh
  2. Read usage data and rank devices and OS versions
  3. Build the matrix to cover 90% of users plus risk devices
  4. Run smoke tests on each matrix device
  5. Collect failures with screenshots and logs
  6. Does each failure repeat on a second run on that device?If not: mark it as a device issue and rerun on a similar device. Back to step 4.
  7. Group real failures by cause and send to the developer
  8. Rerun failed tests and a sample of passing ones after the fix
  9. Do all matrix devices now pass?If not: send remaining failures back with new logs. Back to step 7.
  10. Developer approves the releaseThe agent waits here for your OK.
  11. Matrix coverage and result report

How it decides

It selects devices until a target share of users is covered, adds risk devices such as the oldest supported OS and smallest screen, and treats a failure as real when it repeats on the same device.

  • Cover at least 90% of active users
  • Always include the oldest supported OS and the smallest screen
  • Count a failure only if it repeats twice on the same device
  • Rerun 20% of passing devices after any fix

Make it yours

Every agent is a starting point. You choose these settings for your own situation.

  • User coverage target (default 90%)
  • Risk devices to always include
  • Smoke test list
  • Device cloud provider
  • Refresh schedule (default monthly)

What keeps you in control

It always asks you first

  • Developer approves the release
  • Product owner approves dropping support for a device

Hard limits

  • Never use real customer accounts in the tests
  • Never release based on a partial matrix

It stops when

  • Done: all matrix devices pass
  • Stop: a failure on a major device has no fix and the release is held

Set it up

We guide you through the set-up, step by step

Members get the full set-up guide for this agent. No technical skills needed: you copy, paste and upload.

10 minto set it up in your AI
5 AIsChatGPT, Claude, Copilot, Gemini, Grok
  • One set of instructions to paste into your AI, with the clicks for ChatGPT, Claude, Microsoft 365 Copilot, Gemini and Grok
  • The agent then walks you through connecting your own data, one source at a time
  • A downloadable copy with the flow chart, the rules and the full guide
Get access to this agent

An example run

What happensThe agent built a 14-device matrix covering 91% of users. Smoke tests failed on 3. A Galaxy A12 on Android 10 crashed when the keyboard opened. A repeat run confirmed it. A second failure was a low-storage device error, marked as a device issue. After the fix the rerun still showed a layout break on a 5-inch screen, so the check failed. The next round passed all 14 and the developer approved.

More agents for qa managers

Quality control · Coding

Public Form Accessibility Agent

Every public form remains accessible after each revision.

7 steps2 checks5 jobs
Coding · Automation

Flaky Test Reproduction Agent

Every flaky test ends with a reproducible cause or a documented set of ruled-out causes.

9 steps1 check3 jobs
Coding · IT and development

Accessibility Regression Repair Agent

Keyboard and screen-reader journeys keep working after UI changes.

7 steps2 checks6 jobs
Coding · IT and development

Test Fixture Data Minimization Agent

Fixtures with no unnecessary personal data and all tests passing.

7 steps1 check2 jobs