Explore & Automate: AI Explores your Web App, Finds Bugs and Writes Verified Tests

Point Testver's Explore & Automate skill at your web app. It explores by clicking, as each role you name, records what is broken, then writes tests in your framework, keeps only the ones that pass twice, and links each to a test case.

T
The Testver team · 15 min read

Writing the first fifty tests for an application nobody has documented is slow work. First you click around to learn what the app does. Then you write down the journeys, find out which pages each role can reach, notice the things that look broken, and only then start writing tests.

Explore & Automate does that whole cycle for a web application, from a single prompt in the Testver AI Assistant. It walks your app the way a user would, by clicking, as each role you name. It records every page and the clicks that reached it, notes anything that looks wrong, and then writes a test for every journey it found, in your own framework and style. A test is kept only when it passes twice, and each one gets a matching test case linked to its script.

This post walks through a real run, on a local copy of our practice shop with a few clearly fake bugs planted for the demo, and covers every part of the skill: the questions it asks, how it signs in, what it counts as a finding, how it stays safe, the tests it writes and what you can do with everything it recorded afterwards.

What you get at the end

Output

Where it lands

Automated tests

In your project, in your existing framework and style, each one run until it passes twice.

Test cases

One in Test Management for each automated journey, already linked to its script and to your requirement.

Findings

Problems it actually hit while exploring: text that never rendered, failed requests, console errors, dead ends, and disagreements with your documents. Deduplicated and ranked.

An access matrix

When you name more than one role: which pages each role reached, and which it was blocked from.

A report

Tables of tests, findings and coverage, plus a plain statement of what it did not do, and why.

App knowledge

Everything it observed, which Ask mode and AI Test Generation use afterwards.

It works with Playwright, Cypress, Selenium and WebdriverIO projects. It is for web applications only: it needs a real browser, so it runs in the AI Assistant's Web mode. For mobile apps, use the AI Assistant's Mobile mode or record a journey with AI Record.

Turn it on and pick it

Explore & Automate is a built-in skill. If it isn't switched on yet, an administrator enables it in Settings → Skills. Its detail view states its first rule plainly: it requires Web mode.

Then, in the AI Assistant:

  1. Make sure the mode picker says Web.

  2. Type / at the start of the message box to open the skill picker.

  3. Choose Explore & Automate. It sits as a chip above the message box.

  4. Say what to explore and send.

The first message can be as short as this:

Explore and automate https://staging.example.com

One round of questions, then the plan

Before it touches your app, it reads your project: the framework, the page objects, the helpers and any earlier runs. Then it asks everything it still needs once, in a single numbered list, and only what you haven't already told it:

It asks

Why it matters

The URL, and whether it's staging or production

Production means it explores read-only.

Scope: which origins and paths are in bounds

External links are always out of bounds.

Restrictions: anything off-limits

"Don't delete anything", "don't change passwords", "stay in the demo tenant".

Roles, from least to most privileged

Optional. With none, it explores as a guest, which is a real run.

Documents: a PRD or requirement

Optional. With one, it can tell you where the app and the document disagree.

It never asks for a password in chat. In our run, the answers were: a local copy of staging, so writes are fine; the shop, sign-in, profile, settings and the admin dashboard; no password or settings changes, nothing deleted, and stay out of /debug and /api; three roles, guest, manager and admin. With a #, we attached the "Shop coupons and checkout" requirement from Sources.

Its first write is the plan, saved as plan.json in its run folder. It reads that plan again at every phase, so your rules hold for the whole run, not just the first few minutes. No browser opens before your yes.

Tip: you can put all the answers in your first message and skip the back-and-forth. Restrictions are worth writing even when they feel obvious. One line like "stay in the demo tenant" is the difference between a run you can leave alone and one you have to watch.

Exploring by clicking, like a user

Once you say yes, the live browser opens beside the chat and you can watch every step. Two habits make its exploration useful:

  • It clicks; it doesn't type URLs. Only the entry address is typed. Every other page is reached by clicking a link from where it already is. Typing a URL skips whatever the page depends on, such as a redirect, a filled cart or an earlier wizard step, so a page that only breaks when you arrive through the app would look fine.

  • It records the path, not just the page. The clicks that reached each page are saved, because that path becomes the test later.

It also declines cookie banners (essential only), takes a screenshot of every page, and treats a page it can't get out of as a finding.

In our run, it started as a guest. On the cart it checked each coupon rule from the requirement: an expired code is refused, an unknown code is refused, and WELCOME20 takes exactly 20% off (it checked the arithmetic). It walked the five checkout steps with made-up details and the sandbox test card, confirmed the order was placed and the cart was empty afterwards, then checked what a guest can reach: the admin dashboard, the profile and the settings all sent a guest to sign in.

Signing in: you type, it waits

When it needs a role it doesn't hold, it goes to the login page and pauses. An "Over to you" card appears in the chat. You sign in yourself in the live browser, including any two-factor step, then click Continue. It checks the sign-in really took (it refuses to save a login that left you on the login page) and carries on as that role.

Sessions are saved so it doesn't ask again:

  • encrypted on disk, inside your project;

  • tied to the site they were captured on, so they're never replayed against another;

  • expired after seven days.

It checks for a saved session before asking you to sign in. In our run it had an older manager session, restored it, noticed the header still said "Sign in" because the app had been restarted, and handed the browser over instead of carrying on as a guest under the wrong label.

The tests it writes never contain a password. When a test needs to sign in, it reads the credentials from environment variables, so your test files stay safe to commit.

What counts as a finding

Findings come only from things it actually observed:

Source

Examples

The page's text

A rendered undefined, NaN, [object Object], unrendered {{placeholders}}, raw error strings.

The page structure

Controls with no accessible name, error pages, broken links, dead ends.

The browser

Console errors, uncaught exceptions, failed requests, 4xx and 5xx responses.

Your documents

Where the application disagrees with a requirement you gave it.

The same console error on forty pages is one finding with a count of forty, not forty findings. A disagreement with your document is labelled possible spec drift: a question for you, never a reason to "correct" your document.

In our run, the profile said "Member since: undefined" where the requirement asks for a month and year, and the coupon SAVE10 took 15% off where the requirement says 10%. Both were recorded, with evidence.

Nothing is filed as a defect for you. Findings are proposals. Tell the AI Assistant which ones are real defects and it creates them for you, or raise them yourself on the Defects page.

More than one role: the access matrix

With two or more roles, it finishes the pages as the first role, then re-walks the same pages in the same order as each other role. That's what makes the runs comparable. The result is an access matrix: every page against every role, marked reachable, blocked or errored.

From that matrix the report derives privilege inversions: a page a lower-privileged role can reach that a higher-privileged one can't. That's arithmetic over what was observed, not an opinion. Our run found one: the admin got "Access denied" on the admin dashboard, the page the less-privileged manager could open. It retried with fresh state to rule out a glitch before recording it.

Staying safe

The skill's safety rules aren't advisory, and your restrictions sit on top of them:

Rule

What it means

Your restrictions are the rule

Applied literally, and re-read from the saved plan as the run goes on. In our run, it only looked at the account settings page, because our rule said no settings changes.

Stay in scope

It never follows an external link.

Irreversible actions need a decision

Anything that destroys data, spends money or contacts a real person. If your restrictions don't cover it, it asks you once, records the answer and follows it for the rest of the run. Silence is never a yes.

Synthetic data only

Made-up names and addresses, sandbox cards, never a real card number.

Production means no writes

If you said production, it explores read-only, and the report says so.

It won't log itself out

That would end the session you gave it and quietly turn the rest of the run into a guest run.

Still, point it at a test or staging environment. It fills forms and places orders; the read-only rule is a safety net, not the plan.

Writing the tests: every journey, green twice

When exploration is done, it moves straight to automation. Selecting the skill is the request for the whole cycle.

  1. The queue comes first. Before writing a single test, it queues every journey it found and marks each one critical or normal. That order decides what it works on first, never what gets a test.

  2. Your style. It writes in your project's framework, with your page objects, helpers and fixtures. Before it changes a shared helper, it checks who depends on it. In our run twelve tests did, so it kept its change additive.

  3. One navigation, then clicks. Each test replays the recorded click path from the entry URL, instead of deep-linking to each page.

  4. Scoped runs. It runs only the test it just wrote, never your whole suite, fixes what fails, and runs it again.

  5. Green twice, or quarantined. A test is kept only if it passes twice. One that won't is quarantined with its failure attached rather than shipped as if it worked.

  6. A test case for each green test, marked automated, mapped to its script and linked to your requirement.

The quarantine rule matters. In our run, the SAVE10 test failed, because the app gave 15% where the requirement says 10%. The test was right and the app was wrong, so the test stayed red as the record of that defect instead of being weakened until it passed. The run ended with 14 tests written: 10 automated and 4 quarantined, each of the four pointing at a real defect in the app.

When a run stops: say continue

Everything it learns goes into a run folder in your project, .testver/explore/<run id>/: the plan, each page, the findings, the access matrix, the automation queue, the tests and the report. That's what lets a stopped run resume.

A long run can stop on purpose: near its working limit it writes a checkpoint and tells you how far it got. It can also stop for other reasons. Ours stopped in the middle of creating the test cases, when the AI credits ran out. After a top-up we typed one word, continue. It read its run folder, found the five green tests that still had no test case, created them and finished the report, without starting over.

If you close the tab mid-run, the run keeps going on the server until it needs your approval. Then it stops cleanly rather than guess at a prompt it can't show you. Nothing is lost: say continue and it picks up from disk.

The report

The run ends with tables, not prose:

  • Automated tests: each scenario, its test case ID, the spec file and test name, and how many runs passed.

  • Findings: worst first, with how often each was seen, the evidence, and drift labelled as drift.

  • Access by role: the matrix, with any inversion called out.

  • Coverage: pages explored and journeys automated, per role.

Below the tables comes the part the tables can't show: what it did not do, and why. In our run: no defects filed, no password or settings changes, /debug and /api never used, and the practice modules outside our scope left alone. The same report is saved as report.md in the run folder.

The tests it wrote, in your repository

The tests are the result that matters most, and they're ordinary files in your project. Open Test Explorer and the new spec files sit with the rest of your suite. In our run, those were admin access, catalogue, checkout journey and coupons. Open one and the source is plain Playwright in the project's own style: its page objects, a shared helper to reach the cart, and the assertion. There's nothing extra to install and nothing proprietary in the code.

Pick a browser and an environment and run it from Test Explorer like any other test. They're ready to commit and to run in your CI pipeline.

In Test Cases, filter by the requirement and the new cases are there, all marked automated. Each step uses the app's real names, such as the E-commerce Flow tile, the Add button and the cart badge in the header, because they come from what it actually saw. The Mapping tab shows both links: the requirement it came from and the script that automates it.

Everything it recorded, reused

The exploration doesn't stop being useful when the run ends. What it saw becomes app knowledge.

Ask mode

Switch the AI Assistant to Ask and question your app. Ask reads only what's recorded and never opens the app. We asked which roles can open the admin dashboard and how to get there. The answer came back with the disagreement: the requirement says admins and managers, but exploration saw only managers get in. Every claim carries a numbered source, coloured by kind, and opening one shows the run's own report, read-only, with credentials masked.

AI Test Generation

With Use app knowledge switched on, AI Test Generation takes its navigation from the explored pages. Every generated case says how grounded it is, and each step says whether it was observed and on which page. The assertions still come from your requirement; only the navigation comes from what was explored. There's more on that in AI test case generation.

Tips for a good run

  • Give it a scope, so a run fits. A few areas per run is better than the whole app at once.

  • Name your roles from least to most privileged, so inversions show up.

  • Attach your requirements, so drift is caught. Without a spec it can only tell you what the app does.

  • Start with a guest run on an app you don't know. What's reachable without signing in is often the most valuable finding of the day.

  • Use staging, not production. One environment per run.

  • Let it finish. The automation phase is where a run pays for itself.

  • Read the quarantined tests. A test that won't go green twice usually points at something real.

  • Re-run after a release, so what Ask and AI Test Generation know stays fresh. The newest run wins per page.

Common questions

Does it need me to write any code? No. It helps to be able to read a test afterwards, but it writes, runs and fixes the tests itself.

Can it explore a mobile app? Not yet. Explore & Automate is for web applications. For mobile apps, use the AI Assistant's Mobile mode (beta) or AI Record.

Will it file bugs on its own? No. It records findings; you decide which become defects.

What if it finds something that disagrees with my document? It's labelled possible spec drift: a disagreement, not a verdict. It won't change your document or file a bug on its own.

Can I close my laptop and let it finish? No. Leave the tab open. It needs you for permission prompts and sign-ins.

Which AI does it use? The model you pick in the AI Assistant: Testver cloud AI, or a local model through Ollama or LM Studio. A long, multi-step run like this one works best with Testver cloud AI.

Watch it

The full walkthrough of this run, with every step shown live, is on YouTube: Explore & Automate: AI explores your web app, finds bugs and writes verified tests.

Try it

Open the AI Assistant in Web mode, type /, pick Explore & Automate, and point it at your staging site. The full guide is at docs.testver.com/explore-automate.

New to Testver? Run testver serve in your test project and follow the installation guide. To see the AI Assistant's other modes, read AI Assistant: an AI teammate that writes, runs and fixes your tests.

#AI Testing#Web Automation#AI Test Engineer#Selenium#WebdriverIO#Explore & Automate#Playwright#Test Automation#Cypress

Keep reading

Put these ideas to work

Turn your framework into an AI-powered testing platform

Self-hosted, on the stack you already use. Free to start.

Explore & Automate: AI Explores your Web App, Finds Bugs and Writes Verified Tests · Testver