Auto-Heal: Self-Healing Automation Tests you Can Trust, Proven by a re-run

When a renamed element breaks your tests, Auto-Heal finds the locator in your code, fixes it from the page the run actually saw, and re-runs the tests to prove it. Every setting and screen, from Results to the Auto-Heal tab

T
The Testver team · 16 min read

A developer renames data-testid="username" to username-input. Nothing about the login page changes for a user, but the next morning twenty tests are red. Someone has to work out that it's one locator, find it in the page object, fix it, re-run the tests, and only then get back to real work.

That is the job Auto-Heal takes off your hands. When a test fails because a locator no longer matches the page, Auto-Heal finds that locator in your code, works out what it should be from the page the run actually saw, re-runs the affected tests to prove the fix, and only then keeps it or offers it for your review. If the tests don't pass, your code is put back exactly as it was.

This post covers everything Auto-Heal does: how it works, every setting, and what you see from a failed run in Results all the way to the Auto-Heal tab.

Self-healing you can trust

Many "self-healing" tools work at runtime: when an element can't be found, they quietly click something that looks similar and carry on. The test goes green, but nobody knows what it actually clicked, and your code still has the broken locator. Auto-Heal is deliberately the opposite. Four principles shape everything it does:

  1. Evidence over guessing. A fix is proposed only from what the failing page actually contained, and every proposal is checked against that evidence before anything is edited. No evidence, no edit. The test gets a plain-language reason instead.

  2. Verify or revert. Every fix is checked by re-running all the tests that failed on that locator, with the same browser, environment and cloud target as the original run. All pass, and the fix stands. Any fail, and the code is put back.

  3. Your framework stays yours. A fix changes only the value inside the locator you already wrote. By.id("username") stays By.id(…), and getByRole('button', { name: …, exact: true }) keeps its options. There are no wrappers and no runtime hooks.

  4. Your work is never overwritten. Putting a change back swaps only that locator, so edits you or the AI Assistant made to the same file survive. If the server restarts in the middle of a check, the temporary edit is undone automatically at startup.

Turning it on

Auto-Heal is off until an admin turns it on in Settings → General → Auto-Heal. Each change saves as soon as you make it.

Setting

Values

What it does

Mode

Off / Suggest / Auto

Suggest finds and verifies a fix, puts your code back, and offers the change for one-click Apply. Auto keeps verified fixes in your code straight away, still visible and still undoable. Start with Suggest.

Give up after N attempts per test

1–5

How many tries a test gets. Only attempts that cost something count (an AI call, a capture re-run, a probe, a verification), never free checks.

Then wait before retrying

6 hours to 1 week

How long a test that hit the limit rests before Auto-Heal tries it again. A kept fix resets the count.

Offer Apply fix / Undo fix for

1–30 days (default 7)

The review window. After it, a fix reads Expired and its buttons go.

When a heal succeeds on a CI-gated run

Report only / Update the build

Whether a fully healed run can turn a failed CI build green (see Auto-Heal and CI below).

Open a browser as a last resort

On / Off

The live probe: opens the page fresh when no other evidence exists. Off by default.

Remote hub relay

URL

Only for your own Selenium Grid, or tests with a hard-coded hub address: point them at http://localhost:4499 so Auto-Heal can read their sessions. Cloud providers set up in Testver need nothing here.

You can also override the workspace setting where it matters:

  • Per suite: in a suite's Settings tab, Auto-Heal → Locator repair can be Inherit, Off, Suggest or Auto. A suite can turn healing on even when the workspace default is off.

  • Per CI pipeline: each pipeline mapping can be Inherit, Off, Heal only or Heal + re-gate.

Who can do what: changing settings needs the admin role. Apply fix and Undo fix need the tester role, because they edit source files. Everyone can see what Auto-Heal did. The server enforces all of this, not just the screen.

How it works, step by step

  1. After the run, never during it. Auto-Heal starts only once a run has finished and been saved. Nothing runs in the middle of a test. Runs that fail while another heal is in progress wait their turn; none is dropped.

  2. Only locator failures. Auto-Heal acts when an element could not be found: not found, timed out waiting for it, or matched more than one element. When an element was found but couldn't be used (an overlay blocked the click, or it was disabled), changing a locator wouldn't help, so the test is explained and left alone. Everything else goes to Failure Analysis.

  3. One fix per broken locator. Failed tests are grouped by the exact locator that broke. All the tests in a group are fixed once and verified together. Tests with different broken locators are always healed separately, even when their errors look alike.

  4. Find the locator in your code first. Before spending anything, Auto-Heal finds where the locator lives: the test's own file, then page objects, helpers and locator files (JSON, YAML, .properties, .env, CSV, Gherkin tables). It knows each framework's shapes, from page.locator('#x') and By.id("x") to @FindBy(id = "x"), Appium's AppiumBy.accessibilityId("x") and Robot's id:x, and it recognises your own helpers too. When the same locator appears in several places, the failing stack trace decides which copy actually ran. A locator built at runtime, such as #row-${i}, is healed by changing only its fixed part, never the expression.

  5. Gather evidence and propose a fix, cheapest evidence first (see the next section).

  6. Apply and verify. The fix is applied as an exact edit of that one locator, and every affected test re-runs headlessly with the original run's browser, environment and cloud target. If the test now gets past that step and trips over a second broken locator, that one is healed too and the test is checked again, up to three locators, kept or put back together. A failure that has nothing to do with locators is retried once before anything is decided.

  7. Keep or put back. If every test passes, the fix is kept (Auto mode) or put back and offered as Fix ready (Suggest mode). If the test got further but then failed on something else, the change is put back and offered as Fix ready (partially verified), with what happened next; it is never kept unattended. Anything else is put back and recorded as Reverted.

  8. Record and notify. The run and each test are stamped with the outcome in Results, and any connector subscribed to Auto-Heal Outcome is notified.

Evidence, cheapest first

Auto-Heal tries free evidence before anything that costs time or AI, so the common cases, like a renamed id, need no AI at all.

Evidence

Where it comes from

Cost

Recorded fallback

Tests recorded with Testver keep ranked alternative locators for each step, next to the locator in your code and in a .healing.json file.

Free

Failure snapshot

Playwright's accessibility snapshot at the moment the test failed.

Free

Failure page

The page exactly as it was when the test failed, from the Playwright trace.

Free

Page DOM

Every attribute on every page the run loaded, from the trace.

Free

Live session

The test's own browser or app session, read while it runs: logged in and in the exact failure state. Works for Selenium, WebdriverIO, Robot Framework and Appium, locally, on a grid or in the cloud.

Free

Capture re-run

One plain re-run of the failing test to collect evidence.

One test run

AI choice

When several elements on the page fit, one AI call picks one of them by number. The value always comes from the page, never from the model.

One AI call

AI proposal

One bounded AI call, held to your framework's locator style and checked against the page.

One AI call

Live probe

Opt-in: open the page fresh in a headless browser, reusing a login saved by Explore & Automate when one exists.

One browser

On every piece of free evidence, Auto-Heal first tries exact matching: a renamed test id, id, name, class or attribute; a CSS-module class whose hash changed on rebuild (Button_primary__x7Ks9 → Button_primary__Qz81a); link and visible text; Android resource-ids and accessibility ids. The name of the variable holding the locator counts too: this.emailInput points at email-input even when the old id was meaningless. Only when none of that decides does AI come in. The Auto-Heal tab shows the share of fixes solved without AI.

It reads your code, not just the page

Sometimes the old locator gives nothing to go on, like readonly elementor = this.testId('select') for what is really the remember-me checkbox. Auto-Heal then reads the code around the locator:

  • How the element is used. .check() or toBeChecked() means a checkbox, .fill() or sendKeys() means a text field, selectOption() means a dropdown. This is a hard rule: an element of another kind is never proposed, by any step, AI included.

  • Elements already taken. An element that another locator in the same file already targets isn't the one this locator lost.

  • Your comments. A comment like // remember-me checkbox above the locator, or a locator file's key, helps rank the candidates.

  • Step titles and method names, such as test.step('Accept the terms', …) or a method called acceptTerms().

A comment can support a fix but never force one, because comments go stale. The fix's explanation says which clues decided it, and the re-run still decides whether anything is kept.

What it heals, and what it never touches

Heals

Never touches

Playwright getBy… calls in JavaScript, TypeScript, Python, Java and .NET, and string selectors

Assertion values, URLs, waits and timeouts

CSS selectors: ids, classes, data-testid, data-test, data-cy, data-qa and other attributes

The expressions inside a template string (only its fixed text changes)

Selenium By.id, By.name, By.linkText, simple By.xpath, and @FindBy page factories

Complex XPath, except when the AI proposes another XPath

WebdriverIO, Robot Framework, Selenide, Nightwatch, TestCafe, Puppeteer and Katalon locators

Clicks blocked by overlays, and disabled or re-rendered elements

Appium resource-ids, accessibility ids, Android UiSelector and iOS predicates

Anything without page evidence to justify it

Cypress cy.get(…) selectors and cy.contains(…) text

A fix you undid before: it is never proposed again

Locators in data files: .properties, .env, CSV, Gherkin tables

Reports and logs that merely mention the locator

Framework support

  • Playwright (JavaScript and TypeScript) has the richest evidence: the failure snapshot, the trace and recorded fallbacks, wherever Playwright runs.

  • Playwright for Python, Java and .NET heals from the trace, recorded fallbacks and the live probe. In Java and .NET, save the trace when a test fails and Auto-Heal finds it. Frameworks Testver creates for you already do.

  • Selenium (Java, Python, C#, Ruby, Kotlin), Selenide, WebdriverIO, Nightwatch and Robot Framework heal from the test's live session, locally, through the relay for your own grid, or in the cloud.

  • Appium (in beta in Testver) heals from the live app screen and recorded fallbacks.

  • Cypress heals from the live page in Electron, Chrome and Edge.

  • TestCafe, Puppeteer and Katalon heal from recorded fallbacks and the live probe.

Cloud runs heal too. Testver ties each cloud session to the run that created it and reads the page through the provider, with the same evidence quality as a local run. See Cloud Grid Execution.

From a failed run to a fix, in Results

You don't have to go looking for Auto-Heal. Its outcome shows up wherever you look at a failed run in Results.

  • The run header sums it up in one chip, such as Auto-Heal · 1 healed, 2 fix ready, 1 reverted.

  • Each failed test carries a chip: Healed, Fix ready, Fix ready (partial), Reverted, Undone, Fix expired, or Not healed. Hover over Not healed to see the reason.

  • Open a test and its Auto-Heal card shows the files the fix touched with the locator before and after, where the evidence came from, a confidence score, and the next step: Apply fix, Undo this fix, or Fix in AI Assistant, plus Open verification run.

  • When Auto-Heal didn't act, a Not healed automatically card gives the reason in plain words (not a locator problem, a locator built at runtime, no page evidence, or the attempt limit reached), with Fix in AI Assistant and View group.

  • Failure Analysis shows the same Auto-Heal verdict on each failure group, so nobody sends the AI Assistant after a fix that's already waiting for review.

  • Verification runs appear in the runs list with a Heal Verify badge, so you can open the exact run that proved a fix.

When you hand a test to the AI Assistant from an Auto-Heal card, the Assistant is told what Auto-Heal already tried, so it doesn't repeat it.

The Auto-Heal tab: your review queue

Results has its own Auto-Heal tab, next to Runs, covering every repair across every run, suite runs and test plan cycles included. It's a work queue first and a log second.

  • Overview: total repairs, healed, fix ready, reverted, undone, and the share solved without AI.

  • Fixes ready for review: every verified fix waiting for you, with its test, file, evidence source, confidence and when it was prepared.

  • Locator hotspots: up to five tests whose locators broke again and again, marked ×2, ×3 and so on. When the app changes faster than a test's locators, that test's locator strategy deserves a human look.

  • Activity: the full history, filtered by state (Healed, Fix ready, Reverted, Undone, Expired, Stale), by evidence source, or by searching for a test, locator or file.

Click a row to expand it. You see what the state means, why the fix was chosen (for example, "No element with test id "username" exists on the page, but "username-input" does: the id was renamed"), the exact before and after for each file, the source, confidence and time, how many tests the change fixed together, and the actions: Apply fix, Undo this fix or Fix in AI Assistant, plus Open test, Open run and Open verification run.

Applying and undoing

  • Apply writes the verified change into your files and marks the fix Healed. When several runs suggested the identical change, applying it once retires all of them.

  • Undo puts the old locator back in every file of the change, all or nothing, and marks the fix Undone. It asks you to confirm first, because Auto-Heal remembers the rejection and never proposes that exact change again. Your version control is the way back if you change your mind.

  • Both re-check the change against your current files first, so neither can damage code that has moved on, and both wait while a heal is being verified.

A kept fix is an ordinary change in your working tree. Review it and commit it like any other.

When fixes retire

A suggestion doesn't wait forever. A fix becomes Expired when it's older than the review window, and Stale when your code has changed since it was prepared, so the recorded edit no longer fits. Either way, its row stays in Activity with the reason, only the buttons go, and a fresh fix is worked out from current evidence the next time the test fails.

Auto-Heal and CI

When a pipeline is gated on Testver through CI/CD Pipelines, you choose what a heal means for the build:

  • Report only (the default): heals happen and are reported, and the build verdict doesn't change.

  • Update the build: the gate waits while healing runs. If every failed test is healed and verified, the build turns green, labelled "healed & verified" in your CI provider. Anything less stays failed, and Suggest mode never turns a build green.

The CI Runs list shows the heal detail for each run, such as "2 healed & verified" or "1 fix ready".

Notifications

Subscribe a Slack, email or other notification connector to Auto-Heal Outcome and your team hears about it when a locator is healed, a fix is ready for review, or a heal is reverted.

Built to be safe

  • No endless loops. A test that keeps failing for another reason uses up its attempts and then rests for the waiting period you set.

  • Only the failing tests re-run, never the whole suite.

  • Crash-safe. Each edit is journaled before it's made, so a server restart mid-check puts the code back at startup.

  • Verification runs can't be faked. They authenticate with a secret that exists only inside the Testver server.

  • The relay is local only. It serves WebDriver clients on the same machine, forwards only to your configured hub, and never sends the hub's credentials anywhere else.

  • The live probe stays on your site. It opens only web pages, never cloud metadata addresses, and never follows a link to another site.

Tips for tests that heal well

  • Name locator variables meaningfully. this.emailInput tells Auto-Heal what to look for; this.el3 tells it nothing.

  • Add a short comment when a locator's name doesn't say what it is. Tests written by Testver, from AI Record, the AI Assistant, Explore & Automate or a new framework, already put a one-line description above every locator.

  • Record with Testver. AI Record saves ranked fallback locators for every step, the cheapest evidence there is.

  • Keep each locator in one place, in a page object.

  • Keep Playwright traces on. The trace is the richest evidence Auto-Heal gets.

  • Watch the hotspots. A test that heals again and again is telling you its locators need a sturdier strategy.

Common questions

Will it change my tests without telling me? No. In Suggest mode nothing changes until you click Apply. In Auto mode, verified fixes are kept, but every one is shown in Results with one-click Undo, and it's an ordinary change in your working tree.

Does it use AI on every failure? No. Exact matching on free evidence handles the common cases. AI runs only when that can't decide, only with evidence, and its answer must match the page before it's even tried.

Can it heal a test that fails behind a login? Yes. For Selenium, WebdriverIO, Robot Framework and Appium, Auto-Heal reads the test's own logged-in session. Playwright's trace already holds the logged-in page.

What does verification cost? A headless re-run of just the affected tests (cloud minutes, for a cloud run). A suggestion waiting for review is reused when the same test fails again, rather than verified again.

Try it

Turn Auto-Heal on in Settings → General → Auto-Heal, set the mode to Suggest, and run a suite. The next time a renamed element breaks a test, open Results → Auto-Heal, read the change, and click Apply fix. The full guide is at docs.testver.com/test-automation/auto-heal.

New to Testver? Run testver serve in your test project and follow the installation guide, or start with Meet Testver.

#AI Testing#AI Test Engineer#Auto Heal#Selenium#API Test Automation#Playwright#Cypress

Keep reading

Put these ideas to work

Turn your framework into an AI-powered testing platform

Self-hosted, on the stack you already use. Free to start.

Auto-Heal: Self-Healing Automation Tests you Can Trust, Proven by a re-run · Testver