Most AI chat tools can tell you how to write a test. Then you copy the code, paste it into your project, fix the imports, run it, paste the error back, and go round again.
The Testver AI Assistant skips the copying. It works inside your test project the way a teammate would: it reads your files, writes and edits tests in your framework, runs them, opens a real browser or a phone and clicks through your app, sends API requests, and builds reports from your results. It asks before it changes anything, and you can watch every step.
This post covers everything the AI Assistant does: its five modes, how you give it context, how you stay in control, the live browser and device, API testing, reporting, asking questions about your application, and what happens when the connection drops.
A chat that does the work
Open AI Assistant (the sparkles icon in the sidebar), type a request in plain English and press Enter. Every request follows the same loop:
The assistant replies as it thinks, in real time.
When it needs to act, it makes tool calls: read a file, edit a file, run a test, click in the browser, send a request.
Anything that changes your project or sends live traffic waits for your permission first.
It keeps reading, editing and running until the job is done, then sums up what it did.
The whole conversation, including every tool call and file change, is saved as a session you can reopen.
What makes the difference is that it is grounded in your project. It sees your folder structure, your existing tests, page objects and config, and the results of your latest runs. When it suggests a fix, it can apply it to the file on disk and re-run the test to prove it worked.
Five modes, one screen
The mode decides which tools the assistant gets and which instructions it follows. Pick one from the chip in the message box, or click a row on the welcome screen.
Mode | What it's for |
|---|---|
Web | Writing and fixing end-to-end tests, page objects, locators and frameworks. It can open and drive a live browser. |
Mobile (beta) | iOS and Android testing with Appium, on a phone or emulator on your computer, or on a cloud device. |
API | Requests, schemas, collections, chains, contract testing and service-to-service flows. |
Reporting | Summaries, charts, dashboards, documents and emails from your runs and results. Read-only. |
Ask | Questions about the application you are testing, answered from what Testver already knows, with citations. Read-only. |
A project is web or mobile, so you see four modes: Web or Mobile, plus API, Reporting and Ask. The mode is saved with each conversation. If your message looks like it belongs in another mode ("this looks like an API request"), a banner suggests switching, but it never switches on its own.
Give it the right context
The better the starting point, the better the result. The message box has three shortcuts.
@ attaches a test
Type @ and pick a test from the list of every test Testver has found. The assistant then works on that test and nothing else, with its source, its recent run history and its exact run command in front of it. It also picks the right tools from the test's framework: attach an Appium test while in Web mode and it gets the mobile tools, because that is what the test needs.
@login.spec.ts why is this flaky?is a better opening than describing the test in prose. The assistant starts from the real file and its real history, not from your summary.
/ adds a skill
Type / to pick a skill: written instructions the assistant follows for that message, such as your framework conventions or a whole procedure like Explore & Automate. Built-in and custom skills are both listed, and your message records which ones it used.
# adds a source
Requirements, notes, JIRA stories and other documents you already keep in Sources can go with a message without uploading them again. Type #, or use the paperclip → Add from Sources, and pick up to five. Short documents are read whole. For a long one, the assistant gets its beginning and its outline and searches it for the sections it needs, so nothing is cut off silently. Test cases it writes from a source are linked to that source, so they count in its coverage.
Upload files
Use the paperclip → Upload from computer, or drag files onto the chat. PDFs, Word documents, spreadsheets and CSVs, text and config files, API descriptions (HAR, proto, GraphQL, WSDL) and code and test files are all read. The assistant can't read images yet, so paste the text instead. Drop a CSV and ask "convert my login test into a data-driven test using this file".
It knows your test library
In Web, Mobile and API mode the assistant knows every test Test Explorer lists, without searching your spec files for them. Ask "which tests cover checkout?", "what is failing?" or "what does the admin login test do?" and it answers from each test's file, tags, steps, the page-object methods it calls, its linked test case and its run history: flaky, failing since, last passed, getting slower.
Run by name. Say "run the admin login test" and Testver builds the same command Test Explorer's Run button uses, including your own run-command template.
Pick a browser or environment. "Run the login test on Firefox against Staging" uses the names from Settings → Run Configuration. If a name doesn't exist, it tells you which ones do.
See what a change affects. "Which tests would a change to LoginPage affect?" is answered from the Code Map: every test that uses that method, page object or file.
Never the whole suite. The assistant runs only the tests it wrote or changed, or the ones you name. Asked to run everything, it points you to Test Explorer → Suites, which is built for long runs.
Watch it work
The assistant often makes many calls in a row. Instead of one line each, calls are gathered into one row that says what is happening while it works (Editing pages/LoginPage.ts) and sums it up when it's done (Read 7 files, edited LoginPage.ts), with lines added and removed, failures and time taken. Click the row to see every call, and click a call to see its arguments, its result or its diff.
When a request changes files, it ends with a Changed N files card: every file marked New, Deleted or modified, with its own +/− count. Files changed by a command the assistant ran, such as a formatter or a code generator, are included too. Review opens all the diffs in one dialog, and Open in Git takes you to the Git screen to commit them.
API responses, chain results, dashboards and reports are never folded into a row. They stay in the conversation as cards, because they are the answer you asked for.
You stay in control
Permission before every change
Before the assistant edits a file, runs a command or sends live browser, API or mobile traffic, a Permission required card appears above the message box. It shows the tool and, for files and commands, the exact path or arguments.
Button | What it does |
|---|---|
Allow | Approves this one call. The next one asks again. |
Always allow | Approves this tool for the rest of the conversation. |
Deny | The assistant is told no, and carries on without that action. |
Manual or Auto
Next to the mode picker is a Manual | Auto setting. Manual, the default, asks before every change. Auto approves actions so a long task runs through without stopping at each step. It belongs to the chat: every new chat starts in Manual, and you can switch at any time, even mid-task.
Even in Auto, four actions always wait for you, because they are hard to undo or reach other people: deleting a file, sending an email, deleting a saved login, and the manual sign-in pause, where the assistant hands the browser to you to log in.
Stop at any moment
While the assistant works, Send becomes Stop. Press it the moment it heads the wrong way, refine your request, and send again.
Web mode: a real browser you can watch
When the assistant opens a browser, a live preview splits in on the right. It is a live mirror of the real browser, not a screenshot:
Tabs, a URL bar, back, forward and reload.
An AI | User toggle. Switch to User to take over: dismiss a cookie banner, enter a one-time code, then hand control back and ask it to continue.
Pause and Resume to halt the assistant before its next browser action.
Watch mode, which flags each action on the page as it happens.
Element highlight, viewport presets (Mobile, Tablet, Desktop, Full HD), a DevTools panel with network and console, and Picture-in-Picture.
A typical Web-mode request:
Write a Playwright login test for /login with valid, invalid
and empty-field cases. Use our LoginPage page object, run it,
and fix it until it passes.The assistant reads your project to match its style, writes the test, runs it, reads any failure and fixes it, all in the same conversation. A small badge next to the mode picker tells you whether browser actions run on the Server or on your own machine through the Local Browser connector.
It can also set up your browsers and environments. Ask "set up browsers and environments for this project". It reads how your code picks the browser and the application URL, checks each entry against your code, and saves only the ones your code would accept to Run Configuration. Nothing is ever removed.
Mobile mode: drive a real device (beta)
In Mobile mode a session bar appears above the message box. Pick a Provider (a phone or emulator on your computer, or a device cloud such as BrowserStack, LambdaTest or Sauce Labs), a Device and an App, or upload a build, then click Start session. The device mirror appears on the right, and the assistant can explore your app, write Appium tests and run them.
When a test run takes over the device, your taps pause so they don't clash with it, and come back when the run ends.
With no session live and no device picked, the assistant runs on the device you last used, then your local devices, then your default cloud provider. It never starts a paid cloud run on a provider you didn't choose.
Cloud sessions use real device minutes, so End session when you're done.
To record a journey of your own taps instead, use the AI Record agent, which uses the same devices and app library. There is more on Mobile mode in mobile app automation testing with AI.
API mode: from one request to whole services
Describe what you want to test, such as "build an auth + CRUD test chain for /users", and the assistant sends the requests, validates the responses and saves them as collections and chains. Each response shows as a full card: method, status, timing, headers and a formatted body.
Chains that judge every step. A step passes only when its status, every validation and every extracted value are right. A
200with the wrong body is a failure. Steps can poll until a value appears ("wait until the order is CONFIRMED").Contract testing. Check every endpoint against your OpenAPI spec, list the breaking changes between two spec versions, or write and verify Pact contracts.
Microservices. Service-to-service auth (OAuth2 client credentials, mTLS), a correlation id followed across services, local mocks that record what your service sent, WebSockets and server-sent events, Kafka and RabbitMQ events, gRPC, and basic load tests with latency percentiles.
Files. Uploads in API mode are saved into your project, so the assistant can import a spec, a Postman collection or a HAR, run a CSV as test data, or upload a file to an endpoint.
The full walkthrough is in API automation testing with AI.
Reporting mode: answers with real numbers
Reporting turns questions about your runs into answers, charts and documents: "summarise yesterday's failures", "show the top 5 flaky tests this week", "how many automation scripts have never run?". It is read-only: it can't edit files, run tests or change test cases.
Every number is copied from a result: run history, the API call log, suites, test management and the list of every script in the project. Each answer states the time window it covers and what it left out, such as Auto-Heal verification re-runs, which would skew a pass rate.
Dashboards appear in the chat as interactive cards with KPIs, charts and tables. Save one to My Reports, Share a public link with an optional expiry, or Download it as HTML, PDF, Word or Excel.
Documents. Ask for an executive, QA daily, release or trend report and get a finished PDF, Word or Excel file.
Email and schedules. "Email this report every Monday at 9" works from the chat. Emails are always previewed, and nothing is sent until you approve.
Bring your own results. Drop a JUnit XML or a CSV of results from CI into the chat and report on it like your own run history.
Ask mode: what does the app actually do?
Ask answers questions about the application you are testing, such as "how do notification preferences work?", "which roles can reach user administration?" or "is checkout working?". It is the only mode with no browser, no files and no way to run anything, and that restriction is enforced on the server.
Every answer is built from five kinds of record, and every claim is marked with where it came from:
Source | What it adds |
|---|---|
Exploration | What an exploration run actually watched the app do: pages, controls, which role could reach what. |
Defects | What testers logged as going wrong, open or fixed. |
Test cases | What your test cases say the app should do, with each case's last run. |
Automated tests | The tests in your repository and their latest results. |
Documents | Requirements, uploaded documents and linked JIRA tickets. |
Hover a numbered marker to see its source, or open it in a new tab. A test that has never run proves nothing, and Ask says so. If nothing is recorded on a topic, Ask tells you instead of guessing, and the question lands in a Searches that found nothing list: a to-do list for whoever keeps the knowledge. Passwords and one-time codes are masked in every answer.
Attach Sources in Ask mode and the answer comes only from them, still with citations. It's the fastest way to get up to speed on an application you've never tested before.
Example: fix a failing test
On the Results screen, open a failure group and click Fix in AI Assistant. The chat opens with a starter message, often with the test already attached. Nothing is sent until you press Enter.
The assistant reads the test, its page objects and the failure, and explains what went wrong.
It proposes the edit. You see the exact file and approve it.
It re-runs that one test and reads the result. If it still fails, it keeps going.
The Changed 1 file card shows the diff. Click Open in Git to commit it.
It doesn't lose your work
A task runs on the Testver server, not in your browser tab, so short network trouble doesn't stop it:
Your Wi-Fi drops or the laptop sleeps. The task carries on. A Reconnecting to Testver… strip appears, and once you're back, the reply, the tool steps and any permission card sent meanwhile catch up.
The AI service can't be reached. Testver retries for up to two minutes and continues from the part of the reply that had already arrived, instead of starting over.
The AI stops before finishing. If it announces a next step and then stops, Testver reminds it once to carry on. If it still stops, a Continue button picks the task up, with its file changes kept.
Teammates changed a file. If a file in the conversation is newer on the remote, a notice offers Pull now first, so the assistant doesn't work on old code.
Every conversation is saved. Open History (the clock icon) to search, filter by mode, pin, rename or reopen a session, with its tool calls and diffs restored.
Your choice of AI
The model picker next to Send lists Testver cloud AI, a managed model Testver picks and tunes for you, and any model you added to a local server such as Ollama or LM Studio in Settings → LLM. With Testver cloud AI, a small ring beside the picker shows how much of your credits you've used. Hover it for the numbers.
Tips for better results
Pick the right mode first. It decides everything the assistant can do.
Be specific. "Fix the flaky selector in checkout.spec by switching to a data-testid locator" beats "fix my tests".
Attach, don't describe. Use
@for the test,#for the requirement, and drop the log or the CSV.Read the diffs before you move on.
Use Allow, not Always allow, for file edits until you trust the pattern. Save Auto for tasks you're watching.
Stop early if it drifts, then clarify.
Take over the browser for logins, one-time codes and cookie banners.
End mobile sessions when you're done.
Common questions
Does it change my code without asking? Not in Manual mode, which every chat starts in. In Auto mode it approves changes on its own, but deleting files, sending emails and the sign-in pause still ask every time.
Which frameworks does it work with? The ones your project uses. It reads your project and writes in its framework, language and style. To start from nothing, it can create a framework for you.
Can it run my whole regression suite? No, on purpose. It runs the tests it wrote or changed, or the ones you name. Long runs belong in Test Explorer → Suites.
Can I use a local model? Yes. Add a model to Ollama or LM Studio in Settings → LLM and pick it in the chat. Larger models reason better about complex code, so Testver cloud AI is the better pick for long, multi-step tasks.
Can it read screenshots? Not yet. Paste the error text, or describe what the screen shows.
Where do my conversations go? Into History, with every tool call and diff. Delete is permanent and asks you first.
Watch it
Try it
Open AI Assistant from the sidebar, pick a mode, and start with something real: "@checkout.spec.ts why did this fail last night?" The full guide is at docs.testver.com/tools/ai-assistant.
New to Testver? Run testver serve in your test project and follow the installation guide, or read web automation testing with AI for a hands-on walkthrough of Web mode.