Tests

Every test in OttoTester has two sides:

The test case is the spec — what the test should do, in plain English. A name, a description, an intent, a priority, and a list of steps. It’s readable; it isn’t code.
The test script is the executable — the Playwright code that actually runs.

The planner writes the test case. The generator writes the test script from it. The executor runs the script. The two are linked but kept as separate records, so you can read and reason about what a test does without opening Playwright code.

Anatomy of a test case

The test-case side carries:

Name and description — what the scenario is.
Intent — the kind of test: happy-path, negative, edge-case, validation, or error-handling.
Priority — P0 through P3.
Steps — the ordered actions and checks.
Status — planned (case exists, no script yet), scripted (script written, ready to run), or blocked (can’t proceed — usually a real app issue the healer couldn’t get past).

Pass/fail isn’t part of the test case. That’s a result, recorded on the script side after a run.

Changing a test

You can change a test in three ways:

Edit the test script directly. Your edit is saved as the test’s current version — the next run uses it, and if the test later fails, the healer fixes your version rather than throwing your edit away. The fastest way to fix one test.
Regenerate the test. Re-run planning and generation to rebuild the test from scratch — useful when the app changed enough that a patch won’t do.
Add a hint. A hint doesn’t change this script directly; it teaches the generator, so future generations — of this test and others — handle the page correctly.

Edit the script to fix this test now. Add a hint to stop the same problem coming back across regenerations and other tests.

Edit and pin

To change a test for the long haul, open it from the Tests page and edit the script directly. Your edit becomes the test’s current version — the next run uses it, and if the test later fails the healer repairs your version rather than throwing your edit away.

That’s what “pin” means in practice: there’s no separate pin toggle, because every saved edit is treated as the authoritative version until the next edit or a rollback. To restore an earlier version, see version history below.

Version history

OttoTester keeps a history of each test. When a test changes — an agent updating it, or you editing it by hand — the new version is saved alongside the old ones.

For any test you can:

See its history — the changes to the test, newest first, with who made each.
Look at an earlier version — see what the test’s spec and script were at an earlier point.
Roll back — restore a test’s script to an earlier version. The rollback is saved as a new step, so nothing in the history is lost.

Nothing you or an agent does to a test is gone for good — there’s always a way back.

Fixtures are a kind of test

Most tests are ordinary tests. Some are fixtures — a test whose job is to sign in, so the tests after it start already authenticated. A fixture has the same two facets (a case and a script), but is scoped to a role. See Fixtures and test data.

Tests and suites

Tests are grouped into test suites — named sets you run together. A test can belong to more than one suite.

Concept → Test Authoring Templates, Test suites, Fixtures and test data, Healing and hints
Workflow → Fix a failing test