Reference docs

What a reference doc is

A reference doc is a file you attach to an application. The file’s text is extracted and stored on the application, and OttoTester feeds it into the planner on every run.

Use them for anything the planner would benefit from knowing that isn’t easy to convey in the Agent Prompt:

• A product spec describing what the app is supposed to do.
• A list of test scenarios the team has agreed should be covered.
• A .feature file in Gherkin notation, if you already write one.
• A help article explaining how a feature works.

The Agent Prompt is still the right place for a one-line steer — “focus on checkout” — but anything richer belongs in a reference doc.

Why reference docs sit on the application

Three levels can hold context, and OttoTester puts reference docs at the application level on purpose:

Where	What lives there
Application	Reference docs, application description
Variant	Base URL, browser settings, auth fixtures
Template	Saved run configuration, including the Agent Prompt

A product spec or a feature file describes the app’s contract uniformly — staging and prod should be tested against the same spec. Attaching docs to the application means you upload once and every variant benefits. Environment-specific things — credentials, browser timing — already have natural homes on the variant.

What formats are accepted

• .txt — plain text.
• .md — Markdown.
• .docx — Word documents. Text is extracted server-side; formatting is dropped.
• .pdf — machine-readable PDFs. Scanned or image-only PDFs aren’t read.
• .feature — Gherkin / Cucumber feature files.

You can add any of these as an uploaded file, or by pasting a link to one (see below). A link can also point to a normal web page — OttoTester reads the page’s main content.

Upload and manage

On the application page, open the Reference docs tab and drop files into the upload area (or click to pick them). OttoTester extracts the text immediately and the file appears in the list with its name, size, and the date it was uploaded.

To replace a doc, delete the old one and upload the new version. To remove one, click the row’s delete button — the doc is gone from the application’s context the next time the planner runs. Proposed tests it produced are removed with it, except ones you edited or built a test from: those are kept and marked document deleted.

The extracted text is what OttoTester stores; the original file isn’t retained. If you want to revise the wording without re-uploading, the better path is to edit the source doc on your side and re-upload — that keeps your source of truth and OttoTester’s view in sync.

If a read fails

After upload, OttoTester reads the document with an AI model to find its requirements and suggest tests. Sometimes the model doesn’t answer or its provider is busy. When that happens, OttoTester tries once more on its own, 2 minutes later. The row shows an amber Retrying pill with the reason, for example “No response from model, retrying in 2 mins”. There is nothing to do while it waits.

If the second try also fails, the row shows a red Failed pill. Click it to see why, then click Retry to read the document again. A failure that a retry can’t fix, such as an AI key the provider rejected, shows Failed right away.

Long documents are read in rounds

A document with more than a handful of requirements is read a few requirements at a time. Each round reads the whole document but writes tests only for its own requirements, so nothing loses context. The row shows how far it has got, for example “Reading · 40 of 120 requirements”, and tests from finished rounds appear on the Proposed tab right away. A document with no headings or requirement IDs first shows “Finding requirements…” while OttoTester works out what they are.

If a round fails both of its tries, the row says how far the read got and keeps those tests. Click Read the rest to carry on from where it stopped. It does not start over or redo finished rounds.

Add a reference doc by link

Sometimes the spec you want to feed OttoTester already lives at a URL — a Confluence or Notion page, a published requirements doc, a PDF on SharePoint. Instead of downloading and re-uploading it, click Add a link next to Upload and paste the link.

• The link can point to a web page or to any supported file (PDF, Word, Markdown, text, .feature).
• For a web page, OttoTester saves its main content — the article body, not the site’s menus, sidebars, cookie banners, or footer. This is best-effort; some unusual page layouts won’t read cleanly, and you can always fall back to uploading.
• OttoTester reads the link once and saves a copy of its text. The doc row shows it came from a link, the link itself, and “Saved from this link on <date>.”

It’s a snapshot — Refresh to update it

Adding a link takes a one-time snapshot. OttoTester does not silently re-read the link later, so a link that later changes or stops working can never break a test run. When you want the latest, use the row’s Refresh from link action: it re-reads the page and updates the saved copy and date, and nothing else. It doesn’t regenerate your proposed tests and makes no AI calls.

If a Refresh finds the content has changed, the row shows a small note — “Content changed since these proposed tests were made.” — with a one-click Re-propose that keeps any proposed tests you’ve edited or built a test from. If a Refresh can’t read the link (it’s offline, needs a sign-in, or 404s), your previous saved copy is left exactly as it was.

Adding a link that’s already saved for this app doesn’t make a second copy — OttoTester tells you it’s already there and points you to Refresh instead.

Reference docs suggest tests

When a reference doc is uploaded, OttoTester reads it and suggests tests from it. The proposed tests appear under Tests → Proposed, where you can review, edit, archive, or add your own. See Proposed tests.

They aren’t run — they’re ideas the planner pulls from while planning, so the tests it writes line up with what your docs describe.

Say what each document is

Each document row has a Counts as line with three choices:

• Requirements: a spec, user stories, acceptance criteria or a compliance standard. Its parts are things the app must do.
• Defects: an exported bug list, QA findings or an incident write-up. Its parts are bugs that must not come back.
• Reference: a guide, help pages, a manual or an existing test plan. It describes how the app works and only produces proposed tests.

When OttoTester reads a document, it suggests one of the three and says why, for example “Suggested by OttoTester: this reads like a how-to guide.” The suggestion is selected until you pick. Click any choice to set it, including the one already selected, which confirms the suggestion. The note then reads “You chose this.”

A document that has not been read yet, whose read failed, or that was read before this setting existed shows Reference with no note. Nothing counts as requirements or defects until a read or a person says so.

Changing it takes one click. It does not read the document again, makes no AI calls, and leaves the document’s proposed tests as they are. A document that mixes kinds gets one label: pick what most of it is.

Review what OttoTester found

When a read finishes, a Review what OttoTester found window opens on the Tests page. It does this for every document, whatever it counts as. If you are not on the page, it opens the next time you are. If several documents are waiting, they open one after another.

The window lists what OttoTester found in the document, each with the document’s own words beside it. Pick what the document is at the top, untick anything wrong, and click Confirm. Unticked items lose their proposed tests. For a Requirements or Defects document, the items you keep start counting toward coverage. A Reference document’s items are called topics and never count.

Later closes the window until your next visit. The found · Review label on the document’s row opens it any time.

If the list was found before OttoTester kept the document’s words, or the document changed since, it can’t be confirmed. Click Read this document again to read just that document. Your other documents and their proposed tests stay as they are.

Related

• Concept → Applications, Proposed tests, Variants
• Workflow → Steer what the planner tests