Skip to main content

Start the interview

In the guided interview, you describe your application and the Wizard drafts the tests.

1. Start the Wizard​

In your editor, open a terminal in the repository folder and start the Wizard:

Terminal
1make run-web

Each line of output starts with the name of its server, api or web. Keep the terminal open. Open the address that the web lines show, for example http://localhost:5173. You see the opening screen:

The opening screen of the Wizard in the browser

2. Build Spec and/or Connect Target​

Above the text box, choose:

  • Build Spec only if you only wish to generate the specification.yaml
  • Build Spec and connect target if you wish to generate both the specification.yaml and the Target connection.

3. Describe your Target​

Type a short description. Say what the application does and who uses it:

A research assistant for policy staff. It answers questions about internal policy documents and cites the source of each answer.

You may also upload documents. They help the Wizard build more representative specifications.

4. Choose the Perspective​

The Perspective says who the evaluation is for. End user is the default.

PerspectiveGuiding question
End userIs the user getting what they came for?
EngineerIs the agent performing as intended, correctly and efficiently?
ComplianceAre our responses audited, grounded, and within compliance policy scope?
SecurityIs our target safe against malicious attacks?

5. Select Capabilities​

Select Capabilities to tell the Wizard what your application can do, such as Tool Use or File & Data Management. Under Confidence, say how sure you are that the list is complete. This step is optional. If you skip it, the capabilities stay unspecified.

6. Run the Interview​

The Profile opens once you start the Interview. There are a few key steps you will go through to complete your specification.

StepYou do this
Define product requirements
1. Understanding your targetAnswer the Wizard's questions
2. Features / Expected BehavioursChoose what to measure
3. Acceptance criteriaCheck each Criterion and its grader
Design Test Cases
4. PersonasCheck the simulated users
5. ScenariosCheck the situations to test
6. Test CasesChoose the tests to run, then download

The browser saves each interview. To open one again, select it in the list on the left.

The Wizard steps, from the start to the specification download

Next step​

Define product requirements.