Turning Acceptance Criteria Into a UAT Script

Fifteen acceptance criteria from one worked case, sorted by what stops each one becoming a UAT step, then one scenario end to end, filled and blank.

Numbers in this note carry source tags ([POLICY], [THREADS n=10], [MINE]). What the tags mean.

This piece uses the returns case published on this site: damaged-delivery refunds at a mid-size online retailer, built only from public sources, namely a published returns policy, help centre articles, ten public complaint threads I selected myself, consumer law and a walk through the published customer journey. No client, no engagement, no interviews, and no system. The case has five requirements with fifteen acceptance criteria between them, and no UAT script. The script below is mine [MINE], it has never been run, because there is nothing to run it on, and the sorting of the fifteen criteria is mine too. You can repeat it from the published case and disagree line by line.

A UAT script is a sequence of steps that the people who will live with a change follow to decide whether to accept it, where every step names who acts, what they do, the result they should see and the acceptance criterion that result proves. An acceptance criterion says what must be true. The script says how a person who did not write it will find out, in what order, and with what in front of them.

In short

  • Write the expected result column first. Whatever stops you filling it in is the real defect in the criterion, and it shows up in four kinds: nothing, a clock, an open question, or an outcome no user can see.
  • In the returns case, five of fifteen criteria become steps as written. Three need a clock the tester cannot wait for, five have an expected result that depends on an open question, and two describe something no user can observe.
  • Writing the script found two rows the register never wrote: what happens to a single file above the size limit, and what happens when the goods-in lead tries to reopen on day six.
  • Split the scenario where the actor changes, not where the screen changes. The case's own stakeholder map then tells you who has to sign each section.

What does a UAT script add that acceptance criteria do not?

Order, people and data. Wikipedia's article on acceptance testing defines it as testing "conducted to determine whether a system satisfies the acceptance criteria", and says user acceptance testing "should be undertaken by the intended end user, or a subject-matter expert (SME), preferably the owner or client of the solution under test". It also says how to cut it: UAT "should be executed against test scenarios", and they are broken down into sections "where the actor (player/customer/operator) or system (backoffice, front end) changes".

So a scenario crosses requirements, each section of it has an owner of its proof, and every "within five working days" becomes a test order delivered on a stated date.

The Gherkin reference from Cucumber adds the rule that sorts the fifteen criteria below: "An outcome should be on an observable output. That is, something that comes out of the system (report, user interface, message), and not a behaviour deeply buried inside the system."

Fifteen criteria, sorted by the first thing that stops the expected result

For each criterion, write the expected result a tester would compare against. Stop at the first obstacle. Four classes, checked in this order:

  • Answer: waits on an answer. The expected result depends on an open question in the case.
  • Clock: needs a clock. The result is known, but nobody can wait 48 hours inside a session. It needs aged test data, which is a request to the delivery team.
  • Witness: not a user's test. The result exists only inside the system. UAT can witness it, not run it.
  • Runs: runs as written. A business tester with a test order can decide pass or fail.
Criterion In short First obstacle Class
REQ-014 / 1 3 files, 10 MB each, 4th rejected None. Size boundary missing, see below Runs
REQ-014 / 2 Decision within 48 hours of the form timestamp Weekends: clock or working hours, OQ-04 Answer
REQ-014 / 3 Partial only above 40 EUR Do partial refunds ship at all, OQ-05 Answer
REQ-015 / 1 Breach goes to a named individual, never a queue A breach takes 48 hours Clock
REQ-015 / 2 Due: earlier of 5 working days or 24 hours from assignment OQ-02. Its fallback does not say what happens to this formula Answer
REQ-015 / 3 One message on escalation, one on reassignment, none on reopen Needs an escalated case Clock
REQ-016 / 1 Photos deleted 90 days after close The 90 days are [MINE, OPEN, OQ-01] Answer
REQ-016 / 2 Reopen inside the window keeps evidence The window is OQ-01 Answer
REQ-016 / 3 Deletion logged, log outlives the photos Not a user output Witness
REQ-017 / 1 Goods-in reopens within 5 working days of receipt None. Day six missing, see below Runs
REQ-017 / 2 Reasons from a list, at least one dock photo None Runs
REQ-017 / 3 Money never reversed automatically, same 48 hour clock The 48 hour clock Clock
REQ-018 / 1 Pre-check approves when BR-04 and BR-11 are met None Runs
REQ-018 / 2 Pre-check cannot refuse, hands over the reason None Runs
REQ-018 / 3 Each automated approval records the rule version A record, not an output Witness

The count: five run as written, three need a clock, five wait on an answer, two are not a user's test.

The five that wait are not badly written. Each names its open question, and the case gives each question a fallback. The sort shows what those fallbacks cost in test terms: a step with two possible expected results, which the filled script below writes once, with the open question in the cell.

The two witness lines are the two most about trust: the deletion log and the rule version behind each approval. Dropping them from UAT would be wrong, and so would asking a Customer Care manager to read a database. The delivery team shows the record, the business signs that they saw it, and the script calls it a witness, not a pass.

What did writing the script find that a review did not?

Two rows past the boundary. The register template asks every criterion for "the boundary. If a threshold appears anywhere above, say what happens exactly at the threshold, not only above and below it." REQ-014 criterion 1 does that for the count of files, not for their size. A tester holding a 12 MB photograph has no expected result, so the developer will choose one. REQ-017 criterion 1 has the same shape: the goods-in lead can reopen "within five working days of receipt", and nothing says what happens on day six. Both criteria read as complete. You find the gap when you have to type the test data.

Who signs each section. Splitting by actor gives four sections: customer, pre-check, Customer Care, goods-in. The case's stakeholder map gives the decision rights: the Head of Customer Care approves and vetoes, the returns goods-in lead decides in practice, legal and data protection vetoes retention, and the customer has no right inside the company. So a tester plays the customer, and someone with a decision right accepts that section. Wikipedia's "preferably the owner" points at the Head of Customer Care, who also owns three of the four open questions behind the table (OQ-02, OQ-04, OQ-05; OQ-01 sits with legal). The person who signs UAT is the person who has not answered yet.

Clock steps are a fixture request. Three criteria need 48 hours or five working days to pass. The script turns each into a precondition: a test case with a known timestamp in the past, prepared by the delivery team. That request goes in before the session is booked, or the session reaches step 3.1 and stops.

The UAT script, filled in

Scenario UAT-01, damaged delivery from form to reopen. One case, four sections, each starting where the actor changes. The column "Proves" points at the criterion; "Evidence" is what the tester keeps. Nothing here has been executed.

Preconditions. Test order A, delivered 3 days ago, value 40.00 EUR. Test order B, delivered 15 days ago. Test case C, submitted on a Friday at 23:50, no decision recorded, prepared by the delivery team. Three photographs under 10 MB, one fourth file, one file of 12 MB. A named person on the duty rota.

# Actor Action Expected result Proves Evidence
1.1 Tester as customer Open order A, start a refund The form lists the accepted file formats on the form itself REQ-014 / 1 Screenshot
1.2 Tester as customer Attach three photographs, then a fourth The fourth is rejected before upload, the message names the limit REQ-014 / 1 Screenshot
1.3 Tester as customer Attach the 12 MB file No expected result. Not stated in REQ-014, raised as a new open question before the session - Question id
1.4 Tester as customer Submit Case created. Tester notes the submission time REQ-014 / 2 Time noted
2.1 Pre-check, witnessed Order A is inside BR-04 and meets BR-11 Full refund approved, no person involved REQ-018 / 1 Case state
2.2 Tester as customer Submit a case for order B Not approved. The case reaches a person with the reason in BR-04's words. The customer is not told they have no rights [LAW] REQ-018 / 2, BR-04 Screenshot, message text
2.3 Delivery team, witnessed Show the record behind the approval in 2.1 The rule version is recorded REQ-018 / 3 Witness signature
3.1 Customer Care Open case C Assigned to the named person on the rota, not to a mailbox, group or queue. Escalation time: 48 clock hours after submission, or two working days if OQ-04 falls back REQ-015 / 1, REQ-014 / 2 Assignee name
3.2 Customer Care Read the due date on case C Earlier of five working days from submission or 24 hours from assignment. Pending OQ-02 REQ-015 / 2 Due date
3.3 Customer Care, then tester as customer Customer Care reassigns case C, then the tester reopens it as the customer, checking the inbox after each One message naming the holder and the date. One more after reassignment. None after the reopen REQ-015 / 3 Messages
4.1 Goods-in lead From the goods-in screen, reopen order A's case on day one after receipt Reopened without going through Customer Care REQ-017 / 1 Case state
4.2 Goods-in lead Try free text, then pick a reason and attach a dock photo Free text not accepted as the reason. Reopen needs at least one dock photo REQ-017 / 2 Screenshot
4.3 Customer Care Check the refund on the reopened case Not reversed. Case held by a named owner with a 48 hour clock REQ-017 / 3 Payment state
4.4 Goods-in lead Try to reopen on day six No expected result. Not stated in REQ-017 - Question id

Sign-off. Section 1 and 3: Head of Customer Care. Section 2: Head of Customer Care, with the delivery team as witness on 2.3. Section 4: returns goods-in lead. Out of this scenario: REQ-014 / 3 until OQ-05 is answered, REQ-016 until OQ-01 is answered, with REQ-016 / 3 as a witness step for legal and data protection.

Fourteen steps, and three of them say they cannot be judged yet: two have no expected result at all, and one waits on OQ-02. Send that version round before the session: three surprises on the day become three questions asked in advance.

The UAT script, blank

Scenario. [UAT-xx, one journey, named by the outcome] Preconditions. [Every test order and case with its dates and values. Every clock step becomes a precondition here, with who prepares it and by when]

# Actor Action Expected result Proves Evidence
1.1 [Who acts. A new section starts when this changes] [One action] [What the actor sees. If you cannot write it, write why, and raise the question] [REQ-xx / n] [What the tester keeps]
2.1 [System, witnessed] [The rule it applies] [The observable result] [REQ-xx / n] [Witness signature for anything no user can see]

Sign-off. [Per section: a person with a decision right in the stakeholder map] Out of this scenario. [Each criterion left out, and the open question or witness step that holds it]

Where does this method stop?

It tests a register against itself. Every expected result above comes from the case's own criteria, so a criterion that is wrong but clear passes straight through. That is what the review log and the definition of ready are for, before the script is written. It says nothing about non-functional criteria, which have their own piece, or about how many scenarios a release needs: one journey here covers eleven of fifteen criteria, a property of a small slice.

What it does reliably is move the arguments forward. A criterion that cannot produce an expected result would have produced an argument at the end of UAT, and you find it at the desk instead. The criteria themselves are in acceptance criteria that survive a sprint review, boundary rows in decision tables, and checks 02 and 13 in the scorecard. The full case is at /case.

Read next

All field notes · The portfolio guide · More in Requirements That Survive Review