Test evidence

Storing test evidence with the test results might be a good practice.

When one reads test results, and only knows about action and expected result texts, he/she has to trust that the test script actually did what is written in the texts given.

In order to tackle this uncertainty, evidence may be stored with test results. Doing so reinforces the credibility of the results, in as much as a human could check manually that the automatic test script did the right thing.

As introduced in the quickstart guide, the StepUserApi.evidence() method, available in ScenarioDefinition and StepDefinition classes, lets you save evidence while the test is executed.

Assertion routines defined in the Assertions class can be used to collect evidence as well. Set the optional evidence parameter to either True or a string describing what is being checked.

Test evidence is saved with the scenario JSON reports in the ‘evidence’ list of each action or expected result execution.