There’s plenty of stuff in a test script, but the two main things, without which the whole thing is pretty pointless, are:
- An Objective for the script
- The Test Steps to be executed by the tester
These go hand-in-hand. The Test Objective is a simple statement of what the script is intended to achieve. The Test Steps are the instructions to the tester as to what needs to be done in order to achieve the Objective.
The Test Objective
The Objective is the raison d’etre for the script. A test analyst will have examined the requirements for and design of the system and said “We need tests for this, this and this”. These test requirements are expressed in each test script as the Test Objective.
The Objective needs to be highly specific and succinct, or people will lose focus – for example:
- Demonstrate the handling of a telephone request for account balance
- Prove that anonymous payments are posted to the Unapplied Cash account
- Show that Gold customers more than three months delinquent are routed to the Debt Recovery process
A broader, more loosely-expressed Objective has a tendency to produce similarly loose scripts, with a consequent negative impact on the quality of the testing.
The Test Steps
The Test Steps are the series of actions that the tester needs to perform in order to achieve the Objective. Each action is accompanied by a brief summary of what should result from the action – the expected result(s). Not all of these results may actually be observable, either directly or indirectly. In such cases, the result exists to provide an awareness of what is happening so that the tester does not simply proceed blindly through the script but instead gains an understanding of how his actions are using the system to achieve the Objective. Where the result is observable, however, any deviation from it needs to be investigated.
An example Test Step sequence:
|TEST STEP||EXPECTED RESULT|
|Create a Gold-level customer with $2M credit limit and 90-day billing||Customer created;
Credit checked and approved;
Sales Manager assigned
|Load transactions for the customer to create an invoice and account balance between £10,000 and $25,000||Transactions loaded and priced;
Customer invoice created
|Click the “Settle” button||Settlement screen is displayed|
Note that the level of detail expressed in a test script is the subject of an entirely separate discussion.
Types of Test Step
Each of the steps in any test script is intended to serve one of four distinct purposes:
- Setup of the environment in readiness for testing
- Achievement of the Objective
- Verification of final or interim results
- Illustration of the process surrounding the test
These are the steps that get the data and environment established ready for the real business of testing. This is where you create a customer, buy items, create an invoice and do whatever else needs to be done to get everything in place ready for the steps that achieve the Objective.
These steps are directly targeted at doing what the Objective says the system needs to be proved to do. Once these steps have been executed, the system will have taken the data and environment created in the Setup steps and used it as the starting point for performing the action stated in the Objective.
These are the steps that are executed in order to show that something has happened or has been done. When used after a Setup step, they provide an opportunity to show that the data and environment are as they should be. Used after the Objective has been achieved, they provide the proof that the system has worked as expected. Either use provides a good point for the capture of test evidence using whatever facilities are available (the same is true of the Setup and Objective steps themselves). Test evidence may be a screen capture, say, of an online customer account statement, or the printed result of a database query.
The illustrative step exists solely to provide a link between the Test Steps and the process that surrounds the test script and that was used to create it. An illustrative step
- Describes an action that would be taken in the real world …
- … for which no test user action would be appropriate or necessary …
- … but which provides detail essential to a fuller understanding of the process surrounding the test script
For instance, “Call customer to confirm delivery method“. The “delivery method” will have been predetermined by the script objective and Test Steps, so there is no need for the tester to attempt to contact this mythical customer. However, knowing that the action would be performed in the real world helps those using the script to follow the overall process. The existence of illustrative steps in the script create a context for the other script steps and so provide
- The tester with a fuller description of the overall process that is being executed so that the test steps are not executed blindly but with an awareness of the surrounding process
- Analysts, designers and SMEs investigating apparent script failures with information necessary to relate the Test Script to the business and technical processes that they understand
Illustrative steps should be tagged with “(No Action)” to make it clear that the tester does not actually have to do anything. The result should be appropriate to the step action – in the example above, it might be “Customer requests courier delivery at extra cost“. In other cases it can acceptable to simply put “N/A“.
Overlapping Script Phases
The test step steps will generally occur in the order given above – that is, Setup, Objective and Verification, with Illustrative steps inserted when and where appropriate. However, this does not mean that there is a complete break between them, creating identifiably separate phases in the script. Overlapping of the step types is likely to occur in any, even slightly, complex script, such that an Objective step may be followed by a Verification step and then by another Setup step. Some simple scripts may not even have any Setup steps (e.g. scripts that demonstrate system configuration).
Here’s a fictitious example of a test script
|Objective:||Show that Gold customers more than three months delinquent are routed to the Debt Recovery process|
|[TYPE]||TEST STEP||EXPECTED RESULT|
|(Setup)||Create Gold customer||Customer created|
|(Setup)||Set Gold customer delinquency threshold to three months||Threshold set to three months|
|(Verification)||Go to customer details screen, view details, save screenshot||Customer is Gold level with three month delinquency threshold|
|(Setup)||Go to the order entry screen and create purchases for any amount – save screenshot||Invoice is created and customer balance updated|
|(Illustrative)||Customer’s billing date arrives (no user action required)||Transactions are priced and the customer invoice is produced (no action)|
|(Objective)||Pricing and billing run as automated processes overnight (no user action required)||Transactions are priced and the customer invoice is produced (no action)|
|(Verification)||Go to the customer account screen and confirm a new invoice for the transactions is present – save screenshot||Invoice is present and includes all the transactions, properly priced|
|(Setup)||Advance the system date to invoice due date plus three months and one day||Customer becomes delinquent and is tagged for Debt Recovery|
|(Verification)||Go to the customer account screen and confirm that the account is tagged for Debt Recovery – save screenshot||Customer account is tagged for Debt Recovery|
Test scripting is hard – let no-one tell you otherwise. We often find ourselves writing test scripts when only a bare description of the system exists and with only a sketchy understanding of how it’s meant to work. At the very least, by focusing hard on achieving
- A meaningful test Objective
- Broadly-executable test steps
we should be able to provide a sound platform for getting the system tested and thereby proving that the project has spent the customer’s money well.