Background
My background is a user acceptance tester (end user / quality assurance testing) for a UK financial services company). I am one of a team of eight testers, working in a department called Model Office (literally because we model the activities of the company).
My role is that of a Project Leader, and as such one of my responsibilities is to formulate, create, and document testing strategies for the developmental projects undertaken by my company.
Because we do end user testing, our primary concern is proving the functionality of the application, and demonstrating this functionality to the business areas of the company who will use the application.
This is achieved by a series of controlled testing processes, which start, with the creation of the testing strategy.
This document is explained over.
Testing Strategies
The testing strategy is a word processor document that defines exactly what Model Office will do when we are testing the development project. This document circulated around the relevant business areas for approval and is ultimately signed and dated by a senior manager of each area to formally approve the contents.
This circulation and approval is done to mitigate the risk that Model Office is testing the application in such a way that the business areas will not sign off the application, because it has not been proven to their satisfaction.
Due to the nature and contents of the strategy document (which will be detailed later) Model Office will not commence testing unless the testing strategy document has been formally approved.
Contents
Every development project, and the application being developed, is different, and so the testing strategy formulated for each project is different.
However, some elements of the testing strategy are pretty common to all development projects, and I have listed those below.
One thing which I always ensure is present, is that the document is well formatted and professional looking. This sounds petty, but these documents will be circulated around quite a few people for inspection, and will also be circulated at least twice (once as a draft, and once as final for signing), so care should be taken to ensure that the document does not reflect badly of you.
Should formatting would be;
All of these will greatly contribute to appearance of the document, and are not difficult to achieve using a modern word processor.
I use Word 6 in work and Word 97 at home. Using Word it is the work of 30 minutes to set up a template with the necessary styles (Headings, etc) to use for all documents. The remaining sections can be easily automated (table of contents is simple, and can easily be set to update automatically when printing).
I have such a template available, and it is available from my company site (www.amjec.co.uk ) on the testing documents page - http://www.amjec.co.uk/testdocs.htm
Scope
Each strategy document should start with a scope of the testing strategy.
This section details the precise extent of the testing, and allows the reader to instantly identify;
I also generally include the exclusions from the testing, along with precise details as to why those areas are not being tested. I find that this avoids the situations where people assume that something will be tested, and that they ‘just haven’t read it in the strategy’.
Project Management
This section of the strategy defines how I shall run the testing project. As the testing is (strictly speaking) done on behalf of the users, we include this so that they have a feeling of control. I also find that the business areas knowing how the project will be ran is a comfort factor to those people.
This section tends to have three subsections.
Problem Management
This section should contain how problems;
This section of the document details how changes to the application will be handled.
As most changes will be made as a result of a problem being found during testing and raised to development, the change processes must dovetail precisely with the problem management processes.
This section should contain;
This section details the roles and responsibilities of the business areas involved in the development / testing project.
This section should always, at least, include the areas that will be approving the testing strategy.
The inclusion of this section gives Model Office the commitment from the business areas that they will support the testing efforts, and will ensure that resources are available for checking of results etc.
I cannot give any examples for this section, as the structure of our companies’ will be different. However, if an example is necessary, I have made one available, from my company site (www.amjec.co.uk ) on the testing documents page - http://www.amjec.co.uk/testdocs.htm
Testing Strategy
This section of the document details all of the testing processes that shall be executed to prove the functionality of the application. This is also broken up into smaller sections;
The order in which all the processes shall be executed, and the reasoning for this order (priority, risk, size, importance etc).
Whilst this is different every time, the first phase of testing I always specify is ‘stabilisation testing’.
This testing is designed to use as much of the system functionality as possible, in as short a time (generally no more than two days).
What I want to prove with this type of testing is that the software has been installed well enough to merit the complete testing strategy being executed, and that time spent testing will not wasted due to an obvious installation error.
Process a lot of test cases (perhaps entering the details of many clients), without the application falling over.
However, an old version of a record layout (e.g. postal code is not in record) has been installed. The first time the ‘group clients by post code’ report is run against those cases they fail.
This means that all the times spent entering those clients has been wasted.
This problem could easily have been rectified by entering sufficient clients to meet the criteria for all the reports, and then running all the reports to ensure that the database layout and report code is correct.
This is stabilisation testing.
The purpose of the order of testing is to define (and have agreed) the areas of the application which will be tested first.
We do not want to test the low priority, low use, and low risk areas of the application first, as there is less likelihood of finding significant errors.
We would rather test the high risk / priority areas, to ensure that these are functional and bug free (we can probably wait to fix a spelling mistake, but not an error in an important calculation)
Testing Methodologies
A testing methodology is basically ‘the method that shall be used for testing’
For each module of testing you should identify a testing methodology, and explain why this is the most suitable methodology for the module.
As part of the methodology you should also detail how the results of the testing will be documented (screens which will be printed, reports that will be produced, jobs that will be ran etc)
Three useful testing methodologies can be applied to most testing projects are identified below.
Stabilisation Testing
We have already discussed this methodology
Guerrilla Testing
This is testing that opportunistically seeks to find severe bugs wherever they may be. Guerrilla testing is much less planned than the previous kind.
The extent of planning is usually "Mr(s) X , you’re the most experienced tester, and you have a knack for finding bugs. From now until the end of the project, bash away at whatever seem to be the shakiest parts of the application."
Guerrilla tests are usually not documented or preserved.
I must at this point acknowledge Brian Marick (Testing Foundations) and Steve Stukenbourg's (Pure Atria) article "The Test Manager at the Project Status Meeting" for the above quote. I have always termed this as 'unstructured testing', but there description is much better.
Unfortunately, I have lost the URL for this article, but do have a copy. If you want a copy, it is available from my company site (www.amjec.co.uk ) on the testing documents page - http://www.amjec.co.uk/testdocs.htm
Better still, mail me the URL, and I'll post a link to it.
This is testing carried out to prove that existing functionality of an application has not been impaired.
It usually consists of running a series of tests which have previously been proven as accurate, and ensuring that these cases continue to yield the same results.
Regression Testing is usually down towards the end of the testing project, when (hopefully) most of the major bugs have been fixed..
Test Data / Test Cases Modules
The final section (and usually by far the largest) is the details of the precise test cases / data that shall be passed through the application.
This section should contain all the test cases and the anticipated results of passing those cases through the application.
I will not go into great detail, as this is both the most self-explanatory section, and also the most subjective.
However, I can relate a short ‘war story’ concerning test cases from my own experience.
Previously we have requested business areas to supply test cases that we would process, and return for checking.
This was to allow the users to ensure that all the specifics they needed to see in the application were functioning correctly.
However, what tended to happen would be each area would overlap test cases, as there are always areas of common interest.
For example, one of our applications produces the legal documentation between our company and the client when a new life assurance policy is taken out.
These are very important documents, and many areas would be interested in seeing that they were correct;
To circumvent this, I now create all the test cases, include them in the testing strategy, and when I circulate this document state in the memo that the details of the test cases may be changed, or if necessary more can be created.
The benefits of this are twofold;
Home | Read my CV | Search the WWW | Pictures
User Acceptance Testing | Learning to Program in C