How can a project manager specify a testing task statement of work for a fixed-price contractor?
I'm hiring a contractor to do testing in my project. All other works like design, architecture, programming, analysis are done in-house. All testing activities have to be accomplished by the contractor, including (if necessary) test planning, test scripts, test cases, etc.
The problem with working fixed-price for this type of work is establishing where the contractor will stop: you cannot write "untill all tests have passed" as system quality lies outside his responsibility.
A statement of work usually contains the focus on the work to be done, in this case you will need to specify
* Test Planning
* Test Development
* Test Execution
* Follow-Up
and more general stuff like planning, Hand-over, the place to be, resources by name, payment schedule, etc.
Test Planning. Can the contractor start from a list of functional and non-functional requirements? Or will he need to create a set of test requirements? 'What to test' should be as clear as possible, eg functional correctness, security, usability, performance, portability, documentation ... Also important here is Test design Coverage: it might look trivial that this is 100%, but maybe the core processes/features of your system need more attention (read # of tests written against them) than the simple admin functions for, I don't know, creating parameters or something. If it is of less importance, you could state that these functions are out of scope for this testing assignment.
Test Development. How will the test scenario's be documented: do you have a tool available, or will it be in a spreadsheet or word processing program. If you already have a template available (or the contractor) than add it to the SOW. If the contractor needs to create one, than a review and acceptance process should be described. The same counts for all the tests: who will review them and how will they be accepted. Decide upon the level of detail: will the tests be high level, so that a knowledgeable person needs to execute the tests, or will you require detailed testprocedures, so that anybody without prior knowledge, can perform these tests.
If automated testing is required, than also here the details of the technique and review process of test scripts should be described and agreed to.
Finally, for the gathering of testdata, your people will have to supply them in some form, so the contractor can create specific test cases: who at your side will be available: business, functional analysts ...
Test Execution. How many testruns must be performed? One run is doing each test once, regardless of the outcome. What happens when there are blocking errors that prevent the tests to be performed? What may the contractor expect as to the resolution so he can continue the test execution (he shoulnd't be sitting idle). How will bugs be reported (a tool, a bug log in a spreadsheet, ...), what info needs to be supplied in a bug ticket and how will it be communicated with the technical people. If you are doing scrum, then tests are usually developed and (re-)executed as part of the sprint. In that case the number of sprints will have to be determined.
Follow-up. Reporting like test execution coverage, passed product, etc. Maybe also bug status reporting, # bugs per application area, ... Attendance to weekly status meetings or daily stand-up.
Finally some other stuff:
Planning. List the relevant milestones of the project and how this contractor should allign his work to it. Or add the specific milestones for quality assurance.
Hand-over. Knowledge transfer between consultant and client, so that you can continue when the contractor has left.
People. Who is the resource(s) of the contractor that will do the job. Sometimes specialised test consultants send in a senior profile for test planning and development, but junior (= often cheaper) profiles for test execution.
Hope this was useful
Write it exactly like how you started spelling out the question in more detail in your second paragraph. Make sure to include the iterations that testing takes from when a bug is first reported, to someone fixing it, to it being retested, etc. as well as specifying how bugs should be reported and what information the bug report should contain.
Very good comments from others.
2 points to add:
1) You have to watch when the same person is planning and executing the test. If it is Time and Materials they may plan for too many tests. Fixed cost too few.
2) the testers can't control the quality of the code so you may need a per cycle or per case charge. Otherwise you may reach the quantity of agreed to tests but still have more work to do.
2.5) Medium term you should consider owning test definition, if not execution. This may not be feasible short term.
