Product reviews are an important feature of any project but too often the results of those reviews is a mass of poorly coordinated, uninformative documents. Here are some hints on how to do it better.
Any project will produce things. That’s what they do. The main thing that they produce is what PRINCE2 calls the Project Product, this being whatever it was that the project was required to create — a technology system, a housing estate, a process for organising sheep dog trials. On their way to producing these things, the project also produces other things — interim products that will be put together to create the project product. In the IT project, these would be such things as technology subsystems, user documentation and training guides, server and networking hardware, hosting, support and maintenance contracts — the list goes on. And during the processes that create all these products, the project inevitably produces a storm of documentation.
IT projects are particularly good at this process of document production, because any IT project is essentially a series of transformations. The requirements are transformed from ideas in the customers’ heads to requirements specifications documents. These are then used to create functional design documents and technical design documents that describe a potential system that will deliver a service according to those requirements. Those building the system will take the functional and technical designs and use them to produce the components that will go into the actual system, and to integrate those components to form the complete delivered system. Inevitably, more documentation will be produced as all this goes on, adding to the overall flurry. And, as we have seen elsewhere, different versions of each of these pieces of documentation will exist at different times. And they all need to be verified before being released for general use.
Everything that is produced, whether document or product, needs to be checked in order to confirm that what has been created is what was required to be produced. There are various ways to do this, but, generally, the main two are testing, which is applied to a product that does something (and which will be addressed elsewhere when I can find the time), and reviewing, which is applied to a product that doesn’t — usually a document.
The details of the review process will also be worked out elsewhere but for now it’s enough to know that it consists of one or more people performing a visual inspection of the product in order to
- Determine the extent to which it complies with the requirements of that type of product (where more than one of that type will be produced, such as test plans)
- Determine the extent to which it complies with the requirements of that specific product (for instance, the test plan for version X of component Y)
- Document the changes required to make it conformant
In addition to this, reviews performed after the first —re-reviews — will also need to confirm that changes identified by the reviews performed so far have been applied correctly, since once all changes have been applied the product can be released.
The results of the each review is recorded in a Review Document. This serves a number of purposes:
- A means of communicating to the product owner those changes that need to be made
- A means of confirming that the changes made have been checked and are satisfactory
- An audit trail of changes made to the product
For each product reviewed, each Review Document needs to record the following information:
- The identity of the product that was reviewed
- The version of the product that was reviewed
- The changes that were identified as being required
Each change requires the following information:
- Details of what needs to be changed
- An optional comment by the person making the change relating to it and how it was applied
- The version of the product in which the change was applied
- The version of the product in which the change was confirmed as correctly applied
This would suit a database, but people seem to enjoy using documents instead, so we’ll work with those for the rest of this article.
The Review Cycle
The Initial Review
The first review is a review of the product only, and any changes required will be included in the initial Review Document. This is given to the product owner (or whoever will be making the change) for him to use to make the required changes.
Once the changes have been applied, the revised product — with an updated version number — is presented for re-review. The intention of a re-review is to confirm that the changes have been applied correctly and that the product is now fit for release to the wider world. If this is the case then the initial Review Document will simply be updated with the relevant confirmation and closed. The product can then be released. However, it may turn out that
- One or more changes have not been applied correctly
- Further changes are identified
Each of these cases is handled separately in order to maintain the consistency of the process and the integrity of the review documents.
If any of the changes were not applied correctly the Review Document is updated for those changes that have been applied correctly. It is then returned to the product owner so that the outstanding changes can be applied or the necessary corrections made.
If further changes were identified a new Review Document is created for the new changes and the previous Review Document updated for those changes that have been made. This may seem a gratuitous creation of Review Documents, but it has a number of advantages: ((To be honest, some of these reasons are a bit thin. However, I’ve worked on projects where a single Review Document — albeit versioned — was used throughout the review cycle and I’ve worked on projects where a new Review Document was created whenever additional changes were identified, and the latter was much easier to operate.))
- It allows each change to be easily tracked to the correct version of the product in which it was originally found
- It allows metrics to be derived for the review process (managers like metrics)
- It isolates the changes for each reviewed version of the product
- Where paper Review Documents are used, it helps prevent incomplete changes being lost in the chaff of completed changes
- In the case of immature products in rapidly changing environments, it reduces the impression that the review process is dragging on by providing a definite sense of progress as Review Documents are closed down once their list of changes is confirmed complete
Both of the above may happen in a re-review session, meaning that several Review Documents may be open at any one time. Note that, whilst it is possible to simply add any new change requirements to the existing Review Document, this can quickly lead to a large and unwieldy document should several review cycles need to be undertaken, so the model of using a new Review Document should be adhered to. ((Or you could adopt a hybrid model wherein a new Review Document is created whenever all outstanding changes have been completed but more have been identified — it’s up to you. However, I’d advise strongly against keeping a single Review Document across all reviews (which becomes unwieldy) or using a model that duplicates changes across Review Documents (which causes confusion).))
Review Document Structure
The name of the Review Documents needs to be appropriately structured so that all the Review Documents relating to the series of reviews that lead to a particular version of a published product version can be tied together. The name should be constructed so as to convey the following information:
- Target product
- Target product version
- Sequence number of the review
This yields a name such as System Operations Manual 1.4.2 Review 1. We are thus able to locate all review documents that led to pre-publication changes to that version of that document.
Further details are revealed in the Review Document header.
- Draft product version reviewed — so we know exactly what was reviewed
- Review Document closure date — so we know whether or not there are any outstanding changes to be applied from this Review Document
- Whether a new Review Document was raised — so we know that there is at least one further Review Document which may contain outstanding work
|Product Reviewed:||System Operations Manual 1.4.2 DRAFT 1.1.0|
|Review Document Closure Date:||15 November 2009|
|Additional Changes Raised?||Yes|
Closure of the Review Process
There are two closure points in the review process. The first occurs when all changes required by an individual Review Document have been confirmed as having been applied. At this point, the Review Document Closure Date is updated in the Review Document header and the document filed appropriately. The second occurs when all Review Documents for the target product version have been closed. The product can now be released and work can begin on the next version.
Sample Review Document
A sample Review Document template, a completed example and descriptions of the document fields can be found here. These can be adapted to fit your specific needs, but, again, I’d advise that the basic review/re-review cycle and Review Document set for a particular product version be retained.
It may seem that the process and document management system outlined above unnecessarily complicates the performance of reviews and the maintenance of the resulting Review Documents. However, the need of any project to record, store and reference information that it generates requires a formalised, observable and repeatable system that works. Experience has shown that many document management systems for product reviews created by projects simply do not work; this one will.