Bug convergence is a milestone in the stabilizing phase of a project life cycle. It is a point at which the new bug rate (found during an application testing) drops below the bug resolution rate. At this point, the rate of bugs resolved exceeds the rate of new bugs found during application testing. Therefore, the actual number of active bugs decreases as the number of bugs resolved increases. After bug convergence, it is important that the number of bugs decreases until zero bug bounce (another milestone).
Recommended Interim Milestones
Bug Convergence
Bug convergence is the point at which the team makes visible progress against the active bug count. That is, the rate of bugs resolved exceeds the rate of bugs found.
Because the bug rate will still go up and down—even after it starts its overall decline—bug convergence usually manifests itself as a trend rather than a fixed point in time.
After bug convergence, the number of bugs should continue to decrease until zero-bug release. Bug convergence tells the team that the end is actually within reach.
Zero Bug Bounce
Bug Convergence and Zero Bug Bounce is the point in the project when development finally catches up to testing and there are no active bugs or the moment. Figure 11 illustrates a zero-bug bounce. After zero-bug bounce, the bug peaks should become noticeably smaller and should continue to decrease until the solution is stable enough for the team to build the first release candidate.
Careful bug triaging is vital because every bug that is fixed risks the creation of a new bug. Achieving zero-bug bounce is a clear sign that the team is in the endgame as it drives to a stable release candidate.
Note that new bugs will certainly be found after this milestone is reached. However, zero-bug bounce marks the first time when the team can honestly report that that there are no active bugs—even if it is only for the moment—and it focuses the team on working to stay at that point.
Release Candidates
A series of release candidates are prepared and released to the pilot group. Each release candidate is an interim milestone.
Other features of a release candidate are:
- Bug Convergence
- Each release candidate has all the elements it needs to be released to production.
- Building a release candidate tests its fitness for release, that is, whether all necessary pieces are present.
- The test period that follows generation of a release candidate determines whether a release candidate is ready to release to production or whether the team must generate a new release candidate with the appropriate fixes.
- Testing release candidates, which is done internally by the team, requires highly focused and intensive efforts, and focuses heavily on flushing out showstopper bugs.
- Testing requires a triage process for resolving any newly discovered bugs.
- It is unlikely that the first release candidate will be the one that is released. Typically, show stopping bugs will be found during the intensive testing of a release candidate.
Pre-Production Test Complete
The focus of this interim milestone is to prepare for a pilot release. This interim milestone is important because the solution is about to “touch†the live production environment. For this reason the team must test as much of the entire solution as possible before the pilot test begins. Activities that must be completed during this interim milestone are:
- Evaluate test results against success criteria.
- Complete site preparation checklist and procedures.
- Complete implementation procedures, scripts, and load sets.
- Complete training material.
- Resolve support issues.
- Complete and test the rollback plan.
The pre-production test complete interim milestone is not complete until the team ensures that everything developed to deploy the solution is fully tested and ready.
User Acceptance Testing Complete
User acceptance testing and usability studies begin during the development phase and continue during stabilization. These are conducted to ensure that the new system is able to successfully meet user and business needs. This is not to be confused with customer acceptance, which occurs at the end of the project.
When this milestone has been achieved, users have tested and accepted the release in a non-production environment and verified that the system integrates with existing business applications and the IT production environment. The rollout and backout procedures should also be confirmed during this period.
Upon approval of release management, software developed in-house and any purchased applications are migrated from secure storage to a pristine archive location. In MOF, this is known as the Definitive Software Library (DSL).
Release management is responsible for building releases (assembling the release components) in the test environment from the applications stored in the DSL.
User acceptance testing gives support personnel and users the opportunity to understand and practice the new technology through hands-on training. The process helps to identify areas where users have trouble understanding, learning, and using the solution. Release testing also gives release management the opportunity to identify issues that could prevent successful implementation.
Pilot Complete
During this interim milestone, the team will test as much of the entire solution in as true a production environment as possible. In MSF, a pilot release is a deployment to a subset of the live production environment or user group. Depending on the context of the project, a pilot release can take the following forms:
- In an enterprise, a pilot can be a group of users or a set of servers in a data center.
- In Web development, pilot release takes the form of hosting site files on staging server(s) or folders that are live on the Internet, only with a test Web address.
- Commercial software vendors, such as Microsoft, often release products to a special group of early adopters prior to final release.
