Reliability Growth Models

Software Reliability Models

Software reliability models help quantify how reliable a software system is and how its reliability improves as testing progresses. Software reliability growth models mathematically describe how reliability changes as we execute the software, observe failures, and correct the underlying faults.

These models help answer questions such as:

• How many more failures are expected if we continue testing?
• How long do we need to test to reach a target failure intensity?
• When can we realistically stop testing and release the product?

In general, a reliability growth model expresses reliability or failure intensity as a function of time or of the number of failures detected and removed so far.

Musa’s Basic Execution Time Model

The basic execution time model, developed by J. D. Musa, is one of the foundational reliability growth models and is based on execution time (CPU time actually spent running the software) rather than calendar time (Musa & Okumoto, 1983).

It is widely used because it is:

• Practical and conceptually simple.
• Based on parameters that relate clearly to real-world quantities.
• Useful for reasonably accurate reliability prediction.

Execution time can later be converted to calendar time (for example, by using average CPU usage per day) if needed for planning and release decisions.

Failure Intensity as a Function of Mean Failures

Let:

• λ(μ) – current failure intensity (failures per unit CPU time).
• λ₀ – initial failure intensity at the start of testing.
• μ – mean (expected) number of failures experienced so far.
• V₀ – total expected number of failures over the life of the software (as time → ∞).

Musa’s model relates failure intensity to the number of failures already experienced using:

λ(μ) = λ₀ × (1 − μ / V₀)  (Equation 5.1)

As more failures are detected and removed (i.e. as μ increases), the failure intensity λ(μ) decreases approximately linearly, approaching zero when almost all faults have been removed (see Figure 43).

The rate of change of failure intensity with respect to the mean number of failures is:

dλ / dμ = − λ₀ / V₀  (Equation 5.2)

That is, the failure intensity decreases at a constant rate as failures are discovered and fixed, which is why this model is often called a linear reliability growth model.

Figure 43: Failure intensity λ(μ) decreases linearly from the initial value λ₀ towards zero as the mean number of experienced failures μ approaches the total expected failures V₀.

Mean Failures as a Function of Execution Time

Let τ denote the execution time (for example, total CPU time used for testing). From reliability theory and Musa’s assumptions, the relationship between the mean number of failures and execution time is:

dμ / dτ = λ₀ ( 1 − μ / V₀ )  (Equation 5.3)

Solving this differential equation gives the mean number of failures as a function of execution time:

μ(τ) = V₀ ( 1 − e^{−λ₀ τ / V₀} )  (Equation 5.4)

This behaviour can be interpreted as:

• Initially τ = 0: μ(0) = 0 (no failures observed yet).
• As τ → ∞: μ(τ) → V₀, i.e. we approach the total number of latent faults that can be exposed by testing.

Failure Intensity as a Function of Execution Time

Substituting μ(τ) from Equation 5.4 into the linear relation λ(μ) = λ₀ (1 − μ / V₀) gives failure intensity directly in terms of execution time:

λ(τ) = λ₀ e^{−λ₀ τ / V₀}  (Equation 5.5)

This is an exponential decay: as more CPU time is used in testing, the failure intensity decreases rapidly at first and then more slowly. Figures 43–45 (in the text) illustrate the relationships between μ, λ, and τ.

Figure 44: Failure intensity λ(τ) decreases exponentially from λ₀ as execution time τ increases.

Additional Metrics from Musa’s Model

Suppose the current failure intensity is λ₁ and the target (desired) failure intensity is λ₂ with λ₂ < λ₁. Using the Musa model, we can compute:

• Additional expected failures Δμ to reach the target:
  Δμ = (V₀ / λ₀) ( λ₁ − λ₂ )
• Additional execution time Δτ required:
  Δτ = (V₀ / λ₀) ln ( λ₁ / λ₂ )

Example 5.1: Applying Musa’s Model

A program is expected to experience V₀ = 100 failures in infinite time. It has currently experienced μ = 40 failures. The initial failure intensity was λ₀ = 10 failures/CPU hour.

Given:

• V₀ = 100
• μ = 40
• λ₀ = 10 failures/CPU hour

1. Current failure intensity at μ = 40 failures
   Using Equation 5.1:
   λ(μ) = λ₀ (1 − μ / V₀) = 10 (1 − 40 / 100) = 10 × 0.6 = 6 failures/CPU hour
2. Failures experienced after 10 CPU hours
   Using Equation 5.4 with τ = 10:
   μ(10) = 100 (1 − e^{−10×10 / 100}) = 100 (1 − e⁻¹) ≈ 100 (1 − 0.367) ≈ 63 failures
3. Failures experienced after 50 CPU hours
   For τ = 50:
   μ(50) = 100 (1 − e^{−10×50 / 100}) = 100 (1 − e⁻⁵) ≈ 100 (1 − 0.0067) ≈ 99 failures
4. Failure intensity at 10 and 50 CPU hours
   Using Equation 5.5:
   λ(10) = 10 e^{−10×10 / 100} = 10 e⁻¹ ≈ 3.67 failures/CPU hour
   λ(50) = 10 e^{−10×50 / 100} = 10 e⁻⁵ ≈ 0.067 failures/CPU hour
5. Additional failures and execution time to go from λ₁ = 3 to λ₂ = 1 failures/CPU hour
   Using the additional metrics:
   Δμ = (V₀ / λ₀) (λ₁ − λ₂) = (100 / 10) (3 − 1) = 10 × 2 = 20 failures
   Δτ = (V₀ / λ₀) ln (λ₁ / λ₂) = (100 / 10) ln (3 / 1) = 10 × ln 3 ≈ 10 × 1.0986 ≈ 11 CPU hours

Figures 44 and 45: As execution time (τ) increases, the mean failures μ(τ) approach V₀, while the failure intensity λ(τ) decays exponentially towards zero.