Software Reliability Testing: Analysis of Principles, Metrics and Plan

This report provides a comprehensive overview of software reliability testing, starting with the core principles of reliability in software engineering, including failure causes, wear out, and redundancy. It discusses the challenges in reliability testing, such as unidentified bugs and the need for reliability growth prediction. The report details methods for measuring reliability, including mathematical modeling and various reliability measurement processes like inter-rater reliability and test-retest reliability. It outlines the steps to develop a test plan, emphasizing product analysis, test strategy design, and resource planning. Furthermore, it includes a discussion on test cases, providing examples and highlighting the importance of clear descriptions and expected results. The report concludes by differentiating normal verification testing from reliability testing, underscoring the importance of proper management and planning in reliability testing to ensure the development of robust and failure-free software.

Running head: SOFTWARE RELIABILITY TESTING
SOFTWARE RELIABILITY TESTING
Name of the Student
Name of the University
Author Note

SOFTWARE RELIABILITY TESTING 1
Abstract:
The purpose of this study to discuss about the reliability in software engineering. Presently a
very important role is being played by the computers I n our daily life and high quality
software is always needed there Although, to test a software completely is not possible; but
proper testing and sufficient testing will improve the reliability of a software to great extent.

2SOFTWARE RELIABILITY TESTING
Table of Contents
Introduction:...................................................................................................................2
1 Principles of reliability in software engineering:.....................................................3
2 Issues and challenges:..............................................................................................4
3 Measuring reliability:...............................................................................................5
4 Reliability Measuring Process:................................................................................6
5 Steps to Develop a Test plan:..................................................................................8
6 Test case:................................................................................................................12
7 Example test plan:..................................................................................................15
Scenario:...................................................................................................................15
8 Determine reliability of system:............................................................................16
9 Normal testing for verification vs. reliability testing:............................................17
Conclusion:..................................................................................................................17
References:...................................................................................................................18

3SOFTWARE RELIABILITY TESTING
Introduction:
Reliability testing in software system testing is a technique for testing, which relates
to test the ability of a software to the function of a consistently given environmental
condition, which help the uncover issues in software functionality and design. The purpose if
this paper is to describe about the reliability in software testing. At present the reliability,
testing is very important as the systems are getting fast day-by-day and the system requires
failure free software. The paper will discuss about the principles of reliability, the challenges
and also it will discuss about the test plan with a suitable example. There are several steps in
the testing phase, the paper will also discuss about that. This paper will try to give an idea
about reliability of software and the models and metrics used for that. The paper will also
focus on how the reliability of a software can be improved.
1 Principles of reliability in software engineering:
Reliability in software engineering can be defined as, in a specific period within a
specified environment, probability of failure-less operation of a software. The mechanical
and electronic parts can become old with their usage and time but software will not wear out
or rust in its life cycle. The software will remain same unless the intentional change or up
gradation (Wohlin et al., 2013). Reliability is a part of the both non-functional and functional
testing such as load, stress, security and network testing. The operational profile is able to
help the guide managerial as well as the engineering decisions throughout life cycle of the
product, including requirements of the specification, design, testing and
implication.Example- Mobile software. The core principles of reliability in software
engineering are as follows:
 Failure cause
 Wear out

4SOFTWARE RELIABILITY TESTING
 Repairable system concept
 Time dependency and life cycle
 Environmental factors
 Reliability prediction
 Redundancy
 Interfaces
 Failure rate motivators
 Built with standard components.
The activity diagram for the reliability testing is given below:

5SOFTWARE RELIABILITY TESTING
Fig: Activity diagram
2 Issues and challenges:
At present, for the reliability testing of a particular software, huge challenges can
come in software system process. In general, it is believed that at the reliability testing phase
of a software some unidentified bugs can come which is one of the biggest challenges faced
by software testing currently (A and K, 2013). The reason behind this is, after passing the
present reliability test of a software there is no guaranty that no error or bug will come in the
future. The current challenges and issues are as follows:
 Reliability growth prediction and assessment
 Reliability estimation based on failure free working
 Reliability claims based on process quality
 Fault tolerance based on design diversity
 Formal verification.
3 Measuring reliability:
The major issues involved at the time of measuring reliability of a software are as
follows:
 Estimate the maintenance process of the stability of the software.
 Estimate the test suit efficiency for detecting the residual faults.
 Forecast and estimate the exposure risk for residual fault in the system
software during the operation.
 Focus more accurately scare identification of fault resources in software.
Mathematical Modelling in Software Reliability:
m (n) is the expected number of faults removal by nth test case.

6SOFTWARE RELIABILITY TESTING
1. Exponential model: m(n+1)-m(n)=b(a-m(n))
2. Imperfect Debugging Model: m(n+1)-m(n)=b(ap-m(n))
3. Generalized model: b(n+1)= bp/ 1+ (1-r)(1-bp)n+1/r
4 Reliability Measuring Process:
Reliability is basically a measure of consistency of one method or metric. Every
method or metric that the tester uses, including methods to uncover the usability problems in
the interface as well as expert judgement, have to be assessed for the reliability (Juristo,
Moreno and Vegas, 2013).
Even before establishing the validity, the tester can establish the reliability. One of the
most reliable software model for reliability is The Jelinski Moranda model. Many other
existing models are extensions or variants of this model, which is the basic. The assumptions
that are exist in this model are:
 The program contains n number of the initial faults, which is a fixed constant
but unknown.
 The intervals of time between the failure occurrences are independent to each
other.
 A fault that is corresponding is removed confidently whenever any failure
occurs.
The rate of program failure at the interval of ith is:
λ (ti) = Ø [ N – (i-1) ], i=1,2,…..N
Where,
Ø = proportional constant
N = initial number of faults

7SOFTWARE RELIABILITY TESTING
ti = distance of time between (i-1)th and (i)th failures.
The most common four ways to measure reliability for an empirical metric or method
are:
 Inter rater Reliability: The extend in which the observers or raters make
response in same way to a phenomenon that is previously given, is a measure
of reliability. Examples:
o The interface problems is being identified by the evaluators.
o The severity of any problem, Experts rating.
 Test retest Reliability: It is the most important part in software testing that
the tester must have to enable the ability of retesting of the software system
(ArXiv, 2015).
 Internal consistence Reliability: This measures how consistent the
participants are in responding to a set of the items.
 Parallel Forms Reliability: It defines when same or similar type of results are
getting from the variation of slight on the evaluation method or question too
establishes reliability (Yamada and Osaki, 2014).

8SOFTWARE RELIABILITY TESTING
Figure: Reliability measurement process
5 Steps to Develop a Test plan:
A test plan can be defined as a detailed document which outlines testing objectives,
resources (software, manpower, hardware) required for the testing process, testing strategy,
testing estimation, test schedule and test deliverables (Amin, Grunske and Colman, 2013).
The most important part in software testing is test plan. One of the most referable paper about
operational profile development is from John Musa. It break downs and describes the system
into five profiles that are different from each other. The profiles are given in the picture
below.

9SOFTWARE RELIABILITY TESTING
Figure: SRE process
The benefits of test plan are as follows:
 Guides the thought of developers and relate that with the actual rules that need
to be followed.
 Help the people who are outside of the development and testing (such as
customers and business managers) to understand about testing.
 Helps the developers to assume an idea about how much effort they will have
to give.
 Some important aspects such as test scope, test estimation, test strategy are
involved in test plan. Therefore, the management can review the plan and can
reuse it for other projects (Porru et al., 2017).

10SOFTWARE RELIABILITY TESTING
To create a test plan there are eight steps. The steps are as follows:
1. Product analysing: To analyse the product
2. Design the strategy of testing: Expert have to design a strategy for testing
3. Define test objectives: The expert have to define the objectives for testing
4. Define the test criteria: The criteria that is needed for testing also have to be
define.
5. Resource planning: Planning for the needed resources
6. Test environment planning: Planning for the environment of the test
7. Schedule and estimation: Assume within how much time the test will be done.
8. Determine the test deliverables: What variables will be needed for the testing.
Fig: Steps Test Plan
Analyse the
product
Design test
strategy
Define test
objectives
Define test
criteria
Resource
planning
Plan test
environment
Schedule and
estimation
Determine
test
deliverables

11SOFTWARE RELIABILITY TESTING
Compared to the other testing types reliability is costly. So proper management and
planning is required at the time of doing reliability testing and this includes all the testing
processes such as test schedule, test environment etc. (Okamura, Dohi and Osaki, 2013).
To begin the reliability testing the tester has to follow these things:
 Establish the reliability goals
 Develop the operational profile
 Planning and executing tests
 Use the test results for driving decisions.
The reliability testing can be performed in three categories. They are - Measurement,
modelling and Improvement (Birolini, 2013).
The major parameters that are involved in the software reliability testing are:
 Probability of failure free operation
 Time length of failure free operation
 The environment in that it is executed.
Figure: Reliability