CO4403 OOSD Assignment: Critical Analysis of Software Testing Paper

This report presents a critical analysis of a research paper focusing on software testing principles, particularly the fifteen best practices outlined for effective software development. The analysis delves into each principle, including simplicity, readability, single responsibility, and avoiding over-protectiveness, evaluating their validity and impact on the testing process. It also covers the importance of test behavior, maintainability through refactoring, the necessity of test failures, reliability, and the balance between happy and sad tests. Further examination includes how tests should not dictate code, the significance of fast feedback, the 4-phase test design, simplicity of fixtures, test dependency, and the use of test doubles. The report concludes with a summary of the research's contribution to the field, emphasizing the quantification of quality aspects related to testing principles for improved application within the software testing process. Desklib provides a platform for students to access similar solved assignments and past papers.

Running head: CRITICAL ANALYSIS OF RESEARCH REPORT
CRITICAL ANALYSIS OF RESEARCH REPORT
Name of the Student
Name of the University
Author Note

1
CRITICAL ANALYSIS OF RESEARCH REPORT
Table of Contents
Assessment of the paper contribution:.................................................................................................2
Critical summary of the research carried out and presented in the paper:...........................................7
References:...........................................................................................................................................9

2
CRITICAL ANALYSIS OF RESEARCH REPORT
Assessment of the paper contribution:
Here in this section a discussion on the interpretation of the findings and key issues that has
been raised in the paper has been provided.
In the paper the authors have provided some of the most important issues in the software
testing and also provide some of the best principles that should be followed for effective software
testing.
The authors have mentioned a total of 15 best principles for effective software testing which
is required for effective software development (Bowes et al. 2017). Hence this will be discussed in
details in this context.
Principle #1: Simplicity
According to the first principle, simplicity is one of the first and most important criteria of
proper software testing. This point is a valid one because if code becomes extremely complex, then
not only it is hard to organize, but it is difficult to execute as well (Tengeri et al. 2015). The aim of
designing the code is not to make it complex, but to assist in smooth software development process
of which testing is an essential part. Hence this principles is an essential principles if success of
software testing is taken into consideration.
Principle #2: Readability and Comprehension
According to this principle the readability of code is important for effective software testing.
This is indeed very much important as a proper readable and comprehensive code is easy to analyse.
It is in fact one of the most important qualities that makes a code suitable for successful software
development (Fucci and Turhan 2014). Proper readability of the code is necessary as it is quite easy
to interpret which makes it easier for debugging, maintaining and extending the code.

3
CRITICAL ANALYSIS OF RESEARCH REPORT
During the testing of the software it often requires to identify the bug in the developed code
when the testing process does not executed successfully. However if the readability of the code is
not ensured during the development phase it makes the process complex which is not desired at all.
Principle #3: Single Responsibility:
It is often observed that a when a test tries to verify several condition in one test, it creates
complexity when the test is not successful. Hence the principle recommends to assert a single
condition for verification against each test case. However, according to the principle if multiple
assertion need to be defined, then strict focus on only one behaviour for each test case is required to
avoid complexity.
It is required because when a test case has more than one condition per test, it is not easy to
identify the root cause of the problem, if the test fails.
Principle #4: Avoid Over-protectiveness:
According to the principle, having redundant asserts in a test code should be avoided as it
lead to over-protectiveness, which often hamper the actual objective of the testing (Bowes et al.
2017).
Although in most of the cases this is very effective approach, it is not always a bad idea to
have multiple asserts in a single testing (Fucci et al. 2015). In some cases, multiple assertion is
required. The example of Guard Assertion might be referred here. However in most of the cases, it
is effective to avoid redundant asserts in a test code for avoiding complexity of the process.
Principle #5: Test Behaviour:
According to this particular principle, focusing on test behaviour is way more effective than
focusing on test implementation.

4
CRITICAL ANALYSIS OF RESEARCH REPORT
It is often seen that when an implementation details is preferred in a testing process and if
for some reason the test does not provide the expected result, then then it requires to update the
overall implementation focused test from the scratch (Tosun-Misirli and Oivo 2015).
Hence some effective strategies are to emphasis on test behaviour, access public interface
for the class that is designed for the testing process. If this approach is followed then the above
mentioned issues is possible to avoid.
Principle #6: Maintainability: Refactoring test code:
Software testing is done so that it is easier to maintain code, however if the code is not
properly designed, it becomes difficult to maintain the testing process itself which creates additional
difficulties for the system. The reason for the difficulties might be attributed to the fact that
production code evolves along with the test code. Hence if proper maintainability of the test code is
not ensured it is not possible to manage test code along with the production code (Inozemtseva and
Holmes 2014).
One thing that is always neglected in the testing process is that test code is also a code and
hence proper effort should be put in this section as well. Duplication should be avoided at it creates
issues in the code maintenance. Hence checking for the number of duplication in the test code and
also the number of refactoring applied in the code is a comprehensive way to check whether the
testing process complies with this principle or not.
Principle #7: A test should fail:
Whether the aim of any test is to ensure success, a test that never fails is not a god idea
either. This implies that there is possibility of not including any assertion in the test case (Bowes et
al. 2017).

5
CRITICAL ANALYSIS OF RESEARCH REPORT
It is the responsibility of the testing team or any professional to ensure that enough assertion
is provided in the testing process as the testing frameworks is not capable of recognizing this kind
of simple mistakes (Yusifoğlu Amannejad and Can 2015).
In order to verify that this principle has been properly followed, it is recommended to
identify whether a test fails or not. However it should be noticed that this does not implies that, one
should aim for the test to fail, it basically means that with assertion included in the test process,
chances are very less that the test would not fail even for a single test case.
Principle #8: Reliability:
According to this principle, a test should have the behaviour as defined or as expected.
There should be definite reason for which the test pass or fails, randomness in the test should be
avoided for effective software development.
Actually what this principle implies is that a randomness in the testing process means it is
hard to identify the issues and make correction accordingly (Bowes et al. 2017). Hence if there is
any source that contributes to the non-deterministic behaviour, it should be isolated and removed as
well.
Principle #9: Happy vs. Sad tests:
Happy test confirms behaviour and Sad test breaks system. Due to confirmation bias, happy
test is often preferred. However it should be noted that a good testing process should always
consider equal no of both the test process as it is important to analyse the test code in terms of
partitioning strategy and boundary case analysis (Bavota et al. 2015).
In order to compare between Happy vs. Sad tests as software testing process and analyse
whether the process complies with the principle or not, the author has mentioned that it is not easy
to automate measurement of this principle.
Principle #10: Tests should not dictate the code:

6
CRITICAL ANALYSIS OF RESEARCH REPORT
It is important to analyse the production code in terms of the size of the test code, number of
assertions per test case and conditional test logic. However it should be kept separated from the test
code, otherwise the test logic might interfere with the production code testing (Vahabzadeh Fard
and Mesbah 2015).
This measurement criteria will help the developer to make an effective estimation of the
applicability of the code before the testing process of the software is conducted. Hence this is an
important strategy.
Principle #11: Fast feedback:
Timely feedback is one of the essential requirement for test automation. It not only helps in
analysing the test result, find issues but also reduces delay in the testing process which happens due
to error in test code logic, flaws in code implementation and complexity in test process (Tengeri et
al. 2015).
The run-time for test cases is an excellent way to measure this principle as it helps to
identify if there is any problems encountered in the testing process and provides a way to
investigate the issue further for finding solution for executing the test process properly.
Principle #12: 4-phase test design:
The 4 phase test design suggest 4 phases such as t the setup, execution, validation and tear-
down stages for effective testing. Although the setup or the execution stages are often skipped by
novice developer, it becomes essential when execution is embedded in the fixture (Fucci et al.
2015). However it is possible to verify if this step has been followed or not by simply checking the
number of non-assert lines place before the first assert line is found in the test case.
Although the novice developers often skip this step, it is necessary for defining test
behaviour and test result for the test reader which is an important part of the testing process.
Principle #13: Simplicity of fixtures:

7
CRITICAL ANALYSIS OF RESEARCH REPORT
Some time for the setup phase, code is often needed to duplicate. A special test case method
known as fixtures is considered for refactoring the shared code for avoiding duplication. Hence it is
required to keep the fixtures simple. If the fixtures are not simple, it creates complexity as the test
case increases in a test class.
Principle #14: Test (in) dependency:
According to the principle an effective testing process should ensure test execution in
random order (Bowes et al. 2017). If there is requirement for further test after the original one or if
there is static field in class even after the testing framework creates new instance of the class, then
the principle is violated.
This is an effective process for testing because if a test for two methods of a class depends on each
other, failure in one test case will hamper the other test. However if the test are run in isolation, it is
possible to identify the bug in the testing more quickly with less complexity (Herzig 2014).
Principle #15: Use of test doubles:
Test doubles is an excellent strategy for effective test process. Sometimes in the testing
process, classes that relies on non-existed external resources, are required to be tested. In such
cases, it is important to have test doubles for mocking out external dependencies associated with the
testability of target class so that the testing process is executed and delay is avoided in the testing.
The best way to ensure that the testing process complies with the principle is to check
whether test doubles are defined for the process or not.
Critical summary of the research carried out and presented in the paper:
The research has explored a very interesting and important topic in the field of software
development. In the software development process, testing plays an essential role and it need to be
properly executed as well. However often the process is full of complexity and hence
comprehensive guide is required for the same.

8
CRITICAL ANALYSIS OF RESEARCH REPORT
The authors have described a core list of testing principles that deals with different quality
aspects and also provided way to quantify then as well. The quantification of the quality aspects
with reference to the testing principles makes it easier to work with the principle and apply them
properly within the testing process.
In this paper the author has taken a very detailed approach in describing the principles
behind the successful software testing. The authors have described a total number of 15 principles
that makes it clear what should be the goals and characteristics of effective software testing process
for ensuring success.
These 15 principles have been identified after conducting a two-day workshop with the
industry partners. At first a primary list was defined which has been further refined through the
academic and industrial training background of the authors. Along with that the recommendations
provided in the practitioner oriented testing books has also been referred to further refine the list
which makes the research more authentic and validates the findings.
The paper has not only discussed about the principles, it have also provided some rules for
identifying if a given software testing process complies with the principles or not. The rules that
have been recommend in the paper have the potential to serve as a comprehensive guide for
implementing best strategies in the software testing process.
In the paper the author has also presented relevant work that discusses about test smell.
Although the concept was presented, however there should be more details and more importance
should have been provide in describing this principles. In the paper this has not been properly
described. However a basic overview has been provided and comparison has also been made with
the standard test smells and the findings have been listed in a table for easy reference. The table also
describes metrics for quantification of the principle.
However it should also be noted that out of 15 principles defined in this paper, eight
principles do not map to existing test smells and it was not possible to quantify 2 of the additional

9
CRITICAL ANALYSIS OF RESEARCH REPORT
principles, the authors were only able to propose metrics for the six remaining principles. Hence
there are scope for improvement and this should be done through systematic literature review and
designing tools that provides more automated and improved methods for quality testing in the
software testing process.
References:
Bavota, G., Qusef, A., Oliveto, R., De Lucia, A. and Binkley, D., 2015. Are test smells really
harmful? An empirical study. Empirical Software Engineering, 20(4), pp.1052-1094.
Bowes, D., Hall, T., Petric, J., Shippey, T. and Turhan, B., 2017, May. How good are my tests?.
In Emerging Trends in Software Metrics (WETSoM), 2017 IEEE/ACM 8th Workshop on (pp. 9-14).
IEEE.
D. Tengeri, A. Besz ´ edes, T. Gergely, L. Vid ´ acs, D. Havas, and ´ T. Gyimothy, “Beyond code
coverage - An approach for test suite ´ assessment and improvement.” ICST Workshops, pp. 1–7,
2015.
Fucci, D. and Turhan, B., 2014. On the role of tests in test-driven development: a differentiated and
partial replication. Empirical Software Engineering, 19(2), pp.277-302.
Fucci, D., Turhan, B., Juristo, N., Dieste, O., Tosun-Misirli, A. and Oivo, M., 2015. Towards an
operationalization of test-driven development skills: An industrial empirical study. Information and
Software Technology, 68, pp.82-97.
Herzig, K., 2014, November. Using pre-release test failures to build early post-release defect
prediction models. In Software Reliability Engineering (ISSRE), 2014 IEEE 25th International
Symposium on (pp. 300-311). IEEE.

10
CRITICAL ANALYSIS OF RESEARCH REPORT
Inozemtseva, L. and Holmes, R., 2014, May. Coverage is not strongly correlated with test suite
effectiveness. In Proceedings of the 36th International Conference on Software Engineering (pp.
435-445). ACM.
Tengeri, D., Beszédes, Á., Gergely, T., Vidács, L., Havas, D. and Gyimóthy, T., 2015, April.
Beyond code coverage—An approach for test suite assessment and improvement. In 2015 IEEE
Eighth International Conference on Software Testing, Verification and Validation Workshops
(ICSTW) (pp. 1-7). IEEE.
Tosun-Misirli, A. and Oivo, M., 2015. Towards an operationalization of test-driven development
skills: An industrial empirical study. Information and Software Technology, 68, pp.82-97.
Vahabzadeh, A., Fard, A.M. and Mesbah, A., 2015, September. An empirical study of bugs in test
code. In Software Maintenance and Evolution (ICSME), 2015 IEEE International Conference
on (pp. 101-110). IEEE.
Yusifoğlu, V.G., Amannejad, Y. and Can, A.B., 2015. Software test-code engineering: A systematic
mapping. Information and Software Technology, 58, pp.123-147.