SIT707: Software Testing Report on Black Box Testing Methods

This report, prepared for SIT707 Software Quality and Testing, presents an in-depth analysis of software testing methodologies, with a particular emphasis on black box testing. The report begins with an introduction that highlights the importance of software testing in ensuring desired functionalities and identifying potential malfunctions. A comprehensive literature review follows, exploring various testing processes, including unit testing, acceptance testing, and performance monitoring. The review examines the significance of web applications in the current digital landscape and the role of test automation, particularly Selenium, in improving efficiency and accuracy. Furthermore, the report discusses the use of Selenium for functional testing, database verification, and the integration of Selenium into agile testing frameworks. The report also covers fault injection techniques and the factors influencing bug fixing decisions. The conclusion emphasizes the critical role of software testing in ensuring software functionality and preventing costly failures. The report is supported by a detailed bibliography of referenced sources.

SIT707 Software Quality and Testing
Name of the Student
Name of the University
Authors note

Introduction
In the aspects of software engineering the testing is one of the most important
activities that makes sure that all the desired functionalities are met by the developed
software. Testing of the developed system helps in validating the behavior as intended by the
users as well as identify different potential malfunctions in the test cases [1]. As the
pervasiveness, complexity as well as the criticality of the developed systems are increasing
day by day therefore ensuring the desired levels of quality as well as dependability of the
developed web based systems becoming crucial and expensive.
Literature Review
There are different types of processes for testing which are available that includes
testing a smaller segment of code by the development engineer (often referred as unit
testing), Real life user based functionality testing through the customer validation of an
information system (known as the acceptance testing) and monitoring the performance of the
application at run-time for a network-centric SOA (service-oriented application) applications.
According to Altaf et al., (2015), the current public generation is very much
dependent over the web as it is very much essential in the present situation. In the current
time most of the product applications that are available and executing are mainly online
applications which keep running through a web based program. In this aspect testing of these
programs is very much important as it provides information regarding critical aspects of the
application. For a specific type of web application there can be numerous of association
which utilizes this web application. So all of these web applications needs to be habituated
with the different types of clients from distinctive types of regions physically [22]. Tedious
situation can occur when a web application tested physically. Due to this reason test
automation is utilized in this case. Here the authors implemented the test automation, and for

that they have utilized a product device so that they can execute repeatable tests against the
application which needs to be tested in this case. While the test automation is executed there
are several of focal points which needs to be considered. The test automation is more efficient
while compared with the manual automation as the test automation is exceptionally exact
with the manual automation while consisting more prominent preparing pace [7]. Currently,
for this test mechanism there are various of business and open source devices are accessible.
In this aspect the selenium main open source device that is utilized for test computerization.
The software automation testing actually improves the effectiveness of the test procedures of
the program. As per the authors test automation is able to provide a quick criticism to the
engineers. The test automation is also important in this aspect as in many situation it
discovers the imperfection within the software which can be missed in the manual testing
procedures. Also, in the test automation boundless emphases for the testing can be performed
for testing of same code ceaselessly commonly.
As per the Holmes and Kellogg, (2006), in maximum of the scenarios Selenium is
selected for functional testing of the web related applications. The authors has defined the
Selenium and as per them it is actually an open source type of project that is used for testing
something within the browser [9]. This is established by the ThroughtWorks and currently is
very much popular in the community of the users and developers. One of the important goal
Selenium is becoming the de facto open source replacement of the WinRunner which is
proprietary tool in nature [8]. Both of the authors has described the standard environment that
is important of the testing using the Selenium. In this aspect the authors has also described
some important lessons regarding script writing, continuous integration and the utilization of
the Selenium recorder. Regarding this the authors has also demonstrated how long it can take
for writing and maintaining of the scripts within an development environment which is
iterative in nature [11]. The authors has described how the Selenium testing is able to cover

all the functional requirements with the tests and how often this tests needs to be executed.
As per the authors the Selenium is one of the most important addition that is integrated to the
toolkit of agile testing and it is utilized for the maximum of the web application related
projects.
According to the authors de Castro et al., (2013), Selenium RC tool is one of the main
important web application testing tool which requires checking the available in within the
databases. To perform this verification procedures new types of functions has been integrated
within the core of Selenium’s framework. Through this function closing and opening of the
database can be done and through this testing data can be compared with the data that has
been stored within database which is utilized by the application of web itself [14]. This
solution is actually consisting an important aspect which will contribute to the quality
improvements of the system so that effort that is needed for test procedures can be reduced.
Reducing the effort is quite important in this case as verification of the database elements and
UI of the system will be done at the similar time while the scripts of test will be executed
[15]. The authors has successfully described the impact of the tools that has been proposed in
this case in terms of automation and the effort rate while the progress of the new web
application project is executed. Regarding this the authors has executed a test from which
they have found a significant reduction in the test effort while automated tests are executed
on the databases in comparison with semi-automated and manual execution.
The authors Banabic and Candea, (2012), has demonstrated that black box testing
procedure of the software is crucial so that proper functionality of the software programs can
be ensured. The authors has described the process of fault injection which is an important
method of robustness testing of the software related systems but it is used rare of the cases.
The main reason behind this is that it is very much labor intensive [20]. Due to this reason
system testers which needs high reliability requirements that includes MySQL, for

performing the process of fault injection through an ad-hoc manner. In this aspect the users
introduced the AFEX which is a specific type of tool and technique that is used for
automating the whole process of fault injection. Here the AFEX defines the entire process
from choosing faults that need to be injected to setting up the environment of testing. This
also includes injection process performing and characterization of the results of the tests [4].
In this AFEX approach metric driven search algorithm is utilized which aiming to increase
the amount of discovered bugs within a definite time amount. To perform the functionality of
the AFEX approach it is tested by the authors in on the real world application on which it
performed quite well when it is compared with other types of black box approaches.
In this aspect, authors Murphy-Hill et al., (2014), has elaborated that for an engineer
there are several types of options for fixing a bugs that is identified during the testing phase.
In this aspect all of the fixes might can cause introduction of the other bugs [17]. So, in such
type of situations the engineers needs to choose the fixing of the bugs wisely. In this type of
situation it needs to be assessed whether used code for fixing the bug actually caused the bug
or not. It is also need to be assessed whether the solution is fixing the whole problem or just
covering the bug symptom [19]. Regarding this the authors has investigated alternative fixes
for the bugs and has shown an empirical study related with how the engineers makes the
decision regarding fixing of the bugs. From their study they have found that there are various
of non-technical factors which can influence that how a bug will be fixed.
According to the authors Xu et al., (2012), one of the common way of performing the
system testing, of the web based application is performing it through the web browser. In
most of the cases for managing and recording these tests replay tools are used and in this case
Selenium IDE is one of the common tool [4]. Specification of the mining form such kind of
tests can be beneficial for the understanding purposes and for the debugging and verification
of the system during a test. In this aspect the authors has presented a specific type of

approach regarding mining a behavior description from a test suite of Selenium IDE such that
it can capture the test performance at the high abstraction level, simulation can be done of the
behavior and all of the tests can be done again from the specification. Following that, the
authors mined the test action patterns which represented expressive functions and converted
the Selenium IDE into test of abstraction. These are comparable with the tests that has been
also used for the techniques of model mining [3]. From these abstract tests authors has
synthesized a high-level of Petri net which is capable of capturing both of the data values and
the temporal constraints. Here evaluation has been also done by the authors and for the
evaluation purpose authors has used eight types of test suites regarding two types of real
world system and Amazon was one of them. These test suits has been done for the testing of
the security which aimed at XSS vulnerabilities and SQL injection. From the results the
authors has determined that the approach of them quite effective for production of the
abstracts for minimization of the complexity of the models.
Evaluation and Justification
I will do this part
Conclusion
From the above discussion it can be concluded that software testing is very much
important so that it can be ensured that all the functionality of the software is working
properly. It is also important in the aspect of ensuring that the testing application or the
software should not be failed in any of the circumstances as it can be very much expensive in
the future development process. Regarding this here software testing procedures has been
discussed. In the introduction section of this report importance of the software testing has
been defined. In the following section a vast literature review has been done which includes
surveys of the latest academic papers. Through this part of this report through and deep

understanding of the software testing has been gained. Following that evaluation and
justification has been provided regarding testing methods and the tools.
Bibliography
[1]Altaf, J.A. Dar, F.u. Rashid and M. Ra q, “Survey on selenium tool in software testing,”
Green Computing and Internet of Things (ICGCIoT), 2015 International Conference on,
Noida, 2015, pp.1378-1383.
[2]Holmes and M. Kellogg, “Automating functional tests using Selenium,” AGILE 2006
(AGILE' 06), Minneapolis, MN, 2006.
[3] A.M.F.V. de Castro, G.A. Macedo, E.F. Collins and A.C. Dias-Neto, “Extension of
Selenium RC tool to perform automated testing with databases in web applications,”
Automation of Software Test (AST), 2013 8th International Workshop on, San Francisco,
CA, 2013, pp. 125-131.
[4]Radu Banabic and George Candea. 2012. “Fast black-box testing of system recovery
code.” In Proceedings of the 7th ACM European conference on Computer Systems (EuroSys
'12). ACM, New York, NY, USA, pp. 281-294.
[5] E. Murphy-Hill, T. Zimmermann, C. Bird and N. Nagappan, “The Design Space of Bug
Fixes and How Developers Navigate It,” in IEEE Transactions on Software Engineering, vol.
41, no. 1, 65-81, Jan. 1 2015.
[6] D. Xu, W. Xu, B.K. Bavikati and W.E. Wong, \Mining Executable Specifications of Web
Applications from Selenium IDE Tests,", 2012 IEEE Sixth International Conference on
Software Security and Reliability (SERE), Gaithersburg, MD, 2012, pp. 263-272.

[7] J. Bau, E. Bursztein, D. Gupta and J. Mitchell, “State of the Art: Automated Black-Box
Web Application Vulnerability Testing,” 2010 IEEE Symposium on Security and Privacy,
Oakland, CA, USA, 2010, pp. 332-345.
[8] Le and J.T. Ray eld, “Web-application development using the Model/View/Controller
design pattern,” Proceedings. of the Fifth IEEE International Enterprise Distributed Object
Computing Conference (EDOC'01), Seattle, WA, 2001, pp. 118-127.
[9] Cox, M.G. and Harris, P.M., 1999. “Design and use of reference data sets for testing
scientific software.” Analytica Chimica Acta, 380(2), pp.339-351.
[10] Kelly, D., Thorsteinson, S. and Hook, D., 2011. “Scientific software testing: analysis
with four dimensions.” IEEE software, 28(3), pp.84-90.
[11] Kanewala, U. and Bieman, J.M., 2014. “Testing scientific software: A systematic
literature review." Information and software technology 56(10), pp. 1219-1232.
[12] Hinsen, K., 2015. “The approximation tower in computational science: Why testing
scientific software is difficult.” Computing in Science & Engineering, 17(4), pp. 72-77.
[13] Bertolino, A., 2007, May. “Software testing research: Achievements, challenges,
dreams.” In 2007 Future of Software Engineering (pp. 85-103). IEEE Computer Society.
[14] Hook, D. and Kelly, D., 2009, May. “Testing for trustworthiness in scientific software.”
In Proceedings of the 2009 ICSE Workshop on Software Engineering for Computational
Science and Engineering (pp. 59-64). IEEE Computer Society.
[15] Joppa, L.N., McInerny, G., Harper, R., Salido, L., Takeda, K., O'Hara, K., Gavaghan, D.
and Emmott, S., 2013. “Troubling trends in scientific software use.” Science, 340(6134), pp.
814-815.

[16] Lewis, William E. Software testing and continuous quality improvement. Auerbach
publications, 2017.
[17] Fenton, N. E., and Martin Neil. "Improving Software Testing with Causal
Modeling." Analytic Methods in Systems and Software Testing (2018): 27.
[18] Kanewala, Upulee, James M. Bieman, and Asa Ben‐Hur. "Predicting metamorphic
relations for testing scientific software: a machine learning approach using graph
kernels." Software testing, verification and reliability 26, no. 3 (2016): 245-269.
[19] Takanen, Ari, Jared D. Demott, Charles Miller, and Atte Kettunen. Fuzzing for software
security testing and quality assurance. Artech House, 2018.
[20] Aleti, Aldeida, Irene Moser, and Lars Grunske. "Analysing the fitness landscape of
search-based software testing problems." Automated Software Engineering 24, no. 3 (2017):
603-621.
[21] Garousi, Vahid, and Dietmar Pfahl. "When to automate software testing? A decision‐
support approach based on process simulation." Journal of Software: Evolution and
Process 28, no. 4 (2016): 272-285.
[22] Mili, Ali, and Fairouz Tchier. Software testing: Concepts and operations. John Wiley &
Sons, 2015.