Literature Review on Testing Methods

This literature review provides an overview of various testing methods including cloud-based testing, mobile testing, testing of context aware systems, agile testing, testing automation, quality assurance and testing on open source projects, and AI in software testing.

qwertyuiopasdfghjklzxcvbnmqw
ertyuiopasdfghjklzxcvbnmqwert
yuiopasdfghjklzxcvbnmqwertyui
opasdfghjklzxcvbnmqwertyuiop
asdfghjklzxcvbnmqwertyuiopasd
fghjklzxcvbnmqwertyuiopasdfgh
jklzxcvbnmqwertyuiopasdfghjkl
zxcvbnmqwertyuiopasdfghjklzxc
vbnmqwertyuiopasdfghjklzxcvb
nmqwertyuiopasdfghjklzxcvbnm
qwertyuiopasdfghjklzxcvbnmqw
ertyuiopasdfghjklzxcvbnmqwert
yuiopasdfghjklzxcvbnmqwertyui
opasdfghjklzxcvbnmqwertyuiop
asdfghjklzxcvbnmqwertyuiopasd
fghjklzxcvbnmqwertyuiopasdfgh
jklzxcvbnmrtyuiopasdfghjklzxcv
Literature Review
Testing Methods
1/2/2019

Literature Review
Table of Contents
Introduction...........................................................................................................................................2
Cloud-based Testing..............................................................................................................................2
Mobile Testing......................................................................................................................................2
Testing of Context Aware Systems/Embedded Systems/Cyber Physical Systems................................3
Testing on Smart devices/Wearable devices..........................................................................................3
Agile Testing.........................................................................................................................................4
Testing Automation/Tools.....................................................................................................................4
Quality Assurance and Testing on an Open Source Project...................................................................5
AI & Software Testing..........................................................................................................................5
Conclusion.............................................................................................................................................5
References.............................................................................................................................................7
1

Literature Review
Introduction
Testing is one of the essential activities that shall be carried out in a project. There are
different forms of testing types and methods that are followed to accomplish the testing
process. With the development of technology, there are newer testing techniques, tools, and
methods that have been introduced. A literature review of some of these advanced
mechanisms is included in the report.
Cloud-based Testing
Cloud-based testing provides the capability to carry out the testing activities in multiple
environments without the requirement of setting up an infrastructure for the same. There are
benefits provided with the inclusion of cloud-based testing processes, such as low costs,
enhanced scalability, and ease of availability. Cloud-based testing refers to the evaluation
methodology in which cloud computing environment, such as Software as a Service (SaaS)
model is used to test the applications. Elasticity and scalability testing, security testing,
performance testing, live upgrade testing, stress testing, latency testing, compatibility testing,
functional testing, and browser performance can be carried out on the cloud. Some of the
popular cloud-based testing tools include SOASTA, ITKO LISA, Load Runner, Blitz, and
Blaze Meter. There are a few challenges that are associated with the cloud-based testing
methods. There are security concerns associated with cloud in terms of information security
and privacy attacks. Integration and interoperability of the applications over the cloud is also
one of the major concerns that may impact the test execution and results (Nachiyappan &
Justus, 2015).
Mobile Testing
Mobile testing is a process in which the applications and programs that are developed for
mobile devices are verified and validated. There are different types of testing activities
carried out under the domain of mobile testing. Mobile functions and behaviour testing is
conducted to validate the functionalities and the behaviour of the device and the applications
under different scenarios. Mobile QoS testing is done to test the system qualities, such as
reliability, scalability, and availability (Zein, Salleh & Grundy, 2016). Cross-use functionality
is of utmost relevance in the current times and it is tested through interoperability
(compatibility) testing. There are mobile users that may belong to different cultural
2

Literature Review
backgrounds and have varied linguistic requirements. Internationalization mobile testing is
done to verify the usability of the device and mobile applications for every user. Security
testing is one of the significant testing types that are carried out to test the security features.
There are different strategies that are used to conduct mobile testing. The use of device
emulators is done to test the mobile applications. Cloud-based testing is also amalgamated
with mobile testing for maximum code coverage (Akour, Falah, A., Bouriat & Alemerien,
2016).
Testing of Context Aware Systems/Embedded Systems/Cyber Physical Systems
Testing of embedded systems, cyber physical systems, and context aware systems is complex
because there is a no defined and step by step methodology for evaluation. Conformance
testing is one of the testing types that may be carried out to test such systems (Araujo,
Carvalho, Mohaqeqi, Mousavi & Sampaio, 2018). The testing method refers to the use of
methods to validate the compliance of the system towards specific set of guidelines and
standards. The testing type is often combined with load testing, volume testing, and dress
testing to determine the overall performance and behaviour of the systems.
The testing of these systems is usually carried out using automation testing tools. The test
cases are designed to target the context variables or values to achieve complete coverage
(Alongi, Vitello, Vitabile & Conti, 2018).
Testing on Smart devices/Wearable devices
There are various components that are involved in smart and wearable devices. Some of these
components include hardware, firmware, integration software, etc. The testing procedures on
these devices are therefore a mix of different types. Material testing is conducted during the
manufacturing phase and involves chemical testing, mechanical testing, and performance
testing. The hardware involved in these devices is tested as per the methods as battery
lifecycle testing, wireless testing, and electrical safety testing. The operating software and
integration data layer is tested using the firmware testing. It involves usability testing,
compatibility testing, database integrity testing, and security testing. It is because all of these
system qualities are essential to be incorporated in the device. Some of the usability aspects
that are validated include gesture, voice input, permissions, security, connectivity, etc. Also,
3

Literature Review
the compatibility and integration among the components and modules shall be present for
adequate functioning of the smart/wearable device (Page, 2015).
The smart devices or wearable devices are coupled with a mobile or web platform as one of
the user interfaces. The testing of the devices also includes the testing of the external
software.
Agile Testing
Agile testing is a mechanism that has revolutionized and modified the traditional testing
methods and activities. The agile testing states that testing is not a phase and it shall be
integrated in the lifecycle of the project. Also, as per the agile methodology, it is not just the
designated testers that shall carry out the testing activities. Rather, all the resources in the
project team must participate in the testing processes. The agile teams test early in the
lifecycle and also conduct regular testing to prevent the bugs and resolve them in a timely
manner. There are various advantages provided with the use of agile testing in a project. Test-
driven development is initiated along with enhanced quality control with the aid of the
feedback provided by the customer and the end-users (Deshpande, Deshpande, Marathe &
Garje, 2010). The changes in the project are also easily carried out with minimal rework and
negligible impact on project schedule and budget.
Some of the testing processes that are conducted as a part of agile testing methodology
include user acceptance testing, pair programming, test driven development, and refactoring
(Bavani, 2012).
Testing Automation/Tools
Automation is one of the key activities that are being witnessed in all of the business
processes and operations. Automation of the testing tasks and activities is also increasing and
there are various automation tools that have been developed.
Test automation is not only restricted to test execution. However, there are all the testing
processes and activities that can be carried out using testing automation. Test case design, test
scripting, test execution, defect tracking, and test evaluation are some of the testing
procedures that are now being automated (Garousi & Elberzhager, 2017).
4

Literature Review
There is a wide range of test automation tools that are now available in the market. Some of
these include Selenium, Appium, Test Studio, Cucumber, Rational Functional Tester, HP
WinRunner, and many more.
Quality Assurance and Testing on an Open Source Project
Quality assurance and testing activities on open source software and projects differ from the
closed source software and projects. Open source software does not have a defined
documented methodology and have unstructured risk assessment processes.
The three primary areas that are involved in quality assurance and testing on an open source
project include defect detection, defect verification, and solution verification. The defects that
are detected are either reported to the fellow developers or are ignored. The verification of the
defect and the solution is conducted thereafter. There is a short-term plan developed for
fixing the issue that may have occurred (Bahamdain, 2015). The use of bug tracker is done to
report the issue again. Self-code review is done to verify the solution.
AI & Software Testing
Artificial Intelligence (AI) is now being applied in the field of software testing and there is a
lot of research that is going on to explore the ways in which AI can be further used to
enhance the testing processes. There are different AI algorithms and codes being used in
software testing field, such as Ant Colony Optimization (ACO), Tabu Search, Bee Colony,
Genetic Algorithm, and Data Mining (Rafael Lenz, Pozo & Regina Vergilio, 2013).
The use of AI in software testing has provided the capabilities to test the smart devices and
applications. The overall accuracy of the testing processes has also improved with the
involvement of AI in testing. AI along with automation testing has led to the increase in the
speed of testing and has also enhanced the overall efficiency.
Conclusion
Cloud-based testing refers to the evaluation methodology in which cloud computing
environment, such as Software as a Service (SaaS) model is used to test the applications.
Mobile functions and behaviour testing is conducted to validate the functionalities and the
behaviour of the device and the applications under different scenarios. Conformance testing
is one of the testing types that may be carried out to test cyber physical systems or embedded
5

Literature Review
systems. The smart devices or wearable devices are coupled with a mobile or web platform as
one of the user interfaces. The testing of the devices also includes the testing of the external
software. There are various advantages provided with the use of agile testing in a project.
Test-driven development is initiated along with enhanced quality control with the aid of the
feedback provided by the customer and the end-users. Test case design, test scripting, test
execution, defect tracking, and test evaluation are some of the testing procedures that are now
being automated. The three primary areas that are involved in quality assurance and testing
on an open source project include defect detection, defect verification, and solution
verification. The overall accuracy of the testing processes has improved with the involvement
of AI in testing.
6

Literature Review
References
Akour, M., Falah, B., A., A., Bouriat, S., & Alemerien, K. (2016). Mobile Software Testing:
Thoughts, Strategies, Challenges, and Experimental Study. International Journal Of
Advanced Computer Science And Applications, 7(6). doi:
10.14569/ijacsa.2016.070602
Alongi, A., Vitello, G., Vitabile, S., & Conti, V. (2018). An Empirical Set of Metrics for
Embedded Systems Testing. IEEE Design & Test, 35(5), 45-53. doi:
10.1109/mdat.2017.2787678
Araujo, H., Carvalho, G., Mohaqeqi, M., Mousavi, M., & Sampaio, A. (2018). Sound
conformance testing for cyber-physical systems: Theory and implementation. Science
Of Computer Programming, 162, 35-54. doi: 10.1016/j.scico.2017.07.002
Bahamdain, S. (2015). Open Source Software (OSS) Quality Assurance: A Survey Paper.
Procedia Computer Science, 56, 459-464. doi: 10.1016/j.procs.2015.07.236
Bavani, R. (2012). Distributed Agile, Agile Testing, and Technical Debt. IEEE Software,
29(6), 28-33. doi: 10.1109/ms.2012.155
Deshpande, S., Deshpande, A., Marathe, M., & Garje, G. (2010). Improving Software Quality
with Agile Testing. International Journal Of Computer Applications, 1(22), 68-73.
doi: 10.5120/440-673
Garousi, V., & Elberzhager, F. (2017). Test Automation: Not Just for Test Execution. IEEE
Software, 34(2), 90-96. doi: 10.1109/ms.2017.34
Nachiyappan, S., & Justus, S. (2015). Cloud Testing Tools and its Challenges: A
Comparative Study. Procedia Computer Science, 50, 482-489. doi:
10.1016/j.procs.2015.04.018
Page, T. (2015). A Forecast of the Adoption of Wearable Technology. International Journal
Of Technology Diffusion, 6(2), 12-29. doi: 10.4018/ijtd.2015040102
Rafael Lenz, A., Pozo, A., & Regina Vergilio, S. (2013). Linking software testing results
with a machine learning approach. Engineering Applications Of Artificial
Intelligence, 26(5-6), 1631-1640. doi: 10.1016/j.engappai.2013.01.008
7

Literature Review
Zein, S., Salleh, N., & Grundy, J. (2016). A systematic mapping study of mobile application
testing techniques. Journal Of Systems And Software, 117, 334-356. doi:
10.1016/j.jss.2016.03.065
8