Literature Review: AI in Testing and Augmented Reality Applications

This report provides a comprehensive literature review on the integration of Artificial Intelligence (AI) in testing methodologies and Augmented Reality (AR) applications. It delves into the definitions and applications of AR, including the use of digital data like 3D assets and videos. The report examines how AI is employed in automation testing, identifying controls, generating technical maps, and verifying outcomes, referencing studies by McKinnel (2019), Li and Du (2017), and Fehlmann (2019). Furthermore, it explores the use of AR in various fields, including HR and medicine, highlighting the benefits of AR in training and data access, as discussed by Yannakakis and Togelius (2018) and Barsom, Graafland and Schijven (2016). The report also includes a case study on Microsoft HoloLens application testing, examining environmental scanning, object placement, and accessibility testing. The report concludes by discussing the challenges and opportunities of AR and AI integration in testing, emphasizing the potential for improved efficiency and user experience. This report provides a detailed overview of the current state of AI and AR in testing and their applications in various industries.

Running head: AI IN TESTING AND AUGMENTED REALITY
AI in testing and Augmented Reality
Name of the student:
Name of the university:
Author Note

1AI IN TESTING AND AUGMENTED REALITY
Literature review on AI in testing and Augmented Reality:
Billinghurst, Clark and Lee (2015) provides the definition of what augmented reality is.
According to them, it is that reality which is augmented with the digital data (Billinghurst, Clark and
Lee 2015). They further explains that the digital data can be in the format of 3D assets, videos,
pictures and texts and assimilation of everything (Billinghurst, Clark and Lee 2015). The system
requires to evaluate reality and then reconstruct to create the digital twin (Billinghurst, Clark and Lee
2015). This also requires to enable users to make interaction with digital data and digital twin
(Billinghurst, Clark and Lee 2015). Furthermore, Yannakakis and Togelius (2018) demonstrates how
it can be used in HR and the reasons behind that (Yannakakis and Togelius 2018). The people,
processes and things can be identified with AR system (Yannakakis and Togelius 2018). It can
permit the staffs to use the system for gaining data related to that workplace (Yannakakis and
Togelius 2018). It can be utilized as the other form related to informal training(Yannakakis and
Togelius 2018). It is method for the staffs to make the knowledge get transferred through including
data to informational database and AR system (Yannakakis and Togelius 2018). It is related directly
to the influences of HR professionals often seeking to develop skills of employees, make the
knowledge transfer facilitated and rise the productivity(Yannakakis and Togelius 2018). Further, the
training to dangerous cases could be facilitated with augmented reality despite putting human beings
in the path of harm (Yannakakis and Togelius 2018). Moreover, training could be given to
complicated activities as the staffs are able to utilize AR software as the guidance to the spot
(Yannakakis and Togelius 2018). Again, the staffs are able to utilize AR for accessing manuals,
training materials and HR politics and other documentation and information (Yannakakis and
Togelius 2018). It can develop the productivity and efficiency and also engagement of employees

2AI IN TESTING AND AUGMENTED REALITY
(Yannakakis and Togelius 2018). Further, employees can feel empowered through understanding
that they have the tools at the fingertips (Yannakakis and Togelius 2018).
McKinnel (2019) mentions that AI-efficient constant testing platform is able to identify the
various controls that face changes. This is done more efficiently than the human beings (McKinnel
2019). Here, constant updates are there to the algorithms even with minute modifications that are
witnessed (McKinnel 2019). As the automation testing is concerned, AI is useful broadly within the
categorization of object application for every kind of user interfaces (McKinnel 2019). In this place
the recognized measures are put into category as the testers and tools are created can pre-train
various controls see outside the setups (McKinnel 2019). As the control hierarchy is seen, the testers
are able to generate the technical map (McKinnel 2019). Thus the AI can look at the GUI to retrieve
the various control labels (McKinnel 2019). Having testing, the verification of outcomes, one
requires to access to the extra test data (McKinnel 2019). For instance, Li and (2017) states the
following,
“Google DeepMind has generated the program of AI using in–depth learning of
reinforcement for playing the video games from own. In this was lots of test data are produced.” (Li
and Du 2017)
Fehlmann (2019) explains that the AI, down the line can observe the users to perform the
exploratory type of testing under the testing site. This is done through utilizing the human brain for
determining and analyzing the applications tested (Fehlmann 2019) (Fehlmann 2019). This in turn,
would bring the users into testing (Fehlmann 2019). Thus the customers can automate the test cases
totally (Fehlmann 2019). As the behavior of user is analyzed, the preference risk is assigned,
monitored and then categorized as per the needs (Fehlmann 2019). Here, the information can be
regarded as the classic case for the automated testing for examining and eradicate various kinds of

3AI IN TESTING AND AUGMENTED REALITY
anomalies (Fehlmann 2019). To determine the bottlenecks, heat maps cam be used in the process
(Fehlmann 2019). This also helps to find out which tests are required to be conducted (Fehlmann
2019). Furthermore, through automating the manual tests and redundant test cases, in turn, the test
can concentrate to make the decisions and connections that are driven by data. Guerrero-Romero,
Lucas and Perez-Liebana (2018) refers,
“… risk based automated helps the users to find out what tests are required to be run for
getting the higher coverage, since limited period of the testing is a complex factor. Having the
assimilation of AI for test creation, the data analysis and execution, the testers are able to do away
permanently with the necessity of updating the test cases through manual way and continually”
(Guerrero-Romero, Lucas and Perez-Liebana 2018).
Thus the controls can be identified, the links between the defects could be spotted along with
the components in much more efficient way (Guerrero-Romero, Lucas and Perez-Liebana 2018).
Understanding Augmented Reality Test:
As per Yannakakis and Togelius (2018), the augmented reality testing can be understood
through the case study of the Microsoft HoloLens Application Testing. The most popular headset in
the current market as per various argument is Microsoft HoloLens (Yannakakis and Togelius 2018).
This is marketed as per the mixed device of reality interacting with physical objects under the actual
scenario (Yannakakis and Togelius 2018). The first step in the testing involves the scanning of
environment (Yannakakis and Togelius 2018). Here, the system needs to find out the meshes that
involves the surfaces and physical objects within various spaces, distances, angles, lighting
conditions and motions and various ambient noise levels (Yannakakis and Togelius 2018). The next
step is placing objects (Yannakakis and Togelius 2018). The scanning is followed through assuring
that the holograms or placed digital objects can move smoothly and quickly (Yannakakis and

4AI IN TESTING AND AUGMENTED REALITY
Togelius 2018). Further, there is the controlling of object (Yannakakis and Togelius 2018). Whether
the use can access the model menu at any point of time is critical to assure at any situations
irrespective of location and size of the model (Yannakakis and Togelius 2018). Due to this QA has
scaled model is to minimize and maximize to fetch whether menu buttons can be easily reached
(Yannakakis and Togelius 2018). Then, there is managing various objects at the same time
(Yannakakis and Togelius 2018). At many times, the user launches various same type o holograms,
and after that launches the animation or rotation effect for a single hologram (Yannakakis and
Togelius 2018). Here, other holograms also begins to rotate also (Yannakakis and Togelius 2018).
The testing resolves the concern (Yannakakis and Togelius 2018). Next the crashes must be made
visible (Yannakakis and Togelius 2018). No visible message is there regarding the crash for
HoloLens (Yannakakis and Togelius 2018). Challenges must be evaluated prior the development
begins (Yannakakis and Togelius 2018). For the current testing the crashes are made visible to users
such that they can relaunch the applications (Yannakakis and Togelius 2018). Then, there is
performing of accessibility testing (Yannakakis and Togelius 2018). The immersion to augmented
reality application sounds as the health effects and headaches motion sickness and eye strain are
concerned (Yannakakis and Togelius 2018).
According to the authors Billinghurst, Clark and Lee (2015), the augmented reality is one of
the emerging technology in the current context. In this report the authors have summarized 50 years
of research and development in the field of Augmented Reality (Billinghurst, Clark and Lee 2015).
Form the research the authors have assessed that AI or the artificial intelligence is the integral part
for this augmented reality technology (Billinghurst, Clark and Lee 2015). Here, the main aim of
implementing this type of technology is seamlessly combining both of the virtual world and the real
world together as per (Billinghurst, Clark and Lee 2015).

5AI IN TESTING AND AUGMENTED REALITY
The authors Chen et al. (2017), have observed that in recent with the development of
artificial intelligence interests has been also increased in the Augmented Reality technology or the
AR technology. In the current aspect there are various of applications of the AR technology which
includes the educational sector also (Chen et al. 2017). The authors have conducted a research on
augmented reality within the educational sector and it has been identified that there are various of
features, advantages, and effectiveness associated with the AR within the educational sector (Chen et
al. 2017). Here, the authors have performed a research on total 55 number of studies in Social
Sciences Citation Index database (Chen et al. 2017). From this research the author has founded
various of trends in the current segment of AR within the educational field (Chen et al. 2017). The
authors have also discussed the future vision and the important opportunities for the further
researches on the AR technology within the educational field (Chen et al. 2017).
The authors Barsom, Graafland and Schijven (2016) have elaborated the utilization of the AR
within the medical field. The authors have identified that the AI plays an important role in this case
for proper functionality of the AR applications in the medical training (Barsom, Graafland and
Schijven 2016). The AR can provide an effective type of medical education as it can join both of the
real and the virtual world for a perfect demonstration of the medical aspects (Barsom, Graafland and
Schijven 2016). From the results of the research it has been identified that AR is quite useful in the
sector of medical training (Barsom, Graafland and Schijven 2016).

6AI IN TESTING AND AUGMENTED REALITY
References:
Barsom, E.Z., Graafland, M. and Schijven, M.P., 2016. Systematic review on the effectiveness of
augmented reality applications in medical training. Surgical endoscopy, 30(10), pp.4174-4183.
Billinghurst, M., Clark, A. and Lee, G., 2015. A survey of augmented reality. Foundations and
Trends® in Human–Computer Interaction, 8(2-3), pp.73-272.
Chen, P., Liu, X., Cheng, W. and Huang, R., 2017. A review of using Augmented Reality in
Education from 2011 to 2016. In Innovations in smart learning (pp. 13-18). Springer, Singapore.
Fehlmann, T., 2019, September. Testing Artificial Intelligence. In European Conference on
Software Process Improvement (pp. 709-721). Springer, Cham.
Guerrero-Romero, C., Lucas, S.M. and Perez-Liebana, D., 2018, August. Using a team of general ai
algorithms to assist game design and testing. In 2018 IEEE Conference on Computational
Intelligence and Games (CIG) (pp. 1-8). IEEE.
Li, D. and Du, Y., 2017. Artificial intelligence with uncertainty. CRC press.
McKinnel, D.R., Dargahi, T., Dehghantanha, A. and Choo, K.K.R., 2019. A systematic literature
review and meta-analysis on artificial intelligence in penetration testing and vulnerability
assessment. Computers & Electrical Engineering, 75, pp.175-188.
Yannakakis, G.N. and Togelius, J., 2018. Artificial intelligence and games (Vol. 2). New York:
Springer.