Mobile Virtual Assistants: Technology, Security, and Future Directions

This report provides a comprehensive overview of Mobile Virtual Assistants (MVAs), examining their emergence, technology, and functionality. It delves into the core components, including voice recognition, text-to-speech conversion, and natural language processing, and discusses their application in smart devices. The report explores the benefits of MVAs, such as simplifying tasks and providing a user-friendly interface, while also addressing limitations and security/privacy concerns. It covers the evolution of MVAs, the underlying AI and algorithms, and their impact on user experience. The report also includes a literature review and concludes with a discussion of future trends and potential advancements in MVA technology.

Running head: MOBILE VIRTUAL ASSISTANTS
Mobile Virtual Assistants
Nam of the Student
Name of the University
Authors note

1MOBILE VIRTUAL ASSISTANTS
Abstract
Virtual mobile Assistants are considered as the next generation User interface (UI) for mobile
and smart device users. Virtual mobile assistants are administrations or services provided by the
technical giant organizations to take care of the regularly expanding demand by the smart device
users for connectivity and mobility. This new age of services enable the users to remotely access
and manage their data utilizing speech recognitions technique over the smart devices. Virtual
mobile assistants responds to conversational voice commands and conveys a single point that
flawlessly connects with an extensive variety of data. These virtual mobile assistants are able to
control the phone calls, deals with the individual exercises through timetable, empowers the
users to their tsk managers by the means of voice interface. The following paper contributes to
the emergence of the virtual assistants, technology and concept behind this technology security
as well as privacy issues due to the use of these devices. More over the benefits of this
technology, limitations of virtual mobile assistants and its future scope is also discussed in the
different sections of this report.

2MOBILE VIRTUAL ASSISTANTS
Table of Contents
Introduction 3
Mobile virtual assistants 4
Emergence of Mobile virtual assistants 4
Technology and concept behind its functionality 4
Components used 5
The general arrangement of components 7
Literature Review 8
Security and privacy issues 9
Issue 1 9
Issue 2 10
Issue 3 10
Issue 4 11
Lessons learned 11
Reasons behind its popularity 11
Benefits of it 12
Limitations 13
Issues impacting its future 13
Possible feature of this technology 13
Conclusion 15
References 16

3MOBILE VIRTUAL ASSISTANTS
Introduction
A mobile virtual assistant can be defined as the software agents in the handheld mobile
devices. The main functionality of this virtual assistant like Cortana, Siri, and Google Now is to
help the users of the device to have a single point entry by using the voice input accomplish
different tasks using the applications (such as searching, messaging etc.). Progress in the field of
the AI (artificial intelligence), natural language processing and neural networks made it easier for
the smart phone users to easily accomplish different types of tasks only via providing the voice
instructions. As mentioned by Webster et al. (2014), this type of software agents/application are
the able to answer user’s questions, provide recommendations for some specific tasks. In
addition to that, these agents are also capable of performing actions as the response to the user’s
voice request while interpreting the request using the natural language processing.
All the virtual assents are mainly are mainly developed using the principles of natural
language processing. This virtual assistant or the software programs utilize algorithms in order
to take in to consideration the specific components from the sentence provided as input to the
assistant and then matches with the most suitable output for the users query or request
(O'Sullivan & Grigoras, 2013). This agent always collects contextual information about its user
in order to manage and plan tasks for the user. The virtual assistants mainly uses the semantic
web technologies as well as different open data which are available on the Internet.
On the other hand, the contextual information about the user are the calendar, location,
time, appointments with others individuals, relations among different tasks as well as
decomposition between them, history of tasks, user-specific interests for different tasks.

4MOBILE VIRTUAL ASSISTANTS
Mobile virtual assistants
The mobile virtual assistants are helpful for different types of users as they can
accomplish different types of tasks. These virtual assistants are becoming more popular because
of its simplicity and extensive variety of capabilities (Webster et al., 2014). It could be useful for
those that need do things in an easier and speedier way with a single interface. The users who
find it harder to type or search on the tiny small smart phone consoles or touchpads. It can
likewise be utilized to help the aged individuals and youngsters to utilize the virtual assistant by
saying the commands and getting the desired outcome as opposed to propelling between the
different applications or going over all through contacts list down to find a particular name when
need to make a call. Since the application can work disconnected (without the internet
connectivity)
Emergence of Mobile virtual assistants
The mobile virtual assistants are developed in order to simplify the life of the users by
reducing the effort which is needed to complete routine tasks such as placing a call, texting
something to someone etc. (O'Sullivan & Grigoras, 2013). In addition to that, retrieving data real
time data about the weather, traffic movement on the streets, news/other data, remotely
controlling smart home devices, requesting product purchases from online store, communication
with other applications (for instance, the capacity to book an cab from a cab booking application
and interacting another applications.)
Technology and concept behind its functionality
Every virtual assistant that are used by the smart phone users is mainly using the
artificial intelligence to parse the user's instruction that user had said or typed on the console

5MOBILE VIRTUAL ASSISTANTS
after that it returns the useful information against the users query (Webster et al., 2014). For this
AI component the main objective is speech recognition of the user of the device. This type of
user related data are very helpful in determining the user behavior of any specific user with
higher accuracy level (this includes travels of the user, habits, search or different topic
preferences of the user). Even though this is considered as one of the great aspects that helps the
service providers such as google and Apple to be able to return impressively useful results for the
user depending on their habits and preferences (O'Sullivan & Grigoras, 2013). This is also
achieved without sniffing into the users email and dipping into the users search history on the
devices. Thorough voice-search history of the users also permits the organizations to learn the
user’s vocal idiosyncrasies, so that the virtual assistants understand the spoken requests of the
users better in the future events.
Components used
For the mobile virtual assistants, there are mainly three components are used. These are
voice recognition, text to speech conversion and natural language processing engine (Webster et
al., 2014). For this virtual agents, there are no pre-defined formats in order to request theses
agents to do something (complete a task) or answer a user’s question.
Device to get input: smart phones or the Echo Dot or other relevant devices gives a
speech-based user interface to the users. Through this interfaces, the user issues voices based
commands or makes inquiries, and the device responds to the query in speech (Chung et al.,
2017). The devices and the interfaces are always ready for listening user commands, and just
when it recognizes the keyword such as Alexa or “Ok Google” from its user, it transmits the
audio file to the respective cloud services for processing.

6MOBILE VIRTUAL ASSISTANTS
Cloud services to send and receive the user data: The respective cloud services get the
audio file from the devices and process the command using the speech recognition technique and
mapping voice commands to different actions and information customized for the user. Given the
decided aim, the cloud services appoint the third party skilled server, worked by either
Google /Amazon for worked in aptitudes or by an outsider for outsider application abilities.
Figure 1: processing of user commands by Virtual mobile assistants
(Source: Created by Author using Visio)

7MOBILE VIRTUAL ASSISTANTS
Skill servers for processing of the user input/feature extraction: The skill servers are
basically responsible for completing tasks the Google/Alexa servers may perform for a user’s
command, for example, checking the weather condition or requesting a pizza from the store.
Some of the basic tasks are offered and completed by Google/Amazon servers and come as
inherent usefulness for all the users, and these are taken care of by their own servers.
The general arrangement of components
Voice recognition: Voice recognition is clearly the simplest and trivial stage of this
whole process. When a user gives the virtual agents a command, the device gathers the users
analog voice signal, changes it into a sound/audio file (converted into binary code for faster
interpretations) and sends it to the processing backend servers (Chung et al., 2017). The
subtleties of the user’s voice, the noises with the command and the neighborhood articulations
make hard to interpret it in a correct way. It is called Human User Interface versus the standard
Graphical User Interface we are utilized to. It is vital here that, consistently, google and apple
gathers a large number of user queries of individuals talking distinctive languages, in various
accents from different corners of the world. At the end of the day with their activities and errors
in voice commands, individual users contribute to the improvement of the different virtual agents
developed by the different organization to provide ease of use to their users. Mobile virtual
assistants today gets about a billion queries for every week, and the backend processing are able
to respond to the queries with only a 5 percent mistake rate (Nguyen & Sidorova, 2017). As of
late, Apple, google and Microsoft are still working on the used algorithms and AIs for improving
their performance and user experience.
Most of the mobile virtual assistants follow a similar architecture in order to interact with
the user and complete different tasks.

8MOBILE VIRTUAL ASSISTANTS
Literature Review
As mentioned in their research paper to determine a model that will help in measuring
the quality of the user experience and usability of the mobile virtual personal assistants, the user
experience needs to be the most important component in the design and development of virtual
mobile assistants. The researchers explored that, visual interaction with the virtual assistants is
another component that must be considered as the important segments to the Virtual mobile
assistants that is not present in the virtual assistants when contrasted with a human assistant. It is
a question worth inquiring as to whether we ought to try and investigate the visual viewpoint
keeping in mind the end goal to enhance the quality of interaction with the virtual assistants
(Webster et al., 2014). In any case, there are a few different segments of the existing virtual
mobile assistants that are known to be the confinement. Initially, building up an ideal tone and
pitch to convey that coveted effect of the voice is also important. Furthermore, the precise
identification of commands given in different languages and accents restrains the reliance and
use of a virtual mobile assistant.
In addition to that, the researchers McTear, Callejas and Griol (2016), opined that in any
assessment of a mobile virtual assistant framework it is important to critically ensure the user
satisfaction while they interact with the assistants (O'Sullivan & Grigoras, 2013). Multimodal
interfaces consolidate visual and sound-related signals to advance the interactions. However,
they raise new difficulties concerning the ease of use and adequacy of such interfaces.
From the results of the Project FASiL or “Flexible and Adaptive Spoken Language and
Multimodal Interfaces” project , a virtual assistant framework was created to comprehend which
factors influence the client encounter and the acknowledgment of multimodal administrations.
Results demonstrate that a conversational and multimodal approach was extremely all around

9MOBILE VIRTUAL ASSISTANTS
acknowledged and upheld by the users (Chung et al., 2017). Moreover, the quality and speed of
the framework criticism and in addition the acknowledgment precision of the talked parts are key
components to a better user interaction.
As per the investigation of the researchers found that the level of user satisfaction
depends on the extent of interactivity, sense of presence of the virtual assistant, degree of
reliability, degree of dependency, extent of focused attention , easiness of use, extent of
satisfaction and the extent of users expectation being met (Webster et al., 2014).
The VPA is relied upon to help and improve the accompanying highlights:
Informing the user: Mobile experts can oversee voice message, email and faxes utilizing
their VPA from the street.
Managing Contact: Scheduling, arranging, aggregate date-book, contact and referral
association, all can be dealt and using the virtual mobile assistant.
Call Control: The virtual assistants empowers remote users to perform gathering calling
and call administration. Warning and sending highlights enable remote clients to be told instantly
when they get particular phone messages, messages, faxes, or pages (Nguyen & Sidorova, 2017).
Web Applications: The virtual mobile assistants enables the users to get to and connect
with the web to offer assistance them source data going from climate, headings and calendars, to
stock execution, aggressive information and news. All are utilizing basic, conversational voice
orders.

10MOBILE VIRTUAL ASSISTANTS
Security and privacy issues
Issue 1
In most of the cases, the virtual assistant sends the users queries or requests to the main
server to parse and process the command or the request. After processing the request the suitable
response to the query is send back to the user’s device. When the user's request are sent to the
main server for the processing, it is sent with unique device ID so that the response to the query.
Even though the user queries that are combined with distinctive device IDs that are not
connected to any of the specific users but Apple provisions this user requests to the Siri with the
corresponding device IDs at least for six months (O'Sullivan & Grigoras, 2013). In addition to
that, after six months when the IDs are deleted they keep the audio clip of the users request for
additional 18 months. Thus in case the main server or the IT infrastructure of the company.
Issue 2
In Most of the cases, it is found that the mobile virtual assistants respond to the indirect
speech of the user. These assistants execute commands/orders conveyed by means of a voice
message, voice recording. In addition, the assistants can execute the same command more than
once, regardless of the possibility that those commands are delivered from the same digital sound
file (Coskun-Setirek & Mardikyan, 2017). This happens as the assistant is not able to distinguish
between a recorded voice and natural voice of a user. In addition to that, these assistants are
additionally ready and area able to execute the commands that goes through obstructions.
Issue 3
The virtual mobile assistants are also vulnerable to the different hacking attacks which in
turn help the hacker to take control of the Users Amazon Echo. After getting control over the

11MOBILE VIRTUAL ASSISTANTS
Echo, they can convert it into a personal Eavesdropping microphone to spy on a user. In
addition to that, it is also found that the virtual assistants like the Google’s Assistant and Apple's
Siri are also can be hijacked by using the inaudible sounds ( sounds that are not audible to the
human due to their too high frequency; more than 20,000Hz ).
Researchers from Zhejiang University were able to hack and get the control over “Siri”
the virtual assistant used for the MacBook’s and iPhones, Cortana that is used for windows PC,
Bixby of Samsung phones and Amazon Echo speakers (Coskun-Setirek & Mardikyan, 2017).
Even though the exploitation of the vulnerabilities of this mobile assistants requires attacker’s
high level knowledge of embedded systems as well as Linux operating, but after gaining the
control over this tools
Issue 4
Again the virtual mobile assistants are not able to differentiate the users from their voice.
Therefore, it is possible for the other users to order or command anything through this virtual
assistant. As a result, the unauthorized and unauthenticated users can order and provide different
commands to the users.
Early in this year, a TV channel of California Channel CW-6 was reporting a children’s
order of doll house and some cookies, eventually the other Alexa of the devices of at the
viewer’s homes also collected the anchors command as the command from its users and ordered
dollhouses and resulted in huge amount of orders for the dollhouse.
Lessons learned
Reasons behind its popularity
Following are the reasons behind the success of these virtual mobile assistants,