Investigating Speech Hesitation Impact on User Communication with IVR

This report investigates the impact of speech hesitation on user communication when interacting with Interactive Voice Response (IVR) systems. It employs a primary research method involving a survey with 40 participants, divided into two groups, to assess their communication behavior and partner perception when using speech interfaces with and without hesitation. The study also proposes a secondary research method using semi-structured interviews to gather qualitative data. The primary data collected through questionnaires will be analyzed using statistical methods like ANOVA to identify significant impacts. The research aims to understand how speech hesitation affects the efficiency of human-computer interaction, particularly in the context of emerging speech interface technologies like Siri, Alexa, and Google Duplex. The findings will contribute to improving the design and usability of speech-based systems by addressing the challenges posed by speech impairments and hesitations.

Running Head: SPEECH HESITATION IMPACT ON IVR
SPEECH HESITATION IMPACT ON USER COMMUNICATION INTERACTING WITH
IVR
Student Name
Williams College

SPEECH HESITATION IMPACT ON IVR
I. Primary Research Method & Analysis Description
Speech interact is linked to language production with their interaction leading to
felicitous communication. Wagner, Malisz, and Kopp (2014) identified the multifaceted
nature of these interactions. Communication includes the complex process of simulation in
technical systems. Bernsen, Dybkjaer, and Dybkjaer (2012) identified the interaction between
human and computer interaction where the computer system acts as a partner and tool in
communication. The book provides one of the foundation bases for interactive speech
system. It includes the development and evaluation process for completeness and consistency
of the speech interactive systems.
The paper evaluates the current scenario in speech interfaces along with the impact of
hesitation on user communication while interacting with IVR (Jameson et al, 2010). The scope of
the current study conducts quantitative analysis on the impact on user's communication behavior
and partner perception while interacting with personal speech interface assistant such Siri, Alexa,
and Google Duplex and so on. This study has identified two dependent variables and two
independent variables. The study has assumed two hypotheses; the first one is that there are no
recent advances in the domain of speech interface. The second hypothesis assumes that hesitation
has no significant impact on communication behavior. The method used by the study includes 40
participants. The participants will be divided into two groups and they will be asked to
communicate and collaborate with a speech interface, which uses speech hesitation. Then the
participants will be asked to fill out an online questionnaire in the college laboratory. The
answers received from the participants will be analyzed to understand the impact on
communication.
In the current study, the primary research method of survey is being used. The survey is
an experimental research design method which entails gathering data from a sample of people
2

SPEECH HESITATION IMPACT ON IVR
(Miller, Birch, Mauthner & Jessop, 2012). The sample of the population is selected such that the
results can later be applied and generalized to the entire population set. It is a critical source of
data for this study and the survey can be conducted in four modes. The four modes are face-to-
face surveys, telephone surveys, self-administrated computer surveys conducted online and self-
administered paper and pencil survey. For this paper, I had conducted a self-administered
computer survey after briefing the candidates regarding the aspects of the study.
The primary method includes first-hand data collection for the study and then analyzing
the results thereof. The primary research method is a methodology is used by the scholar when
he does not want to depend upon data that has been previously collected for the research. The
primary method resorts to such that in-depth analysis of the data can be conducted (Silverman,
2016). The primary research method is pinpointed research that is undertaken to address the
specific research problem and then identify a specific solution. There are various advantages of
primary research method, some of which are;
 Data is collected first hand and it is accurate in nature. This method helps avoid
dilution of data and the process of this method can be designed to meet personal
needs and requirements. As in this case, primary research was customized such
that it can meet the needs of the study.
 The method of primary research focuses on the problem at hand. Its focus is
directed on the subject matter so as to find a suitable solution. This method of
study allows the researcher to go in-depth of the problem and then analyze all
probable foreseeable options.
 The data that is collected can be controlled as primary research acts as a means
to control the way in which the data is collected. Once the data has been
3

SPEECH HESITATION IMPACT ON IVR
collected, it is up to the scholar to make the best use of the data such that a
meaningful insight can be drawn from the research.
 Primary research is a method that is a time-tested process. One can easily rely
upon the results which are obtained by conducting this type of research.
Though primary research has various advantages, there are certain disadvantages as well.
Some of the disadvantages concerning primary research are;
 There are generally high costs associated with conducting primary research. Often
scholars need to spend a large sum of money on the set up of primary research. It
is often not feasible in the case where a large number of complex variables is
involved to conduct primary research.
 This type of research method is extremely time-consuming. Conducting
interviews, sending out and receiving online questionnaires and then analyzing
them are an exhaustive process, which needs patience and time. Evaluation of
results and the application of findings in order to improve the output again
requires additional time frame.
The primary data once collected will be analyzed using statistical methods of ANOVA.
The proposed statistical method will take inputs from the questionnaire; arrange them in a Ms
Excel sheet. From there the input will be given into SPSS or another suitable software package
for analysis of variance. Once the output from the data has been generated then such data will be
used to be compared with studies conducted in the same domain in the past to arrive at findings
and results of the study.
4

SPEECH HESITATION IMPACT ON IVR
II. Proposed Secondary Method & Analysis
Speech interfaces are a new technology evolution involving speech (Ramakrishnan & El
Emary, 2013). It is a software interface which makes use of simulated human speech or employs
human speech, by breaking down speech interfaces to auditory user interface and graphical user
interfaces. Users are often applying speech interfaces with speech impairment and hesitation.
However, speech impairment and hesitation are known to affect the efficiency of interpersonal
communication and also communication with such speech interfaces. The main motivation for
speech interfaces is to cover a large group of users naturally and to be useful. Automatic Speech
Recognition (ASR) efficiency lies in its ability in the creation of an efficient interface.
Shrawankar and Thakare (2013) in their article identified that the sole purpose of speech
interfaces is an enhancement of man-machine communication. These systems need to be
environment-independent in order to increase their performance levels. Speech hesitation and
impairment has been recognized to be a common problem affecting the majority of the world's
population. On the other hand, there is an increasing importance of speech interfaces becoming
an everyday instrument, assisting various situations where people need help. The aim of this
study is to analyze the technological advancements in speech interfaces to understand the impact
of speech hesitation and impairment on it. There still remain difficulties with ASR and can be
classified into Speakers characteristics and environmental conditions. External factors such as
physiological differences and environmental changes along with speaker communication
behavior affect the performance of such systems. In order to conduct any study, it is important to
have a secondary research method proposed such as an interview.
An interview research methodology is a qualitative research technique that engages
conducted individual interviews with a small number of respondents. The interview allows
5

SPEECH HESITATION IMPACT ON IVR
exploring perspective regarding a particular idea, program or a solution (Flick, 2015). There are
three different types of interview structured interviews, unstructured interviews, and semi-
structured interviews. For the purpose of this study, the semi-structured interview would have
been ideal. The semi-structured interview involves components of structured as well as
unstructured interviews. This implies there would have been a pre-determined set of questions
for the interviewees as well some data collection in an informal manner. The additional questions
asked during the interview to clarify further issues would have allowed detailed insights into the
topic.
The secondary method implies collecting data from a data source that is available.
Secondary data is referred to a data that has already been collected and analyzed by someone
(Neuman & Robson, 2014). Secondary data for this study can be used to save time and costs
related to the primary data collection method. There are several advantages of secondary method
research, some of which are;
 The secondary method has information available for research. There is a wide
range of available data sources from relevant sources, which can easily be utilized
for the study.
 It is less expensive and time-consuming in nature, as data only needs to be
extracted from an authentic source.
 The data is quicker to collect and analyzed, as it has already been used by
someone else. It is prepared to ready-to-use data for conducting the study.
Some of the pertinent disadvantages of the secondary method are;
6

SPEECH HESITATION IMPACT ON IVR
 Though this second method is readily available, a test of credibility has to be
performed to check for authenticity of the data.
 Not all secondary data obtained in the method is the latest reports and from
authentic sources. Even in case, accurate data might be available, it might not be
updated.
In the case of secondary method data collection, then analysis adopted can be statistical
analysis. Similar to the primary method ANOVA can be performed. However, each and every
data collected will need to be cross-checked for validity and reliability. It has to be ascertained
that the data collected fits the purpose and goals of the study.
Planned Method
The study has made use of the primary method of data collection and then utilized
quantitative data analysis techniques. In order to conduct the study, the following procedures
were undertaken.
Participants
In conducting this study, 40 participants will select. A letter of invitation through email
will be sent across to college students, indicating the aims of the study. The age group of the
participating students will be 18 years to 26 years. Both genders, transgender, lesbians, and gays
will be sent an invitation for taking part in the study. Exclusion criteria have not selected
students with asphasia or deaf students. Students with speech hesitation and impairment have
been specially selected for the study. A total of 200 emails were sent across to students from
different disciplines. Once the participants accepted the invitation, they were called to the
7

SPEECH HESITATION IMPACT ON IVR
laboratory to take part in the study. Out of 200 emails sent out, 60 students turned up for
participating in the study. The participants were divided into two groups and given instructions
of the study. The participants were called to assemble at 8 am in the college laboratory, where
they were given out instruction pamphlets. The first group, Group A participants were asked to
interact with an Intelligent Personal Assistant’s voice Alexa without hesitance in their
interaction.
Materials
The study has made use of a questionnaire that was designed specially to obtain inputs
for the study. The participants who agreed to take part in the study were made to sign the
participant consent form. A detailed and in-depth questionnaire was prepared in order to obtain
feedback from the 40 participants of the study. The questionnaire was sent across the
participant's email and they were called to the laboratory for taking part in the study. The
participants filled out the questionnaire and then they were provided with a certificate of
appreciation for taking part in the study. Since it was an online questionnaire fill-up; participants
were provided with computers or laptops to ensure that they could easily fill out the forms for the
study. Once filling out of the questionnaire was complete, each participant was handed over
certificates signed by the scholar, thanking them for being a part of the study.
Design
The study design comprises of an experiment that was undertaken in order to conduct the
study. The clinical study design was selected for the purpose as it is appropriate for
psychological studies. Moreover, the clinical study design was used as it involves human beings.
8

SPEECH HESITATION IMPACT ON IVR
This clinical study is aimed at ascertaining the impact of speech hesitation on IVR. This is an
observational study with the required analysis. The observational study is based upon a
prospective cohort study. The longitudinal study including samples of people with common
characteristics was undertaken for conducting the research.
Procedure
The participants of the study were initially sent out emails for taking part in the study.
Once the participants agreed, they were asked to assemble at the laboratory, where they were
briefed regarding the study procedure and processes. After each of the participants understood
regarding the goals and aims of the study, they were handed over with a participation consent
form. Once they signed the form, they were allowed to take part in the study. Firstly, they were
divided into two groups, Group A and Group B, and then they were made to communicate with
and without speech hesitation with an IVR. Post which they were asked to provide their feedback
or input in the questionnaire. The entire procedure took a total time of 6 hours. There were some
problems which participants faced in communicating with IVR they were unfamiliar to. In such
cases, the scholar assisted them with technical support. After the study was completed, each
participant was handed over a certificate, thanking them for their valuable time and for taking
part in the study.
Ethical Implications
9

SPEECH HESITATION IMPACT ON IVR
The study was aimed at catering to ethical guidelines as much as possible. Firstly, the
proposal of the study had been presented to the ethical committee present at the University for
obtaining ethics approval. It included filling out of ethical forms which are a mandatory part for
conducting any research at the University. Secondly, I as the research scholar have tried to
undertake as much as ethicality in my study as possible. The followed the following steps to
ensure that my study resorts to ethical guidelines.
 Every participant was briefed regarding the aims and objectives of the study.
They were also told regarding the implications and the results that we were
aiming at from the study.
 Every participant was made to sign the participant consent form such that they
could form a part of the study.
 None of the participant’s names were disclosed in a study. Full confidentiality of
names and personal information of participants was ensured.
 I did not copy any part of this study from previous research. In case any part was
referred to for the purpose of this study, then appropriate reference was included
for the same.
 Each of the results of the study was cross-checked and verified to ensure its
transferability nature.
10

SPEECH HESITATION IMPACT ON IVR
References
Bernsen, N. O., Dybkjær, H., & Dybkjær, L. (2012). Designing interactive speech systems: From
first ideas to user testing. Denmark: Springer Science & Business Media. Retrieved from
https://books.google.co.in/books?
hl=en&lr=&id=BR7oBwAAQBAJ&oi=fnd&pg=PR7&dq=speech+hesitation+impact+on
+speech+interface&ots=1C2iakzMSQ&sig=iVqy58mE-DhlZ6-
rdkhZjMjG1Kg#v=onepage&q&f=false
Flick, U. (2015). Introducing research methodology: A beginner's guide to doing a research
project. 2nd Edition. UK: Sage. Retrieved from https://books.google.co.in/books?
hl=en&lr=&id=jcOICwAAQBAJ&oi=fnd&pg=PP1&dq=research+methodology&ots=rr
Szb_579W&sig=nB9k-TNBtwMuhV-6KFFwU30DyVI#v=onepage&q=research
%20methodology&f=false
Jameson, A., Kiefer, J., Müller, C., Großmann-Hutter, B., Wittig, F., & Rummer, R. (2010).
Assessment of a user’s time pressure and cognitive load on the basis of features of
speech. In Resource-adaptive cognitive processes (pp. 171-204). Springer, Berlin,
Heidelberg. doi:10.1007/978-3-540-89408-7_9. Retrieved from
https://link.springer.com/chapter/10.1007/978-3-540-89408-7_9
Miller, T., Birch, M., Mauthner, M., & Jessop, J. (Eds.). (2012). Ethics in qualitative research.
2nd Edition. UK: Sage. Retrieved from https://books.google.co.in/books?
hl=en&lr=&id=g2pEAgAAQBAJ&oi=fnd&pg=PP2&dq=research+methodology&ots=s
HHpY1JsL9&sig=B5XtzKvnyEoaFU4vSSMqG-Rw0T0#v=onepage&q=research
%20methodology&f=false
11

SPEECH HESITATION IMPACT ON IVR
Neuman, W. L., & Robson, K. (2014). Basics of social research. 2nd Edition. Toronto: Pearson
Canada. Retrieved from http://library.mpib-berlin.mpg.de/toc/z2007_809.pdf
Ramakrishnan, S., & El Emary, I. M. (2013). Speech emotion recognition approaches in human
computer interaction. Telecommunication Systems, 52(3), 1467-1478. doi:
10.1007/s11235-011-9624-z. Retrieved from
https://link.springer.com/article/10.1007/s11235-011-9624-z
Shrawankar, U., & Thakare, V. M. (2013). Adverse conditions and ASR techniques for robust
speech user interface. arXiv preprint arXiv:1303.5515. Retrieved from
https://arxiv.org/abs/1303.5515
Silverman, D. (Ed.). (2016). Qualitative research. 3rd Edition. UK: Sage. Retrieved from
https://books.google.co.in/books?
hl=en&lr=&id=9FALDAAAQBAJ&oi=fnd&pg=PP1&dq=research+methodology&ots=
9ncDgA4w6F&sig=ey_cRmQCh0VXD5sTsy2bJQcSvsM#v=onepage&q=research
%20methodology&f=false
Wagner, P., Malisz, Z., & Kopp, S. (2014). Gesture and speech in interaction: An overview. doi:
10.1016/j.specom.2013.09.008. Retrieved from
https://www.sciencedirect.com/science/article/abs/pii/S0167639313001295
12

SPEECH HESITATION IMPACT ON IVR
13