Development of a Mobile Application for Non-Verbal Autistic Children Using Content-Based Image Retrieval

Table of Contents
Introduction.................................................................................................................................................2
Problem statement...................................................................................................................................3
Literature Review........................................................................................................................................4
Methodology...............................................................................................................................................5
Initial Planning........................................................................................................................................5
Planning...................................................................................................................................................5
Requirement............................................................................................................................................5
Analysis...................................................................................................................................................6
Design.....................................................................................................................................................6
Testing.....................................................................................................................................................6
Implementation........................................................................................................................................6
Evaluation................................................................................................................................................6
Maintenance............................................................................................................................................6
Development...........................................................................................................................................6
References...................................................................................................................................................7
Appendices..................................................................................................................................................9
1. Project Proposal...................................................................................................................................9
2. Blueprint............................................................................................................................................19
3. Project Management Plan.................................................................................................................20
4. Questions...........................................................................................................................................21
5. Questionnaire Analysis:.....................................................................................................................22
6. Ethics Section A & B:..........................................................................................................................31

Introduction
The recent years have seen the prevalence of Autism Spectrum Disorder(ASD) increase at
unprecedented rate. So far, the development condition is listed among the major cases of
mental disorders in various parts of the worlds including the UK, US, and India. Note that the
incidence rate in the mentioned regions are 1 in 64, 1 in 68, and 1 in 150 respectively
(Kuriakose & Lahiri 2017). Such realization is quite devastating considering the detrimental
impact of the condition on the autistic child social life. The implications are felt by not only
the autistic individual but also their direct guardians (mainly parents). At this point, the
readers are urged to consider the established facts regarding autism. The lifelong condition is
usually evidenced in the children under the age of 3, and it curtails their ability to
communicate normally, from social relationship, and engage in normal behavior. While this
disorder is barely understood from a casual perspective, most researcher and practitioner
argue that the results from defects in the brain regions dealing with memory retention, motor
skills, and empathy.
Fortunately, the escalation in the prevalence rate is concurrent with the digital revolution.
The modern world is characterized by sophisticated level of Information and Communication
Technology(ICT) advancement that appear instrumental in the alleviation of most of the
communication and socialization challenges faced by autistic children. As far as autism
concerned, major process has been made in the fields of Information Technology and
robotics (Teevan, Dumais, & Horvitz 2017). A range of computer, smartphone, and artificial
intelligence (merged with robotics) software have been developed to help this population
cope with their communication and socialization needs (Golestan, Soleiman, & Moradi
2017). The applications have been tailored to meet the special communication capabilities of
autistic children. Nevertheless, there is too much fragmentation in the emergent intervention
on the same. For this reason, we have been always curious about nonverbal children with
autism, how they express their feelings, needs and thoughts with others surrounding them.
We find it a quite interesting idea to give them a communication medium and see what they
will offer to us. Therefore, this dissertation project proposes we inspire to develop a platform
which include an android mobile application and a web admin panel, together this platform
will help the non-verbal autistic children to communication and give them a way to express
themselves.

Problem statement
By now, it is apparent that different interventions have emerged in attempt to improve the
lives of people spectrum from autism. Some of the common applications include Pocket PC
Picture Exchange Communication System (PECS), Let’s Talk Voice Output
Communications Aid (VOCA), dmTEA, and AUTHIC. Note that the first two focus on
developing the children’s communication skills (Shminan, Sharif, Adzani, & Lee 2017).
dmTEA is a diagnosis tool while AUTHIC improves the autistic child emotional capabilities.
It suite to restate that the mentioned applications are among the various interventions
implemented for autistic individuals. Such a trend demonstrates the problematic state of the
current progress with respect to the disorder. Logically speaking, it is almost impossible for a
single autistic child to benefit from such a fragmented approach. Such a bold sentiment
follows the fact that one cannot access all of the applications regardless of the fact that they
focus on different aspects of ASD. Since here in Oman the unavailability or scarcity of using
application tools in the centers to communicate with the nonverbal and verbal autistic
children.
Perhaps, communication platform of digital content based image retrieval for Autistic
children can solve this problem by enhancing the accessibility of the distinct functions –
includes digital library on a single platform(s).

Literature Review
At the moment, there is a conventional agreement among researchers, academicians, and
practitioners regarding the need to invent large-scope solutions for the growing autistic populace.
(Leroy, Chuang, Huang, & Charlop-Christy 2005) demonstrate their agreement by
acknowledging the fact that ASD is the third most prevalent neurological disorder in the United
States. Similar sentiments are held by (Kuriakose & Lahiri 2017) as they believe that its
prevalence expands beyond the U.S. borders. According to them UK and India are also recording
high incidence rates. Such perspectives are vital in understanding the global status attained by
autism. It is important to know this fact since it supports the idea that the interventions should be
geared towards the attainment of broad needs.
It appears wise to assert that the preceding paragraph does not intend to disregard the current
progress made in the field. For instance, (Kuriakose & Lahiri 2017) propose a VR-based
application that utilizes a simulated environment to teach basic communication and socialization
skills to the autistic children. This innovation can be said to be an extension of Iyer and
Kalbande’s proposed autistic game (2014). Unlike Kuriakose and Lahiri’s invention, the game is
played on a personalized graphical user interface (GUI). At this point, one should reflect on this
project’s thesis: a mobile application -based approach can allow the autistic users to access a
range of interventions from a single application.
When placed into perspective, a huge operating system can be created where both GUI and AR
functions can be accessed by the autistic children. For the sake of this argument, the search
engine concept described by (Zamir, Korn, Fikes, & Lawrence 2010) is quite illustrious: such a
system utilizes a user profile to extract relevant information based on the user’s query. With the
help of the guardian/practitioner, an autistic child can utilize this system to access various
functions including the learning medium (AR or tablet mode), the learning concepts
(communication, basic mathematics, or socialization), and level (based on progress). Of course,
such a platform must adhere to the improvements suggested by (Kodgiwar 2016). According to
her, content based image retrieval (CBIR) allows the system to access data based on graphical
searches. This function is quite beneficial for children spectrum from autism since they are more
efficient with visuals than words (Kamaruzaman & Azahari 2014). In their research, (Agarwal,
Sharma, and Dubey 2016) noted that CBIR can be improved to facilitate sketch image queries.
Such an operating system can only be efficient if it is full of visuals and colorful feature.

Methodology
Iterative Methodology: the system development life cycle to be used is the iterative prototyping
methodology. This was arrived after consideration of the requirements specification, which are
understood and predicted to evolve with time. The process allows the developer to start with a
limited part of requirements and the iteration design gives room to return and add to the
requirements to expand the abilities repeatedly until the system is fully developed, and can be
implemented and later deployed (Anon, n.d).
Initial Planning we will design a mobile application using content based image retrieval (CBIR)
Planning Considering the nature of the investigation, as shown in (Table 1), researcher will
utilize information collected from professionals in the fields of The age of autistic children will
be between 4 to 11 years the status of their situation nonverbal. Case study method of selection
institutions that contain English Speech Language Therapists (SLT), special education Teachers,
child psychologist and autistic children. Number of cases will be between 3 to 5 therapists who
use English language in therapy will be in a groups depend on the number of the institutions.
Number of cases will be between 3 to 5 institutions depending on the availability of those
children and therapists. According to the final statistics gathered by The Ministry of Social
Development the total number of autistic children in Oman around (1242, Figure 1). These
statistics does not show which institution uses English language in their therapy. Case studies
will be taken from the capital area/Muscat governorate because of the density of the institutions
and number of children.
We prefer 3 to 5 therapists from each institution due to the scope of the proposed project: Each
group of the therapists with their assigned autistic children will be having a research designed
using semi structure interview questions from the researcher who will be collecting the response
garnered from the posed research questions will be on both Arabic and the English language.
Once we collect the data in Arabic form will be translated into the English language.
Table 1. Planning methodology of gathering data from the institutions
Group
Type of
Sector Institution Language
Therapis
t Participant age (4 to 11) Observation
1 Private 1 English Male Female Non-verbal
A
B
C
2 Private 2 English D
Requirement the requirements provide an understanding of the proposed system. This established
the main essential requirements, at the best level of knowledge for a start, that is expected to
increase or/ and evolve with the time. Case modeling can be used to identify the products
beforehand like the system specification (Table 2). The requirements will then be classified and

prioritization allocated. They can then be enlisted in vision document or a system charter for
easier reference.
Analysis after collection (Appendance 3 Questionnaires Analysis), the information will be
collated then assessed to enhance the researcher’s gaps of the potential merits and demerits of the
proposed project. It is through this process that applicability of the proposed solution can be
tested before the design and development process begins. Measuring social development by
conducting paper impact of the electronic impact image.
Design as mentioned earlier, the integrated effectiveness of digital content-based image retrieval
application for ASD children will utilize the structure of a typical search engine. The indicators
for observation schedule, and testing emotional expression in the autistic children. The only
difference is that it will be highly graphical due to the content/sketch-based image retrieval
algorithms. While designing the application we will sketch a new image using character 2D
sketch for different ages such as mother, father, grandmother, grandfather...etc. we will be
adding some character with traditional Omani and casual cloths it will be familiar for them.
Testing the software will be tested during the prototype stage. Further, enhancements can be
done only after the prototype proves functionality. Also, testing the functionality of emotional
expression in the participant.
Implementation the system iteration developed inclusive of the evolved and additional
requirements. We will review the code using the therapist’s user feedback received from the
testing and implements the changes and extra functionalities to suit the therapist’s requirements.
Each module will be tested and modified or updated accordingly.
Evaluation the process will be based on the number of ASD users and therapists in the system.
Maintenance once the approved modules will be tested to satisfaction, and the incomplete
modules are redone to full functionality.
Development it is the last stage of development. The successful of mobile application will be
deployed into a fully operational system version. It will necessitate various procedures including
licenses acquisition, prototype development, and network expansion. Each stage demands huge
amounts of financial and human resources. Not to mention the fact that such a project can be
taken up to a six month before completion.
Table 2. Software & Hardware Requirements
Details Software & Hardware SW.Version HW.Version
IDE IDE: Android Studio 3.2
Programming Language Java 1.8 and XML 1.8
Web admin Firebase Cloud Firestore and SDK 17.1.3
Android API 17.1.3
Server Database Firebase Cloud Database 17.1.3
IDE(Integrated development environment) Android Studio 3.2
Android device Support android media framework