Final Report: Human Computer Interaction - Autism Support Project 2020

This final report details a Human Computer Interaction (HCI) project aimed at developing a device to help autistic children and their parents better understand each other's feelings. The project involved creating a smartwatch that monitors the child's emotional state and transmits this data to a mobile application accessible to parents and teachers. The report covers the project's purpose, including the need for improved emotional expression and understanding in children with autism. It describes the development of a persona, Nils, an eight-year-old child with autism, to guide the design process. The requirements elicitation phase involved interviews with parents and reviews of relevant literature, emphasizing the importance of non-verbal communication. The prototyping phase includes the design of a necklace and a smartwatch, along with the application interface, and the report provides detailed descriptions of the prototypes and their functionalities. The project highlights the importance of user-friendly interfaces and accurate emotional sensing for effective support. The report concludes with a discussion of the project's findings and implications for future research and development in assistive technology.

Final Report
Human Computer Interaction
Date: 2020-01-08
Students: Isac Blomberg, Casper Eriksson, Gustav Schmützer, Ambjörn Eriksson,
Elias Germer and Md Nur Uddin Sabuj
Förstasida klar
1

Abstract (sammanfattning) klar
Innehållsförteckning, Elias klar
Purpose, klar
Persona, klar
Requirements elicitation, klar
Prototyping, klar
Feedback (user evaluations) (Casper kör paint och sen klistrar in) klar
Final prototype, application interface, klar
Final prototype, Smart-watch (gustav fixar till) klar
Discussion Ambjörn
Abstract
2

Our initial idea was to create a device that helps autistic children and their parents to
understand each other's feelings better. We created a smartwatch worn by the kids
and that is connected to an app in their parents phones. This watch can sense the
kids feelings and display in the app so that their parents or teachers knows how the
kids feel at all times. The design of our product is very simple, the watch looks
exactly like any other smartwatch on the market and the application interface is
designed to be easy to understand and use.
Table of content
3

Purpose 5
Persona 6
Requirements elicitation 9
Prototyping 11
Final Prototype 17
Discussion 20
Purpose
4

The main goal of our product is for children with autism to easier express their
feelings and for the surrounding to better understand the child’s mood and current
state.
During our elicitation process, we discovered that many children with autism do not
express their feelings very clearly and it can be difficult for the surrounding (parents
and teachers for example) to understand what the child is feeling. This can cause
ambiguities between the parent and the child which in turn, in the worst-case
scenario, can lead to that the child does not get the support it needs.
At this state of our design process, we also found, by several articles, that non-verbal
communication can often be very effective when communicating with a child with
autism, why we decided to try to include this in our product as well.
We also want our product to function as an educational tool where the child is given
the opportunity to associate emotional states with different physical attributes. This is
necessary for the product to be as versatile as possible.
Persona
Our persona is Nils. He is an eight-year-old kid that is diagnosed with autism and he
lives with his family in the Swedish city of Umea. His favourite hobbies are computer
games, watching movies and reading comic books. Since Nils has autism, he does
not express his feelings towards the surroundings as well as others and can
sometimes be difficult to understand. Nils is going to a school close by his house
where he can walk. He is in a class with 20 other students, but he has still not found
any friends. It is hard for Nils teachers and parents to do anything about it because
5

they can’t understand his emotions. If the people around Nils could know how he felt,
they would be able to understand him better which would make life easier for both
Nils and for his parents/teachers.
User goals
● Understanding feelings of people with autism (or of another reason can’t
express their feelings in a clear way.
● Easy to wear and won't affect the people wearing it in any profound way.
● An intuitive and easy interface so all people can use and understand the
application, regardless of their technical background.
Scenario 1
The day is 5th of May. In the Swedish town Umea, the sky is cloudy and calm and
the temperature outside is 16 degrees. It is a very special day for Nils, since it’s his
birthday. Fredrik and Hanna, his parents, has bought him a very special and
expensive gift. The gift is a fully functioning “Batmobile” which Nils can sit in and
drive. Nils biggest dream is to drive a car, and in this case, he is a big fan of Batman
as well. The time comes and it’s time for Nils to open his gifts. Nils rips the paper
apart and according to his parents he doesn’t seem very happy about the gifts.
Fredrik and Hanna, mom and dad, saved the “Batmobile” as a last gift that they
would give him. He opens the gift and they have a hard time reading him if Nils got
happy about the gift or not. They knew how much Nils wanted this “Batmobile”, but
his reactions were not what they expected. Nils didn’t react at all. He just went up to
his room and started playing games. His parents were left alone downstairs and
collected the torn paper. They talked to each other and didn’t understand if Nils were
happy or sad. In this situation it would’ve been great if the parents had an easy way
of knowing what Nils felt, without Nils having to express his feelings openly. A device
that can detect Nils’s feelings and display them in some sort of way. Therefore, we
thought of a device that could detect Nils feelings so his parents can understand him
6

better. It is a wearable device around your neck that can display a person's feelings
and emotions in an app. The wearable device is a necklace that can sense a
person’s state of mind and emotions without the person having to express them to
the surroundings. In this case even if Nils didn’t show his emotions, his parents
could’ve seen his emotions using this app. This would have helped both Nils parents
to know if he really was happy or not and it would be better for Nils in the future if his
parents knew how he feels about all kinds of things.
Another scenario where the device may help Nils is in more public places like school
for example.
Scenario 2
Nils is in class and working quietly with some math problems. In Nils class they are
twenty students and Nils get along with everyone, except for two students, Isak and
Emil. Isak and Emil frequently picks on Nils, some days more and some days less.
Nils does not like when getting picked on and it can change his mood very quickly
from happy to sad. However, Nils does not express his feelings for the teacher via for
example telling the teacher what had happened and therefore everything moves on
as usual. When Nils is sad, he has a hard time focusing on the subjects in school.
The teacher asks him if anything is wrong, but Nils just shakes his head with a “poker
face”. Nils learning gets affected by this and his overall mood gets negatively affected
by this. However, if the teacher could use our device to detect how he feels, Nils
could be sent to the student counsellor etc.
Not only for Nils could this device be useful in especially school environment where
it is not easy to really express yourself or “tell on someone”.
7

Requirements elicitation
Overall mission
We have conducted an interview with a parent to a child with autism regarding the
different difficulties that occur during every-day life and possible solutions for these
problems.
The parent expresses frustration in not knowing what his child is feeling in various
situations. Especially though a child with autism often need a lot of emotional
support. Knowing what a child with autism is feeling when he/she can’t express this
themselves is critical for giving the right support and mental training for them to better
handle their emotions.
In our requirements elicitation phase, we have also gone through several articles and
presentations regarding communication with a child that is autistic. An important
lesson from the presentation Samspel och kommunikation (Interaction and
communication)1 is that when children with autism have difficulties explaining their
1 Samspel och kommunikation, Region Skåne.
(https://vard.skane.se/siteassets/habilitering/dokument/utbildningar/autism-hos-barn---samspel-och-
kommunikation--alla-nivaer.pdf)
8

mood with words, non-verbal communication (i.e. gestures, pictures etc.) could have
some progress instead. This could be integrated with the application in form of
pictures of “smileys” with different facial expressions that the child could point to in
order to express his/her emotions. This feature can also be used by the
parent/school teacher to express their feelings towards the child.
We have also consulted a former preschool teacher who has worked a lot with
children that have different kinds of cognitive disorders. She also underlines the
importance of non-verbal communication in forms of pictures or gestures and think
this should be included in the application. She also stresses the importance of an
accurate sensitivity of the application for when the child experiences mood swings or
elevated levels of stress - in other words, it is important that the application really
sends a notice to the user when needed.
One of her concerns though is the security and the ethics around the application; the
security must be of utmost importance when the application handles information of a
human beings’ emotions. She also says that the ethics of having a necklace on a
child in order to monitor its emotions is questionable.
Requirements for the application
The mobile application is going to be used by parents, school teachers and other
pedagogical contacts with the goal of understanding the emotions of a child with
autism. Therefore, the application needs to be intuitive and easy to use for all people
regardless of their technical background. The application needs to in an accurate
way displaying the emotions of the necklace wearer and send notices in moments of
high stress and/or sudden emotional changes.
Requirements for the necklace
The necklace needs to be discreet, light and easy to wear. It also needs to have a
good battery life, be water- and weatherproof and be overall reliable and durable.
9

Prototyping
Introduction
We made two different prototypes during the seminar: One is a necklace with the
function we presented in Assignment 1 and the other one is a smart-watch which, in
addition to have the same function as the necklace, also gives the wearer the option
to scroll between different smileys/emoticons in order to express their feelings in a
non-verbal way (as we found important during the requirements elicitation).
Construction of the Necklace
Figure 1: Prototype of the necklace
10

The picture above represents the prototype for the necklace. We wanted to make the
necklace discreet and easy (light) to wear. Therefore, it does not have any jewellery
on it (besides from the green sensor). The sensor is supposed to read the wearers
emotional status by several different methods (which we will not go into in this
report).
We constructed the necklace by first merging several paper clips together to form the
band that holds the sensor. The sensor was then made of cardboard, which we
painted green. We wanted to make the sensor a bit over-seized for us to evaluate the
weight of it. If the sensor gets to heavy, the necklace will be annoying to wear
(especially for a small child).
Construction of the Smart-Watch
Figure 2: Prototype of the smart-watch
The picture above represents the prototype for the smart-watch. As oppose to the
necklace, the smart-watch gives the wearer the option to in a non-verbal way express
their feelings by scrolling and displaying different pre-programmed
smileys/emoticons. Because of this, we wanted the touch-display to be big and clear
so that a person easily can swap between the different smileys and so that another
person easily can see the display from a distance (not to long of a distant though).
11

We built the prototype by first deciding how big the display should be and thereafter
cutting out the display from cardboard. The watch-band is also cut out from
cardboard, which we then wrapped in paper to give it some comfort.
Our main goal during this built was to make the watch easy to wear, somewhat
stylish and with a big, clear touch-display.
Construction of the Main interface
12