User Experience Evaluation

User experience evaluation is a set of methods, tools and skills that can be used to discover how a system performs for users and how they perceive it after using or interacting with the interface. The report explains the important features of UX evaluation and discusses the evaluation apparatus used for designing and testing the program. It also discusses the user experience evaluation of the given designs and produces scores based on usability and post-task satisfaction level.

Running head: USER EXPERIENCE EVALUATION
USER EXPERIENCE EVALUATION
Name of the Student:
Name of the University:
Author Note:

1USER EXPERIENCE EVALUATION
Table of Contents
Introduction................................................................................................................................2
UX Evaluation Tasks.................................................................................................................2
Observations...............................................................................................................................3
Results........................................................................................................................................4
Expert Heuristic Evaluations......................................................................................................4
Predictive Model Results...........................................................................................................4
Recommendations......................................................................................................................5
References..................................................................................................................................6

2USER EXPERIENCE EVALUATION
Introduction
User experience evaluation is a set of methods, tools and skills that can be used to
discover how a system performs for users and how they perceive it after using or interacting
with the interface. User experience can be subjective as it varies form one person to another.
It can also be called dynamic and context-dependent as they tend to change over time. The
user experience evaluation treats the interface or the functions of the program as a series of
tests. The experience of the user on using the application and its functionalities are evaluated
and then rated based on a set of predefined parameters and score card. These parameters can
be constructs, methods or dimensions as per the targeted area of the application. The target
areas can be gaming, travel or mobile based. The report will explain the important features of
the UX evaluation and discuss the evaluation apparatus used for designing and testing the
program. The report will also discuss about the user experience evaluation of the given
designs and produce scores based on their usability and the post task satisfaction level.
UX Evaluation Tasks
The primary evaluation tasks included testing the software in various devices and
checking all their functionalities. The application tested here is an android based application
called “Task list app”. The application has three main functions. The first function is to show
all the tasks of the system. The second function is to schedule the tasks and the third function
is the group task management. The main tasks performed by the application is divided into
three parts. The first task represents the steps performed by the user to perform function 1 of
the application. This includes clicking on the phone home screen, then touching the phone
screen to view the screen lock, then unlocking the phone to view the application list, then
clicking on the app to view the welcome screen, then inputting the login details to log into the
application and finally then choosing the first function which shows all the tasks. These steps
are the evaluation parameters of the application’s usability and post task satisfaction.
The steps remain same for the task 2 and the task 3 as well. The second task follows
the same procedure that starts from the home screen and ends after entering the login
credentials. The next step in task 2 is to select the second function from the functions list
which helps to schedule the tasks. After that the user will be presented with a screen to add
members to a group and assign members to a task. The task 3 also follows the same
procedure where the first five to six steps are followed in the previously stated manner. The
step changes after entering the login credentials as here the last function is chosen which

3USER EXPERIENCE EVALUATION
helps the user to perform group task management. The next step allows the user to select the
group he wants to manage the tasks for. The last step allows the user to enter the name of
person and also delete it by long pressing it.
The software was not tested in a tablet, hence no tablet was used. The tasks discussed
above is in respect to persona number 1 who is Mr. Joe Martin. The kind of apparatus that
was used were questionnaires and feedback form from the users based on the SUS and SEQ
models. The phone used for testing the android application is a Samsung Galaxy S4 phone.
Observations
The application interface designs can be evaluated as per their usability. The first
screen design is done in a simple manner and denotes the home screen of the smartphone.
The second screen shows the user lock screen where the user unlocks the phone. The third
screen shown is the screen after the phone is unlocked which lists the apps installed on the
phone. The next screen shows the welcome home screen of the application. Next comes the
screen with the fields to enter login details. The next screen shows the list of tasks or
functions available to perform. The tasks shown on the screen are numbered using 1, 2, and 3.
The first task takes two more steps to complete. The user selects the task as its first step and
the next step is to select the tasks from the list that appears on screen. This method is
followed for all the three tasks. The screens for performing the rest of the tasks remains the
same except for the individual task function screens. The design of the interface is very
simple and minimal colours or word art is used.
The interface is designed in a clear and concise manner for easy understanding and
the screen is not too much overcrowded with information. The screens are designed quite
spaciously and each function is provided with enough space for proper viewing and clicking.
The menu of the screen is quite in the flow of the information and is designed as per the
sequence of events. Maintaining a proper sequence of events is another important aspect of
designing an application. As the smartphones are touch based the screens has been designed
keeping this in mind as each and every option or button is provided its own space for easy
selection. The menu after unlocking the phone lists the installed applications and it can be
seen that a box based icon design is used for denoting the applications. This is done to
increase the user visibility and also make the phone screen design attractive. The home screen
of the application has been kept quite plain and simple with only text messages and no

4USER EXPERIENCE EVALUATION
images. Overall, the screen design models follow the basic outline of the requirements well
without going into too much detailing.
Specific user experience (Login screen)
In this section a specific but crucial part of the UX design will be discussed. This crucial part
is the login page or screen which is needed to access the application. The login page is critical
to the security of the application. The login page as given is properly designed with effective
data validation. As can be seen from the design the login page is very simple and has all the
necessary fields and buttons present. The crucial parts of the login system that is incorporated
in the given application UX designs are:
1. The fields to enter the credentials are designed and labelled properly.
2. The login button is there which is crucial to start the authentication process.
3. The “forgot password” button is there which can help the user retrieve forgotten password.
The parts where the UX design could be further improved was by providing an option or
button for first-time users to register for the app as a new user. This button is missing in the
given UX design. Overall the login part will be well accepted among most of the users and
can be given a cumulative UX score of 4/5.
Results
The main evaluation tools used in this experiment was SUS, SEQ and feedback
forms. The SUS or the system usability scale is an important measure of the user experience.
In this format, the users were provided with a questionnaire where the user’s experience of
using the application was measured in a scale from 1 to 5 where 1 being strongly disagree to
5 being strongly agree. The SEQ or single easy questions were also used to determine the
problematic aspects of the given task and how it can be improved. Further feedback forms
were used after the application was released among select people for feedback. Overall the
results obtained were quite positive. The overall score of the application interface for the
tasks can be given a score of 4/5. The interface design can be given a score of 3/5. The
usability of the interface can be given a score of 5/5. The visibility of the interface and its
options can be given a score of 4/5. The navigation aspect of the application can be given a
score of 5/5 as it is very clear. The SEQ questions used suggested that the feedback and

5USER EXPERIENCE EVALUATION
visual aspect of the interface can be further improved by adding a few related pictures or
icons. Overall, the application interface developed can be given a score of 4/5 considering
everything.
Expert Heuristic Evaluations
The first task lists all the available tasks in the system. This task gets a score of 4/5 as
the listing of the tasks are quite clear. The tasks are numbered properly and are clearly
arranged denoting the flow of the information. The tasks shown are also listed in a clear and
spaced out manner helpful in touch phones. The second task allows task scheduling and its
interface will get a score of 3/5. The scheduling have functions to add members to group and
assign them task. This part needed a bit clarity hence the marks are deducted. The third task
is for group task management. This part will get a cumulative score of 4/5 as the groups are
mentioned clearly and the search function is also properly designed and visible as per
requirements.
Predictive Model Results
The GOMS developed for this task are:
1. The main goal of the interface design is to set up a chain of interfaces that the user can
access that will ultimately lead to the completion of the tasks. For example, viewing
all the tasks is the goal of task 1.
2. Operator is the action performed on the device to achieve the goal. In our case this
would be the touching of various buttons on screen or inputting of details during
login.
3. Method will be the whole sequence of operators performed together to achieve a goal.
For example, the whole sequence of actions needed to complete the first task, starting
from unlocking the phone till choosing the option in the task selections screen.
4. The act of touching an option is called selection.
The KLM scores calculated are (Assuming username and password of 10 keystrokes each):
For common part till login in the application: 8.1 seconds
For task 1: 8.1 + 0.1=8.2s
For task 2: 8.1+ 0.1+0.1+0.1=8.4s
For task 3: 8.1+0.3+2.8(for a name of length 10) = 11.2s

6USER EXPERIENCE EVALUATION
Recommendations
To conclude, the overall design of the application interface or UX is quite simple and
efficient. The only aspects where the UX design is lacking is in the visual graphics part and
the part of clearly stating the more internal and complex functions of the application. For
example, the UX design for the overall application could be made more attractive and good
looking. Another major improvement could be made in making the group task management
and schedule tasks functions a little more clear and easy to understand by adding a few more
designs to explain the following steps under these categories. Overall at a usability score of
4/5 this application UX design is efficient and befitting for the requirements.

7USER EXPERIENCE EVALUATION
References
Hussain, A., Mkpojiogu, E. O., Musa, J. A., & Mortada, S. (2017, October). A user
experience evaluation of amazon kindle mobile application. In AIP conference
proceedings (Vol. 1891, No. 1, p. 020060). AIP Publishing.
Hussain, J., Hassan, A. U., Bilal, H. S. M., Ali, R., Afzal, M., Hussain, S., ... & Lee, S.
(2018). Model-based adaptive user interface based on context and user experience
evaluation. Journal on Multimodal User Interfaces, 12(1), 1-16.
Keystroke Level Model (KLM) Calculator. (2019). Retrieved from
http://courses.csail.mit.edu/6.831/2009/handouts/ac18-predictive-evaluation/
klm.shtml
Maia, C. L. B., & Furtado, E. S. (2016, July). A systematic review about user experience
evaluation. In International Conference of Design, User Experience, and
Usability (pp. 445-455). Springer, Cham.
Muslim, E., Lestari, R. A., Hazmy, A. I., & Alvina, S. (2019, April). User interface
evaluation of mobile application krl access using user experience approach. In IOP
Conference Series: Materials Science and Engineering (Vol. 508, No. 1, p. 012110).
IOP Publishing.
Orfanou, K., Tselios, N., & Katsanos, C. (2015). Perceived usability evaluation of learning
management systems: Empirical evaluation of the System Usability Scale. The
International Review of Research in Open and Distributed Learning, 16(2).
Usabilitybok.org. (2019). GOMS | Usability Body of Knowledge. [online] Available at:
https://www.usabilitybok.org/goms [Accessed 12 Jun. 2019].