Usability Analysis of Microsoft OneNote Nielsen Heuristics analysis

Usability Analysis of Microsoft OneNote
Nielsen Heuristics analysis of OneNote
Introduction
Heuristic evaluation is a type of usability
evaluation method that enables
researchers, developers and business
owners to find usability problems in the
interface of their choice. This allows them
to fix the issues found via such heuristic
evaluation in future versions of the
systems via iterative design approach.
Heuristic evaluation is done by having a
set of evaluators examining the system and
judging for its compliance against a pre-
defined set of design principles also known
as heuristics. The evaluation is done
typically by multiple people as oppose to
just one person because it would be
difficult to just for one person to find all
kinds of usability problems within the
system. As a result, a proper usability
evaluation approach involves having
multiple evaluators for the system.
About the application
Microsoft OneNote (Microsoft Store
version) is a digital notebook application
which is a singular and comprehensive
solution to both quickly accessing and
storing information related to work,
personal data, school, organizational data
and so on. OneNote allows this
information to be stored in Notebooks.
These notebooks have tabbed sections and
within those sections are individual pages.
One can add any kind of handwritten note,
or typed note, files, audio, screen
clippings, photos, video and link to other
documents and files as well. OneNote also
offers powerful searching capabilities that
allows one to find information faster.
OneNote also allows for synchronizing,
sharing, collaboration of notes with co-
workers and peers. People use OneNote to
collect, collect, find, organize and share
information. By having a centralized
repository of important data helps one use
time more effectively and efficiently
thereby increasing user’s productivity.
OneNote also comes equip with
backgrounds, templates, graphical images,
charts, diagrams and various formatting
options to make pages as well as notes
more visually appealing and enjoyable.
Using Nielsen Heuristic Evaluation
As mentioned previously, Nielsen
Heuristic evaluation is the pre-dominant
method of evaluating user interfaces since
the mid of 1990. It has stood its test of
time and is still relevant and highly
practical method of evaluation. However,
usability problems are not that easy to find.
Some usability issues may be readily

apparent, however a lot of them may take
advanced understanding of usability and
an intricate eye for detail (Nielsen &
Molich, 1989). Also, it is not always
possible for the person to be the best
evaluation every single time. Also,
sometimes some of the hardest to find
usability problem may be found by an
inexperienced person or someone who is
not very familiar around the system.
Therefore, it is imperative to involve
multiple evaluators for evaluating the
system in terms of heuristics-based
evaluation.
Keywords
Heuristics; user interface; Nielsen;
OneNote, Microsoft
Literature
Usability inspection as explained by
Nielsen is nothing but a generic name that
has been given to a collection of methods
where evaluators examine or inspect a user
interface against a pre-defined list of
usability parameters. These examining
methods may end up using checklists or
even guidelines as their basis for
discovering usability problems. It is also
difficult to decide what all guidelines are
essential to certain problems, it becomes
even more difficult when the overall
guidelines increase. On the other hand,
heuristics-based evaluation methods are
much quicker since they traditionally
employ limited design principles as well as
heuristics. Since heuristic evaluation is
quite quick, it ends up gathering many
different usability problems with just 4 to
5 evaluators at a much cost-effective
method. At the same time, the same
method cannot be utilized if the users are
real-users as opposed to trained users.
(Wharton et al., 1994). A usability testing
can simply be carried out by having just
one trained evaluator, however by
involving multiple evaluators the overall
effectiveness increases dramatically.
Heuristics used in this report are derived
from (Sutcliffe and Kaur, 2000) and
(Nielsen, 1994). As a result, heuristics
such as direct mapping between real world
and the system, standards and consistency,
visibility of system status, control for user
and freedom among others.
Evaluation Methodology
This method would follow Nielsen’s
recommendation for evaluation of
interface. The methodology begins with
two evaluators familiarising themselves
with the application first. Once they have
familiarized themselves with the
application, they would then carry out a set
of tasks and an observer would list the
problems that has been faced by the
evaluator. Since, the application is a real-
world application and already being used

by millions of users worldwide, the set of
tasks being carried out by the user would
also reflect real-world actions. However,
in practice, certain limitations of
technology and the nature of tests and
observation has to be considered and some
compromises has to be made.
Process
Prior, to this evaluation, two different
personas would be created. The first
persona would be of that a of a student.
The second persona would be that of a
Homemaker. They are both would be
provided brief information about the
application, it’s structure, it features, it’s
uses and benefits. These both personas
would begin using the application. Instead
of creating two different usage scenarios
for both of the personas, both of the
personas would use same set of use-case
scenario. So that the baseline is same
across both of the users and thus
comparison can be made (Blair-Early &
Zender, 2008). Once the users accomplish
the task, the number of issues they face
would be highlighted and so the heuristics
evaluation can be completed (Sim & Read,
2015).
Heuristics
The heuristics being used for evaluating
the interface is listed as below along with
the explanations for the same.
Visibility of System status – There should
be an appropriate feedback along within a
reasonable amount of time to keep the
users updated.
Match between real world and the system
– The system needs to speak familiar
language in terms of concepts, icons, hints,
interactions that is something familiar to
the systems and also contemporary.
Real-world conventions – This means that
the information organization should be in
logical and natural order.
User control as well as freedom – The user
should always feel in control of the system
and not the opposite. This means that there
should be options to exist the current state
or section and even allow options for undo
and redo.
Consistency and standards – Users do not
need to wonder any kind of graphical
elements, UI elements, menus and buttons
mean different things.
Error prevention – The system should
prevent users from making an error in the
first place. If at all the errors are made, the
system should highlight them and allow
for understanding the actual error message.
Recognition instead of recall – Th system
should minimize the user’s memory load
and does not make him remember different
options, menus, objects and so on.

Instructions to use the system should be
readily apparent and visible throughout the
system (Nielsen, 1991).
Flexibility and efficiency – The system
should be providing accelerators for use
with the system which may be hidden from
the regular user but made available for the
advanced user (Fuller, 1995).
Aesthetic and design – The design should
be visually appealing and should also be
minimal. Any unnecessary clutter,
excessive dialogue boxes, cluttered
information presentation should be
minimized.
Documentation and help – The system
should provide help and documentation
wherever needed and even as far as
providing step-by-step instructions for
something which is completely unfamiliar
in the market.
Ranking system
Throughout the evaluation process the
issues found would be ranked according to
their severity and / or impact that they
have on the system. Positive findings
affect have beneficial effect on the user’s
ability. While at the same time, negative
findings have disadvantageous effect on
the user’s ability to perform the given
tasks. At the same time, the issues can be
ranked as follows:
a) Cosmetic issue – Affects the
appearance of the system and
should be fixed only if time
permits.
b) Minor issue – Hinders ability of the
user to navigate and needs to be
fixed only that when it’s possible
(Fuller, 1995).
c) Major issue – Causes confusion or
frustration for the user and needs to
be fixed as soon as possible.
Tasks
a) Create a user account
b) Create 3 notebooks
c) Create appropriate sections and
pages
d) Add content to these sections and
pages
e) Search for certain text using the
search functionality
f) Move pages from one section to
another and into a different
notebook
Persona 1
 Name – Gemma
 Occupation – Homemaker
 Income – N/A
 Internet use – 2-3 Hours / day
 Hobbies – Gardening, Traveling,
Book reading, Cooking
 Motivation – Interested in
experimenting with variety of
culinary skills

Persona 2
 Nathan
 Occupation – Student
 Income – 25k / Annual
 Internet use – 4-5 Hours / day
 Hobbies – Swimming, Cycling,
Video Games, Traveling
 Motivation – Interested in
becoming a professional in his field
while passing out with flying
colours.
Findings
Positive Findings
These positive findings conform to
Nielsen’s heuristics of usability evaluation.
1. Visibility of system status
The registration page informs user
whenever problem is discovered. If
the user starts writing or creating
content without logging in, then the
app notifies the user.
2. Match between real world and
system
The application’s terminology is
quite common and well understood
among the intended audience.
3. Error prevention
If the user is not logged in or non-
registered then they are informed to
log in or create an account.
4. Help & Documentation
Users are provided with help and
documentation tutorials.
5. Error messages are in plain English
and is also easy to understand.
6. User control and freedom
There is dedicated back and
forward button allowing use an
additional layer of control over the
app and how they navigate within
it.
Negative findings
These positive findings do not conform to
Nielsen’s heuristics of usability evaluation.
Heuristi
cs
violated
Issue
Title
Description Sco
re
Recogni
tion
rather
than
recall
Searc
h
icon
locati
on
and
confu
sion
Difficult to
locate search
icon/button
because of its
placement and
size. Also, the
location of
search icon is
unconventiona
l. Apart from
this, users
were unable to
figure out if
the search
button
searches for
the current
Min
or

page. Current
page search
option is
available only
when using
the keyboard
CTRL+F
shortcut.
Consiste
ncy &
Standar
ds
Navi
gatio
nal
issue
s
Both of the
evaluators
were unable to
figure out
where the
notebook was
located within
the application
and how they
can move
around the
pages from
one section to
another and to
a final
different
notebook.
Navigational
method is also
unconventiona
l.
Min
or
Recogni
tion
rather
than
recall
Secti
on
and
Page
butto
Button to
create section
and notebook
is difficult to
locate
Min
or
n
Consiste
ncy &
Standar
ds.
Error-
preventi
on
Word
-
proce
ssor
The word
processor
allows content
to be entered
anywhere.
This means
that wherever
a person may
point his or
her cursor, the
content could
be entered and
placed right
then and there.
This is
different as
oppose to a
traditional
word-
processor
which limits
and controls
the area of
text-input.
This had
confused both
of the
evaluators as
they continued
to move
around and
shuffle their
contents,
Maj
or

taking too
much time to
organize the
content the
traditional
way.
Consiste
ncy &
Standar
ds
Parag
raph
block
Difficult to
insert a
paragraph
block
Maj
or
Recommendations
a) Search icon – The search icon
should be moved to a more
traditional location. It should be
made bigger. At the same time, the
current search should default to the
page being loaded.
b) Navigational issues – The main
navigation for the content i.e., the
Notebooks, the sections and the
pages should be shifted towards a
hierarchical or tree-styled design
allowing more granular control
over how users navigate through
their data when it starts to pile on.
c) Section and Page button – The
section and page button must be
moved upwards, someplace where
it is prominently visible.
d) Word-Processor – The word-
processor needs a major overhaul
and should take in the traditional
approach of data input. This would
also make it easier for more
traditional note-taking and open its
way towards organize notes in
paragraphs and so on as needed.
References
Blair-Early, A., & Zender, M., 2008. User
Interface Design Principles for Interaction
Design. Design Issues, 24(3), 85-107. doi:
10.1162/desi.2008.24.3.85
Fuller, R., 1995. Book Review: Usability
Engineering by Jakob Nielsen. ACM
SIGCHI Bulletin, 27(4), 77-78. doi:
10.1145/214132.570138
Nielsen, J., 1991. Usability metrics and
methodologies. ACM SIGCHI Bulletin,
23(2), 37-39. doi: 10.1145/122488.122493
Nielsen, J., & Molich, R., 1989. Teaching
user interface design based on usability
engineering. ACM SIGCHI Bulletin,
21(1), 45-48. doi: 10.1145/67880.67885
Sim, G., & Read, J., 2015. Using
computer-assisted assessment heuristics
for usability evaluations. British Journal
Of Educational Technology, 47(4), 694-
709. doi: 10.1111/bjet.12255

Depiction of issues
Search icon
Navigational issues (Difficulty in locating notebooks)

Section and Page buttons
Unconventional Word-processor
Paragraph Block

Appendices
Risk Assessment Form
Domain Impact Score Probability
Selection Bias Low Low
Attrition Bias High Low
Detection Bias High Medium
Reporting Bias Low Medium

Data consent form
Data collection form
Q1. Were you able to create account successfully? Yes or No
Q2. Were you able to create notebooks successfully? Yes or No
Q3. Were you able to create pages and sections successfully? Yes or No
Q4. Were you able to add text, images, graphics and other content in your Notes
successfully? Yes or No
Q5. Were you able to search for your content successfully? Yes or No
Q6. How do you rate the design and aesthetics of the application? Excellent, Good,
Average and Bad
Q7. If the application fixed the design issues you encountered, would you use it in the
future? Yes, No or N/A?
Q8. Did you encounter issues with your evaluation? Yes or No?

Full results
Gemma
Q1. Were you able to create account successfully? Yes
Q2. Were you able to create notebooks successfully? Yes
Q3. Were you able to create pages and sections successfully? Yes
Q4. Were you able to add text, images, graphics and other content in your Notes
successfully? Yes
Q5. Were you able to search for your content successfully? Yes
Q6. How do you rate the design and aesthetics of the application? Good
Q7. If the application fixed the design issues you encountered, would you use it in the
future? Yes
Q8. Did you encounter issues with your evaluation? Yes
Nathan
Q1. Were you able to create account successfully? Yes
Q2. Were you able to create notebooks successfully? Yes
Q3. Were you able to create pages and sections successfully? Yes
Q4. Were you able to add text, images, graphics and other content in your Notes
successfully? Yes
Q5. Were you able to search for your content successfully? Yes
Q6. How do you rate the design and aesthetics of the application? Excellent
Q7. If the application fixed the design issues you encountered, would you use it in the
future? No
Q8. Did you encounter issues with your evaluation? Yes