Monitoring Rehabilitation Progress using Augmented Reality Games and Cloud Computing

GRADUATE ENGINEERING PROJECT: MONITORING REHABILITATION PROGRESS ON
PATIENTS BY USING AUGMENTED REALITY GAMES AND CLOUD COMPUTING
(CREHAB)

Table of Contents
Abstract.......................................................................................................................................................4
Introduction.................................................................................................................................................5
Literature Review of Related Work.............................................................................................................5
Methodology................................................................................................................................................7
Cloud Based Web Platform for the System.................................................................................................7
General Architecture...............................................................................................................................7
Website server........................................................................................................................................8
Client Side..............................................................................................................................................8
Web Socket.............................................................................................................................................9
User Interface.........................................................................................................................................9
Patient Interface....................................................................................................................................10
Physiotherapist User Interface..............................................................................................................11
Database...............................................................................................................................................11
Assessment Module..............................................................................................................................11
Representation of Features....................................................................................................................12
Trained HMMs.....................................................................................................................................13
Experimentation.........................................................................................................................................14
Setup.....................................................................................................................................................14
Training Models...................................................................................................................................15
Results.......................................................................................................................................................15
Conditioning.........................................................................................................................................19
Discussions................................................................................................................................................19
Conclusions...............................................................................................................................................20
Recommendations................................................................................................................................20
References.................................................................................................................................................21

Abstract
Advances in technology and computing provide new approaches on how physical rehabilitation can be
achieved. In that sense, the present investigation proposes a savvy, online stage to advance the correction
of patients after hip substitution medical procedure. This paper proposes a novel method for remote
physiotherapy rehabilitation and develops the CRehab cloud based application. A probabilistic
methodology dependent on the advancement and preparing of ten Hidden Markov Models (HMMs) is
utilized to separate progressively the primary issues in the execution of the remedial activities. The goal
was to use affordable technology and offer patients real time assessment of their performance of the
exercise. An experimental method was used to test the system, first using healthy participants then using
two patients with hip problems and one healthy one and the results show the CRehab system is effective
in ensuring the right exercises are performed and can help provide remote rehabilitation at low costs,
efficiently, and with direct interaction.

Introduction
With rising medical costs, the need to ensure patients have the best outcomes, and the need to judiciously
allocate increasingly limited resources, new trends have emerged in medical care, such as using Internet
of Things and remote patient monitoring. Among the new medical trends is the use of remote therapy in
rehabilitation medicine: this concept entails enabling patients to undertake a part of their rehabilitation at
their premises (such as at home) and communicate with their care givers on the evolution of their process
of recovery through the Internet. The use of this technology approach is justified based on medical
reasoning; it will result in improved recovery as more physical activities can be undertaken by the patient
frequently. It is also justified by economic reasons as the number of visits to the therapist and time spent
at the medical facility is reduced; ethics is another justification for the use of the system because it results
in greater democratization of health as the patient is more empowered and can make choices. Mobility is
another justification because the patient travels less to the therapist. However, despite its theoretical
benefits, the use of the remote therapy system for rehabilitation medicine raises another pertinent issue;
the patient will undertake exercises without being supervised and guided by a qualified therapist and this
raises concerns about how correct the movements used by the patient would be.
This graduate engineering project explores available options and proposes the use of a web based
platform to be utilized for remote/ Tele-physical rehabilitation of patients incorporating the use of cloud
computing and augmented reality. This is a novel approach that will ensure the therapist can monitor the
patient as well as ensures they use the correct physical movements to ensure fast and complete recovery
and enjoy the envisaged benefits of such a system (to be called CRehab). The proposed system will be
developed and tested using patients with mobility problems as a result of dislocation or problems with
their hip bone and require a functional and effective program of rehabilitation to recover their full
mobility, stability, and strength. The CRehab system will operate based on two fundamental conditions to
achieve the objectives of remote rehabilitation: first, it uses low cost devices for capturing motion, and
secondly, it uses a cloud platform with integrated artificial intelligent module for augmented reality. This
is aimed at evaluating how correct the motions used by the patient during physical activity are and give
real time feedback to a [patient as well as the physiotherapist. The [paper is organized into the following
sections, following the abstract and this brief introduction; a related works section in which a detailed
literature review of tele-rehabilitation systems is undertaken. This is followed by a section describes the
general cloud based front and back-end platform of the system. This is followed by a section that
describes the module for assessing the CRehab system. The following section is about tests done to
evaluate and prove the functionality and reliability of the CRehab system. The final section discusses the
results of the tests done, conclusions drawn, and recommendations for further studies and how the system
can be improved, especially the AR (augmented reality) AI algorithm.
Literature Review of Related Work
Tele-rehabilitation refers to rehabilitation services delivery through telecommunications networks,
including the Internet and most types generally fall into two categories; clinical therapy (the focus of this
study) and clinical assessment. Tele-rehabilitation and its use in medical/ clinical care have been explored
in diverse fields including audiology, neuropsychology, physical therapy, speech-language pathology, and
occupational therapy. People unable to travel to a medical center can benefit greatly from Tele-
rehabilitation and enables professionals in the field to consult with patients remotely (Kumar and Cohn,
2013). Advances in new computer science technologies as well as advanced telemedicine devices have
spawned an increase in the use of Telemedicine applications with various rehabilitation treatments using
this method being reported. Tele-rehabilitation, despite being relatively new, has grown tremendously
especially in the developed world and its benefits include reduced medical costs and access to quality
rehabilitation care for patients in remote regions. However, it still has its demerits and drawbacks,
including skepticism by patients because of interacting remotely with their physicians and this should not

be underestimated (Peretti et al., 2017). Tele-rehabilitation systems supporting physical therapy anywhere
are attractive since they can help lower health costs and ensure patients enjoy a better quality of life. One
of the Tele-rehabilitation systems that have been put forth is the Kinect-based Tele-rehabilitation system,
christened KiReS (Simonsen et al., 2014). Apart from providing remote Tele-rehabilitation, the KiReS
has two additional functionalities; the knowledge extraction functionality and the Tele-immersion
functionality. The extraction functionality deals with knowledge on patients’ physical therapy records as
well as treatment protocols to select adequate patient rehabilitation exercises. The Tele-immersion
functionality offers an effective, convenient, and user friendly experience for its users during
rehabilitation sessions via a real-time two-way multimedia communication. The KiReS was tested and
showed success for patients with total hip replacement and shoulder problems (Anton et al., 2018).
Tele-rehabilitation is a promising application in assisting survivors of strokes to reco0ver and regain fully
function of their organs. It can be used remotely for supervised administration of occupational, physical,
speech, and other forms of therapy to improve the cognitive, neuropsychiatric, and motor deficits suffered
by stroke patients. According to Sarfo et al. (2018), tele-rehabilitation has been shown, based on
evidence, to have better or equal salutary effects on the higher cortical, mood, and motor disorders when
compared to conventional one-on-one therapy. As the global population continues aging with older
people, chronic diseases also increase and their management continues to place tremendous pressure on
healthcare delivery. A further review of existing literature by Chen et al. (2015) shows that there is
moderate, albeit limited evidence that tele-rehabilitation using various approaches has effects that are
equivalent to conventional rehabilitation programs in improving stroke survivors’ abilities to engage in
daily living activities and it is recommended that more studies and approaches be used in this field to
improve outcomes. Technological advancements provide promising alternatives to augment health care
delivery for such populations and among the newest methods is the use of Tele-rehabilitation Robotics.
This approach combines Tele-healthcare and robot assisted rehabilitation features to offer rehabilitation
services remotely. The effectiveness of this approach has been reported in various studies with
significant statistical and clinical improvements in upper and lower limb function in patients. Further,
feasibility studies have documented significant improvements in patients’ high fidelity kinematics;
however, barriers exist even as Tele-rehabilitation robotics continues to show promise and these will need
addressing to ensure greater acceptance of Tele-rehabilitation (Housley, Fitzgerald and Butler, 2018).
Across most of Northern Europe, the population is aging and coupled with social and economic
challenges, the health care systems in these regions are at risk of being overwhelmed by high costs as
older people disproportionately require and use rehabilitation services. One solution, especially for those
in rural areas is to change the delivery mode to incorporate new technologies and approaches that have
been implemented include the SENDoc (Smart Sensor Devices for Rehab and Connected Health) which
was implemented within the Northern Periphery and Arctic Program. The SENDoc program entails using
wearable sensors and sensor systems in communities with elderly people. The benefits of the approach
include reduced costs of rehabilitation, enhanced outcomes, and better management of patients. However,
the approach is not without its problems that include complexity of the system, design and safety issues.
The program has been noted for its key advantage which is the possibility of changing patient behavior,
enabling patients live independently and empowering patient active participation in their wellbeing. The
system also implies less travel as there is very little face-to-face interactions with care givers (Esquivel et
al., 2018). This article demonstrates that emote rehabilitation is a promising technology for improving
rehabilitation services remotely with significant advantages and some drawbacks as well.
According to Banerji et al. (2016), there is a shortage of technology and manpower in rehabilitation on a
global scale to meet the rehabilitation needs of some five million new patients every year left disabled
with stroke. One approach to solving this problem is the use of Neuroplasticity as the main method of
attaining significant motor recovery. However, most devices used in rehabilitation are either limited to
mechanical repetitive practice movements at the limb level or focus on cognitive tasks only. This can lead