Applications of Artificial Intelligence in Ophthalmology: General Overview

Review Article
Applications of Artificial Intelligence in Ophthalmology:
General Overview
Wei Lu , 1 Yan Tong , 1 Yue Yu , 2 Yiqiao Xing , 1 Changzheng Chen , 1 and Yin Shen 1
1 Eye Center, Renmin Hospital of Wuhan University, Eye Institute of Wuhan University, Wuhan, Hubei, China
2 Hisee Medical Artificial Intelligent Lab, Wuhan University, Wuhan, Hubei, China
Correspondence should be addressed to Yin Shen; [e-mail]
Received 7 July 2018; Revised 6 October 2018; Accepted 17 October 2018; Published 19 November 2018
Academic Editor: Hiroshi Kunikata
Copyright © 2018 Wei Lu et al. -is is an open access article distributed under the Creative Commons Attribution License, which
permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
With the emergence of unmanned plane, autonomous vehicles, face recognition, and language processing, the artificial in-
telligence (AI) has remarkably revolutionized our lifestyle. Recent studies indicate that AI has astounding potential to perform
much better than human beings in some tasks, especially in the image recognition field. As the amount of image data in imaging
center of ophthalmology is increasing dramatically, analyzing and processing these data is in urgent need. AI has been tried to
apply to decipher medical data and has made extraordinary progress in intelligent diagnosis. In this paper, we presented the basic
workflow for building an AI model and systematically reviewed applications of AI in the diagnosis of eye diseases. Future work
should focus on setting up systematic AI platforms to diagnose general eye diseases based on multimodal data in the real world.
1. Introduction
As population aging has become a major demographic trend
around the world, patients suffering from eye diseases are
expected to increase steeply. Early detection and appropriate
treatment of eye diseases are of great significance to prevent
vision loss and promote living quality. Conventional di-
agnose methods are tremendously depend on physicians’
professional experience and knowledge, which lead to high
misdiagnosis rate and huge waste of medical data. Deep
integration of ophthalmology and artificial intelligence (AI)
has the potential to revolutionize current disease diagnose
pattern and generate a significant clinical impact.
Proposed in 1956 by Dartmouth scholar John McCarthy,
AI is a general term that “refers to hardware or software that
exhibits behavior which appears intelligent” [1]. -ough oc-
curred sixty years ago, it is until recently that the effectiveness
of AI has been highlighted because of the development of new
algorithms, specialized hardware, cloud-based services, and
big data. Machine learning (ML), occurred in 1980s, is a subset
of AI, and is defined as a set of methods that automatically
detect patterns in data and then incorporate this information
to predict future data under uncertain conditions. Deep
learning (DL), occurred in 2000s, is a burgeoning technology
of ML and has revolutionized the world of AI. -ese tech-
nologies power many aspects of modern society, such as
objects’ recognition in images, real-time languages’ trans-
lation, device manipulation via speech (such as Apple’s Siri,
Amazon Alexa, and Microsoft Cortana), and so on.
-e field of healthcare has been the forefront of the AI
application in recent years. Multiple studies have shown that
DL algorithms performed at a high level when applied to
breast histopathology analysis [2], skin cancer classification
[3], cardiovascular diseases’ risk prediction [4], and lung
cancer detection [5]. -ese impressive research studies inspire
numerous studies to apply AI in ophthalmology. Advanced
AI algorithms together with multiple accessible data sets, such
as EyePACS [6], Messidor [6], and Kaggle’s data set [7], can
make breakthroughs on various ophthalmological issues.
-e rapid rise in AI technology requires physicians and
computer scientists to have a good mutual understanding of
not only the technology but also the medical practice to
enhance medical care in the near future. Miguel Caixinha
and Sandrina Nunes introduced conventional machine
learning (CML) techniques and reviewed applications of
CML for diagnosis and monitoring of multimodal ocular
Hindawi
Journal of Ophthalmology
Volume 2018, Article ID 5278196, 15 pages
https://doi.org/10.1155/2018/5278196

disease, without the mention about DL [8]. Litjens et al. [9]
detailly introduced various DL methods for different tasks
and provided an overview of studies per application area,
whereas the “retina” section majorly focused on the fundus
images only. Lee et al. [10] introduced the AI development in
ophthalmology generally. Rahimy [11] focused on DL ap-
plications in the ophthalmology field, without the mention
about CML. Louis J. Catania and Ernst Nicolitz systemically
reviewed AI and robotic applications in multiple categories
of vision and eye care but mentioned little about AI di-
agnosis of retinal diseases [12].
In this review, we systematically reviewed the application
of AI (both CML and DL) in diagnosing ocular diseases,
including the four leading cause of adult blindness diabetic
retinopathy (DR), glaucoma, age-related macular degenera-
tion (AMD), and cataract. We also introduced the existing AI
methods, the ophthalmic imaging modalities, detailed steps
for building AI models, and evaluation metrics in AI di-
agnosis. We hope we can provide both ophthalmologists and
computer scientists a meaningful and comprehensive sum-
mary on AI applications in ophthalmology and facilitate
promising AI projects in the ophthalmology field.
2. AI Algorithms
As we mentioned above, ML is one subset of AI and includes
DL and CML (Figure 1(a)). -e defining characteristic of ML
algorithms is the quality of predictions improved with ex-
perience [13]. -e more data we provide (usually up to
a platform), the better the prediction model we can achieve.
Supervised learning and unsupervised learning are two
forms of ML. Supervised learning is to train a model from
already labeled training data, tunes the weightings of the
inputs to improve the accuracy of its predictions until they
are optimized, and then map test data sets as corresponding
outputs. It may expedite classification process and would be
useful for discriminating clinical outcomes. Unsupervised
learning is to train a model with unlabeled data (without
a human-labeled process), infers a function to describe
hidden structures that usually invisible to humans, and
could bring new discoveries, such as new encephalic region
relevant to Alzheimer’s disease [14] and new impact factors
of cardiovascular diseases beyond human’s recognition [4].
So far, methods adopted in most research studies are in
supervised form because the accuracy and efficacy are better
under supervised condition [15].
CML can get satisfactory outcome with small data sets, but
a cumbersome step to select specific visual features manually
prior to classification is indispensable [16]. -is selection can
result in a set of suboptimal features and overfitting (the
trained model is not generalized to other data except for the
training set), which limits CML algorithms’ application.
Existing CML algorithms used in AI diagnosis include
decision trees [17], random forests (RF) [18], support vector
machines (SVM) [19], Bayesian classifiers [20], k-nearest
neighbors [21], k-means [22], linear discriminant analysis
[23], and neural networks (NN) [24] (Table 1). Among them,
RF and SVM are the most commonly used CML technol-
ogies in the ophthalmology field [25] (Figures 1(b) and 1(c)).
DL, a burgeoning technology of ML, has the ability to
discover intricate structures in data sets without the need to
specify rules explicitly. A DL network is an NN with multiple
layers between the input and output layers (Figure 1(d)). It has
dramatically improved the state-of-the-art in image recogni-
tion [15]. When applied to image classification, a key difference
between DL and CML algorithms is how they select and
process image features. Features of input data are automatically
learned in an unsupervised way by DL algorithms, avoiding
manual segmenting and depicting lesions’ areas [15, 26].
However, large data set is needed to train a DL algorithm.
Transfer learning is to retrain an algorithm, which has already
been pretrained on millions of general images before, on
a specific data set. -is method allows the training of a highly
accurate model with a relatively small training data set [27].
DL algorithms are known as “black boxes.” -e networks
generate comprehensive and discriminative features that are
much too high dimensional to be accessible for human in-
terpretation. Little is known about how they analyze pattern
and make a decision at the image level [7]. Heatmaps can
show which pixels play a role in the image-level predictions. In
the medical field, the visualization highlighted highly possible
abnormal regions in the input image for future review and
analysis, potentially aiding real-time clinical validation of
automated diagnoses at the point of care. Existing methods of
DL include long-term and short-term memory [15], deep
Boltzmann machines [28], deep kernel machines [29], deep
recurrent neural networks [30], and convolutional neural
networks (CNN) [15]. Among them, the most used DL
method in the medical image recognition field is CNN. -e
CNN consists of multiple convolutional layers that extract
features and transform input images into hierarchical feature
maps: from simple features, such as edges and lines, to
complicated features, such as shapes and colors. It also in-
cludes layers that can merge semantically similar features into
one to reduce the dimensionality of the extracted features, and
layers that can combine these features and output a final
probability value for the class. Existing CNN architectures
used in the medical image recognition field include AlexNet
[31], VGG [32], ResNet [33], and GoogleNet [34–37](Table 2).
3. Building AI Models
Various imaging modalities have been used in AI diagnosis,
such as radiology images (X-ray, CT, and MRI) [38], elec-
trophysiological signal records (electrocardiograph [39] and
electroencephalogram [40]), visible wavelength images (der-
moscopy images and biopsy images [3]), ultrasound images
[41], angiography images [42], and so on. We introduce the
ophthalmic imaging modalities in AI diagnosis in Table 3.
-e steps for building an AI model include pre-
processing image data, train, validate and test the model, and
evaluate the trained model’s performance.
3.1. Data Preprocessing. In order to increase AI prediction
efficiency, raw data need to be preprocessed. -e preprocessed
work includes the following [8, 43]: (1) noise reduction: noise
reduction needs to be performed in almost all relevant re-
search. Denoising can promote the quality of data set and
2 Journal of Ophthalmology

Table 1: Introduction of existing CML techniques in the AI medical field.
Classifiers Principles
Decision trees (i) Tree-like structure
(ii) Solve classification and regression problems based on rules to binary split data
Random forests (i) Ensemble a multitude of decision trees for classification
(ii) -e ultimate prediction is made by majority voting
Support vector
machines
Build a hyperplane that separates the positive and negative examples as wide as possible to minimize
the separation error
Bayesian classifiers
(i) Based on the probability approach
(ii) Assign a new sample to the category with maximum posterior probability, depending on the given prior
probability, cost function, and category conditional density
k-nearest neighbors Search for k-nearest training instances and classify a new instance into the most frequent class of these k instances
k-means Partition n samples into k clusters in which each sample belongs to the cluster with the nearest mean
Linear discriminant
analysis (i) Create predictive functions that maximize the discrimination between previously established categories
Neural networks
(i) Consists of a collection of connected units, which can process signals
(ii) Connections between them can transmit a signal to another
(iii) Units are organized in layers
(iv) Signals travel from the input layer to the output layerMachine learningDeep learningTraditional machinelearningExpert systemRoboticsAINatural language processing...
(a)Tree 1 Tree nTree 2SampleMajority votingClassification... (b)SVM
(c)InputlayerOutputlayerHidden layer.................. (d)
Figure 1: Introduction of AI algorithms. (a) -e relationship among AI, ML, and DL. (b) -e workflow of a RF. (c) -e principle of an SVM.
(d) -e schematic diagram of a typical deep neural network.
Table 2: Concise introduction of CNN algorithms used in AI diagnosis.
Models Layers Top-5 error∗ (%) ILSVRC#
AlexNet (2012) 8 layers 15.3 2012
VGG (2014) 19 layers 7.3 2014
ResNet-152 (2015) 152 layers 3.57 2015
ResNet-101 101 layers 4.6 —
ResNet-50 50 layers 5.25 —
ResNet-34 34 layers 5.6 —
GoogleNet/inception v1 (2014) [34] 22 layers 6.7 2014
Inception v2 (2015) [35] 33 layers 4.8 —
Inception v3 (2015) [36] 47 layers 3.5 —
Inception v4 (2016) [37] 77 layers 3.08 —
∗-e fraction of test images for which the correct label is not among the five labels considered most probable by the algorithm. -e lower the top-5 error, the
better the classifier perform. #
ImageNet large-scale visual recognition challenge.
Journal of Ophthalmology 3

optimize learning process. (2) Data integration and normal-
ization: data collected from different sources should be in-
tegrated and adjusted to a common scale. (3) Feature selection
and extraction: the most relevant features are usually selected
and extracted to improve the learning process performance.
3.2. Training, Validation, and Test. To achieve a good per-
formance, the data set is randomly partitioned into two
independent subsets, one is for modeling and the other is for
testing. -e data in the former sets will be partitioned again
into training set and validation set in most cases. -e
training set is used to fit the parameters of a model. -e
validation set is used to estimate how well the model had
been trained and tune the parameters or to compare the
performances of the prediction algorithms achieved based
on the training set. -e test set is used to evaluate the final
performance of the trained model (Figure 2(a)).
Cross-validation methods have been widely used to esti-
mate and optimize algorithms [44]. -e most adopted cross-
validation is “K-fold cross-validation.” It is an effective method
to avoid overfitting and underfitting. All data are equally
divided into K subsets, 1 for validation and K − 1 for training.
-is process will repeat K times, and average metrics are used
to evaluate the trained model (Figure 2(b)). Fivefold cross-
validation and 10-fold cross-validation are most commonly
used [44].
3.3. Evaluation. Receiver operating characteristic curve
(ROC) is a useful tool to depict algorithms’ performance. It is
created by plotting the detection probability for each algo-
rithm across a continuum of threshold. For each threshold,
the sensitivity and the false positive rate (1 − specificity) are
plotted against each other. -e area under receiver operating
characteristic curves (AUC) is the most used evaluation
metrics for quantitative assessment of a model in AI di-
agnosis. -e AUCs of effective models range from 0.5 to 1; the
higher the value of AUC, the better the performance of the
model [45]. Table 4 provides introduction of other metrics to
evaluate the performance of a model.
4. AI Application in Ophthalmology
Two hundred forty-three articles of AI application in di-
agnosing ophthalmological diseases have been published
(search by PubMed, Sep 20, 2018). Among them, the most
intensively studied are DR, glaucoma, AMD, and cataract
(Figure 3(a)). Figure 3(b) shows the breakdown of the papers
of these four diseases in year of publication.
4.1. Diabetic Retinopathy. Diabetes affects more than 415
million people worldwide, meaning 1 in every 11 adults is
affected [46]. DR, a chronic diabetic complication, is
a vasculopathy that affects one-third of diabetic patients and
can lead to irreversible blindness [47]. Automated tech-
niques for DR diagnosis have been explored to improve the
management of patients with DR and alleviate social burden.
AI was used to predict DR risk and DR progression among
diabetic patients to combat with this worldwide disease
[48, 49].
-e specific abnormalities such as macular edema
[50–53], exudates [53], cotton-wool [54], microaneurysms
[55, 56], and neovascularization on optic disk [57] can be
detected by CML. Based on these hallmarks, the early di-
agnose of DR in an automated fashion has been explored
[58]. Additionally, a system focused on timely and effectively
proliferative DR (PDR) detection has been developed to
ensure immediate attention and intervention [59, 60].
Gulshan et al. were the first to report the application of
DL for DR identification [6]. -ey used large fundus image
data sets to train a deep CNN (DCNN) in a supervised
manner. -ey showed that the method based on DL tech-
niques had very high sensitivity and specificity, and the AUC
came up to 0.99 for detecting referable DR [61]. In the past
two years, a number of DL models with impressive per-
formance have been developed for the automated detection
of DR [46, 62, 63]. Additionally, some studies applied DL to
automatically stage DR through fundus images [62–65],
making up the deficiency of Gulshan’s study that they only
detected referable DR but did not provide comparable data
on sight-threatening DR or other DR stages.
-e majority of aforementioned studies focused mainly
on the analysis of fundus photographs. -ere were some other
imaging modalities used to build models for DR. ElTanboly
et al. developed a DL-based computer-aided system to detect
DR through 52 optical coherence tomography (OCT) images,
achieving an AUC of 0.98 [66]. Despite the good outcomes in
the cross-validation process, the system needs to be further
validated in larger patient cohorts. A computer-aided di-
agnostic (CAD) system based on CML algorithms using
optical coherence tomography angiography (OCTA) images
Table 3: -e ophthalmic imaging modalities in AI diagnosis.
Imaging modalities Image features Applications
Fundus image Show a magnified and subtle view of the surface of the
retina Retinal diseases diagnose
Optical coherence
tomography
Show micrometer-resolution, cross-sectional images
of the retina Retinal diseases diagnose
Ocular ultrasound B-scan Show a rough cross-sectional view of the eye and the
orbit
Evaluate the condition of lens, vitreous,
retina, and tumor
Slit-lamp image Provides a stereoscopic magnified view of the anterior
segment in detail Anterior segment diseases diagnose
Visual field Show the size and shape of field-of-view To find disorders of the visual signal processing
system that includes the retina, optic nerve, and brain
4 Journal of Ophthalmology