Convolution Neural Networks Research 2022

1
Introduction
In machine learning, the application of multilayer neural networks more so the Convolution
Neural Networks (CNN) have been playing a critical role in the development and usability of
pattern recognition systems (as presented in the research paper). Ideally, pattern recognition can
be defined as, “...automated recognition of patterns and regularities in data” (Bishop, 2006). In
their paper on, “Gradient-Based Learning Applied to Document Recognition”, (LeCun, et al.,
1998) posit that it is entirely possible to develop improved pattern recognition systems by having
more reliance on automatic learning instead of manually designed heuristics. Originally, the
paper by (LeCun, et al., 1998), explores several multi-layer neural networks including the CNN
which is the main system reviewed in this paper on their application on pattern processing. On
the very basic, CNN are categorized as an application of machine learning (ML) which is applied
to neural networks (LeCun, et al., 1998).
In practice, the functionality of CNNs lies in their ability to take in, “...an input image, assign
importance (learnable weights and biases) to various aspects/objects in the image and be able to
differentiate one from the other” (Saha, 2018). Different from the likes of ELI5 proposed by
(Saha, 2018), Yann LeCun, Leon Bottou, Yosuha Bengio and Patrick Haffner defined then new
LeNet 5 which is a CNN pattern recognition system. Convolutional Neural Networks form the
basis of the time’s recognition systems and today’s deep learning-based computer vision
(Pechyonkin, 2018).
Such networks have premises in three ideas i.e.: local receptive fields, shared weights as well
as spacial subsampling. The main difference between the time before CNN and after is that
before CNNs, pattern recognition had major dependencies on feature engineering conducted by
hand after which ML models were applied on the features so as to conduct feature classification.

2
Objectively, LeNet 5 is designed in such a way that it automates the feature engineering and aid
in learning from raw data. Primarily, a convolution in cases such as ConvNet are used in the
extraction of features from any given data/image. Equally, the use of Min-Max integrates the
features which have been learnt by the model such that they have, “...minimum compactness for
each object manifold and the maximum margin between different object manifolds” (Shi, et al.,
2015).
The paper argues that, among the key strengths of multilayer networks which have been
adequately trained using gradient descent include the ability of the models to learn despite the
size of data, pitches them as the best systems of use in image recognition problems. This strength
however is an impediment to the model’s implementation speed hence a con when viewed in
light of other systems which perform relatively faster (Mnih, 2015).
Innovation
Application
By the time of publishing the paper, CNN had not been implemented widely since feature
engineering and extraction was widely being conducted by hand which affected the fields in
which deep-learning can be applied to. However, after the posited functionality of the networks
by (LeCun, et al., 1998), CNN models have extensive applications including: image processing
systems, speech processing systems such as text classification and NLP, state of the art AI
systems, “...virtual assistants, and self-driving cars” (Szegedy, et al., 2015). The incremental
trend in application of Convolutional Neural Networks (LeNet) in various fields requiring deep
learning, proves that CNN are better systems in pattern recognition compared to hand-crafted
feature extraction.

3
Contribution of LeNet-5 to CNN.
Before the variant of CNN model introduced by the paper, there existed other CNN systems
such as AlexNet, VGG, GoogLeNet, ResNet, etcetera. However, the LeNet-5 CNN architecture
comprises, “...two sets of convolutional and average pooling layers, followed by a flattening
convolutional layer, then two fully-connected layers and finally a softmax classifier” which
totals to 5 convolutional layers. The original paper is quite long. However, the architecture of
LeNet-5 is proposed in section II. B, where the network is defined as entailing up to 7 layers
having input grayscale images each with a 32 by 32 figure. In the architecture’s 5 convolutional
layers, there is a pattern which is designed such that it follows the following schema:
Figure 1: the original architecture of LeNet-5 for handwritten character recognition as presented by (Brownlee, 2019)
Given the nature of the LeNet-5 Convolutional Neural Network, the LeNet-5 pattern has been
widely adopted in the designing and usage of convolutional neural networks. Therefore, elements
it is possible to find an element of LeNet-5 in various CNN systems.
Basis for future work
In particular, LeNet CNN after the paper proved that CNNs are better methods of pattern
recognition paving way to other similar systems that built on the layer architecture of LeNet-5.