Handwritten Digit Classification and Sample Complexity using Kernel Perceptron

This article explains the implementation of handwritten digit classification and sample complexity using kernel perceptron. It covers topics such as supervised learning, kernel perceptron, cross validation, confusion matrix, Gaussian kernel, polynomial kernel, sparse learning, sample complexity, bias variance trade off, linear regression and 1-nearest neighbour algorithm. The article is relevant for computer science students and provides all the necessary information for the course.

Computer science

Contents
Introduction...........................................................................................................................................1
Supervised Learning..............................................................................................................................1
Kernel Perceptron..................................................................................................................................1
Adding Kernel.......................................................................................................................................1
Training and Testing the Kernel Perceptron..........................................................................................2
Generalizing to K Classes......................................................................................................................2
Cross validation.....................................................................................................................................2
Confusion Matrix..................................................................................................................................3
Gaussian Kernel....................................................................................................................................3
Polynomial Kernel.................................................................................................................................3
Sparse Learning.....................................................................................................................................4
Sample Complexity...............................................................................................................................4
Bias Variance Trade Off........................................................................................................................4
Linear Regression..................................................................................................................................4
1-Nearest Neighbour Algorithm............................................................................................................4
Conclusion.............................................................................................................................................5
References.............................................................................................................................................5

Introduction
In this project to develop the handwritten digit classification and sample complexity. We
are using the python code to develop the handwritten digit classification. The input sources
such as paper document, photographs, touch screens and other devices. The text image
converts to letter by using python code. The handwritten digit also called as kernel
perceptron.Perceptron is an algorithm and it is one of the supervised learning. It has two type
of classifier. One is linear classifier and another one is binary classifier. The binary classifier
using to classify the binary in the set. It represent by vector of number. The second task to
satisfy the sample complexity. It based on linear regression and winnow.
Supervised Learning
The supervised learning is the machine learning and it based on output and input. We
import the training dataset and testing the dataset. In this algorithm analyze the data and
produce the inferred function (CAI, 2010). The training dataset needs to be representative of
real world use of the function. We import the data by using python code and it makes map or
graph related to graph. In this project we implement the kernel perceptorn using python. We
are using the supervised learning algorithm for implement the handwritten digit classification
and sample complexity.
Kernel Perceptron
The task is quasi realistic and working with the large dataset. The text image converts
into latter using the python. First step to import the training dataset. The perceptron
generalize in two ways. The first way to use the kernel function for generalize the
perceptron.The second way generalize the perceptron into a majority network perceptron.It
has two classes and it separate k classes.
Adding Kernel
We adding kernel one by one and it allows to map. It is linear function and it performs
the ploynominal.The kernel mainly using the operating system. It is center part of operating
system.It manages the operating system and hardware also. The micro kernel and monolithic
kernel is kernel types.

Training and Testing the Kernel Perceptron
We are using training dataset for implement the handwritten digit classification.The
given input xt,yt and initialize value α0 = 0.The prediction value
^yt =sin (ωt ( xt ¿))=sin(∑
i=0
t−1
αi ¿ K ( xi , xt )) ¿ ¿
We update the equation for single function and it added the value. It repeated the value and it
called as cycle. The cycle known as epoch. The epoch classifies one by one and it generalizes
the value. First we check the first epoch. If the first epoch table correct then we add the
second epoch. For example we have the 40 element in the epoch the epoch separate three
type of epoch. The element can be separate like {(x1,x1),(x2,x2)….(x40,x40)} and the
dataset is {(x1,x1),(x2,x2)….(x40,x40)} first epoch and {(x41,x41)….(x80,x80)} second
epoch. The epoch is equal to epoch of m.x1=x40=x80=xm.We training the dataset and it
divided to test size. The predict value applied after divided the test size. Finally update the
entire value.
Generalizing to K Classes
We use research method to generalizing the k classes. In this method return the vector
value. We import the large dataset and it generalizes the two classifier function. The python
code work on large dataset. We take the d value. In this task the given value 257 values and it
separate 256 values for 16 x 16 matrixes. And remain value denoted -1 or 1.The result
depending the d value. The value is 1 to 7.So the table value matrix 2 x 7.We split the dataset
and it perform the cross validation.
Cross validation
Cross validation is called as rotation estimation. It generalizes the independent dataset
and analyzes the dataset. The given dataset is known dataset and validation dataset is
unknown dataset. The validation dataset check the dataset and test the dataset. If any error
occur in the dataset then it change the dataset value. If the dataset is correct then it
generalizes the dataset. The cross validation involves the partioning.Partioining means dataset
divided the small datasets. First analyze the dataset and it reduces the variability. The test
error denoted as std and mean value denoted as dt std.

Confusion Matrix
To create matrix for given training dataset. It is classification model. It has positive and
negative model. The positive values contains true positive and false positive. The negative
value contains false negative and true negative. First classify the rate and accuracy. Consider
total number of positive values and small number of FN.The recall defined as the total
number of positive values divided by small number of FN. Precision has low precision and
high precision. Consider total number of positive values and small number of FP.The
precision defined as divided the total number of positive value and small number of FN.
Finally we measure the F-measure. The F-measure value based on recall and precision. We
take the sum of precision and recall value and multiply of precision and recall value. Both
value divided by the two that is called as F measure.
Gaussian Kernel
The Gaussian model measures the input value and its prototype vector. The input value
and prototype is equal to 1.It has one dimensional input value and two dimensional input
value. Consider the equation k ( p , q ) =e−c∨|p −q|∨2.C is width of the kernel and compute the
cross validation for given equation. The Gaussian model based on Euclidean distance. The
Euclidean distance calculates using input value and prototype vector. The Euclidean distance
looks like v shape and the distance between x and 0 for one dimensional input. The two
dimensional input has x value and y value. The two dimensional Euclidean distance between
value(x,y) and (0,0). The Gaussian method based on variance. The variance curve looks like
bell curve. The variance equation is φ ( x )=e−β∨|x−μ|∨2.Consider the sigma and beta. The
sigma denoted as parameter and beta is input values.
Polynomial Kernel
Polynomial is non linear models and it support vector machines. The polynomial kernel
takes only original values. The polynomial kernel is equal to polynomial regression.

Sparse Learning
It perform machine learning algorithm and it calculate accurate predictions. It is using for
image processing and signal processing (CHENG, 2016). Sparse learning is unsupervised
learning and it represents the data efficiency. The sparse learning data called as sparse data.
Consider the x and y matrix values. The x is4 x 4 matrix and y is 1 x 1 matrixes. In this task
we compute the sample complexity.
Sample Complexity
Sample complexity is one of the machine learning algorithms and it represents the
number of training datasets. There are two different variants in sample complexity. The
variants are worst case complexity and best case complexity (Feldman and Xiao, 2015). The
sample complexity implement by python code. Take m and n values. The n denoted as x axis
values and m denoted as y axis values. The n values are 20….100 and m values are 10……
60.The graph based on n and m value using python.
Bias Variance Trade Off
It is one of the estimate theory and it set the predict values. Bias is used for error
correction and it based on target output values. It analyzes the data in given dataset. It sum of
bias, variance and quantity. Consider the m and n value from sample complexity. If the n
value is infinite then the m value increase. If the n values zero then the m values decrease.
Linear Regression
Linear Regression is a linear approach and relation between independent variable and
dependent variable. Dependent variable called as scalar response and independent variable
called as explanatory variables (Christie, 2011). One or more variable present in the dataset
that is called as multiple linear regressions. The linear regression model is called as linear
model. It used to observe the dataset and it predict the value.

1-Nearest Neighbour Algorithm
It allows the sample complexity model and it generalizes the data. In this algorithm
related to nearest neighbour regression model. It calculates the minimum value in the dataset.
The k nearest algorithm is one of the nearest neighbour algorithms. K nearest neighbour
algorithm is supervised algorithm. Compare with k nearest neighbour algorithm it is very
simple and easily calculate the minimum value.
Conclusion
In this project to develop the handwritten digit classification and sample complexity. The
text image converts into letter. In using python code for convert text image to letter. The
input sources are photo, document or any devices. In this task the input source convert to
letter. It is called as kernel perceptron. First adding the kernel for create map. The kernel
divides to two type. The types are one dimensional kernel and two dimensional kernels. The
k classes are using to implement the map. The perceptron to use the kernel function. It takes
positive integer and negative integer. The positive integer and negative integer. The positive
integer controlling the dimensional of polynomial. Test the kernel perceptron after adding
kernel. It based on input and initialize values. Third step generalize the k classes. This step
returns the vector value. The next step crate confusion matrix computes the cross validation
using python. The second task to solve the sample complexity problem using python code. It
take x and y matrix value and plot the graph based on m and n value. Finally compute the
nearest neighbour algorithm and find the minimum value successfully.

References
CAI, Q. (2010). Face recognition algorithm based on supervised neighborhood preserving
embedding. Journal of Computer Applications, 29(12), pp.3349-3351.
CHENG, H. (2016). Sparse representation, modeling and learning in visual recognition.
[Place of publication not identified]: SPRINGER LONDON LTD.
Christie, M. (2011). Simplicity, complexity, and modelling. Chichester, West Sussex, U.K.:
Wiley.
Feldman, V. and Xiao, D. (2015). Sample Complexity Bounds on Differentially Private
Learning via Communication Complexity. SIAM Journal on Computing, 44(6), pp.1740-
1764.