Introduction to Machine Learning Assignment 2022

Introduction to Machine Learning
Abstract
Implementing Machine learning algorithms can used for classification as well as regression,
these machine learning algorithms such as K-means [1] and radial basis function [2], both are robust
algorithms used in machine learning. Steps followed in Machine learning process initiated by
observing the past data provided, this process can be done by both machine itself or operated by
human with some developed instructions. Machine learning algorithms are generally categorized
as supervised learning in which programs are developed to train on data some predefined training
result, in case of unsupervised learning we only provide the data and we develop programs in
such a way that machine can learn the pattern of provided data. In unsupervised method of
learning we are going to implement nonparametric classification or regression algorithm for
work the nonparametric method we are going to implement are RBF neural network k means, on
the provided data. classification and regression both to be done using k-nearest neighbor and
radial basis function neural network. We have performed Cross Validation Performance (MSE)
for these both algorithms and for each of the data set to understand the impact of the all
algorithms on the specific data set.
Problem statement
The main task of assessment to learn and implement some a nonparametric classification or
regression algorithm. Implementing RBF neural network [4], k means [1], on the provided data.
classification and regression both to be done using k-nearest neighbor and radial basis function
neural network. We will make use of k means implementation to help out in the implementation
of the RBF network [3]. We have used two algorithm RBF, K-means. We have performed Cross
Validation Performance (MSE) [6] for both algorithm for classification as well as regression.
We are used activation function (logistic activation function) [3], its equation is given –
y= 1
1+e−u
Where represents the weighted sum, further we found the derivative of the logistic function as
y(1−y). ∇Err = (t−y) y(1−y) x represents the gradient of the error, where x is considered as the
input to the logistic node and t is the target value.
u = w0+∑ 𝑤𝑖𝑥𝑖

Brief description of your experimental approach
We will use 5-fold cross validation [8] of all the three algorithm ,firstly we have divided dataset
into 5 parts which have equal data size ,we have divided dataset in such away as when we have
100 dataset points then they are divided as one third as one class and two third as another class.
Then we have applied all algorithm on provided dataset. Now we will use 80% data for training
and 20% for testing. Now we will implement all 2 algorithms.
RBF
RBNN [3] configured as input, hidden, and output layer. In RBFNN important point is that it will
have exactly one hidden layer, it is one of the limitations of it. We later call it as feature vector
We call this hidden layer as feature vector. A prototype vector is stored in each Rbf neutron, a
prototype vector which is stored in each neutron is one of the vectors from training set. Rbf
neuron performs the task which compares the input vector to prototype. A value between 0 and 1
which is a measure of similarity. If the input vector is equal to the vector prototype, then the
output of that RBF neuron will be 1. As the difference between the input and prototype rises, the
response downfall towards 0.
KNN (K-nearest neighbor)
Common name of K-nearest neighbor is KNN [1]. This is one of the non-parametric algorithms
which is implemented for problems which requires classification as well as regression. But the
fact is the nearest neighbor algorithm is mostly used for classification problems. Moreover,
Nearest neighbor algorithm works in such a way that the all the cases of input data are stored and
aiming at classification which is based on similarity measures. Classification of data points are
done on base of how neighbors are classified.
Steps involved in K-means-:
 First step is selection a random number of neighbors with random centroid
 Assign each point to a cluster, have the nearest centroid.
 Recalculate the centroid of each cluster.
Equation used as -
Ck is known as Kth cluster,
d = Distance between cluster and centroid