MATLAB Implementation of SVM Classifier - EECS 152B Winter 2019

This assignment focuses on designing a linear Support Vector Machine (SVM) classifier using MATLAB's quadratic programming tool. The goal is to classify d-dimensional feature vectors into two classes (+1 or -1) based on supervised learning. The solution involves implementing a soft-margin SVM model to solve a constrained optimization problem with slack variables. The MATLAB function 'quadprogsoftsvm' is used to determine the optimal classifier by finding the weights (w), bias (b), and error (e) for different training datasets (x1, x2, x3, x4) provided in 'hw5data.mat'. The assignment also includes plotting the classifier boundary and analyzing the impact of different 'c' values on the classification results. The results and surface plots are generated to visualize the data separation achieved by the SVM classifier.

Running head: ASSIGNMENT 5
Assignment 5
Name of the Student
Name of the University
Author Note

1Assignment 5
Assignment Introduction:
In this particular assignment the supervised learning method is used for designing a linear
support vector machine which can classify the vectors the d-dimensional feature vectors in
any one of the two classes given by y = +1 or -1. The function of the optimal binary classifier
is given by,
f(x) = sign(wTx + b)
The 4 different training data sets are loaded in MATLAB from hw5data.mat. The labels of
the dataset are x1, x2, x3 and x4.
1.
The given constraint optimization problem is
Min ( 1
2 zT Qz + c∑
i=1
N
εi)
Constraints:
yi ( wT xi+b ) ≥1−εi for all i
ε i ≥ 0
Here,
∈= [ ∈1 ∈2 … ..∈ N ] T
z = [ w1 w2 … . wd b ]T
Now, quadprog in MATLAB is used to solve the problem. The quadprog
objective function in standard form is given by,

2Assignment 5
( 1
2 xT Hx+ f T x)
x
min
such that,
A*x <= b, Aeq∗x=beq, lb≤ x ≤ ub
This problem is a Soft-margin SVM model. The MATLAB function which
solves the problem is shown below.
MATLAB function:
function [w,b,e] = quadprogsoftsvm(data,yvals,c)
rows = size(data,1);
cols = size(data,2);
H = diag([ones(1, cols), zeros(1, rows + 1)]);
f = [zeros(1,cols+1) c*ones(1,rows)]';
p = diag(yvals) * data;
A = -[p yvals eye(rows)];
B = -ones(rows,1);
lb = [-inf * ones(cols+1,1) ;zeros(rows,1)];
z = quadprog(H,f,A,B,[],[],lb);
w = z(1:cols,:);
b = z(cols+1:cols+1,:);
e = z(cols+2:cols+rows+1,:);

3Assignment 5
end
2.
Now, the above MATLAB function is applied to primal form of classifier on the four training
data x1,x2, x3 and x4 and this outputs the w, b and e(error) from original y values. Here, the
value of constant c = 10 is considered.
Input and output for training data x1:
load('hw5data.mat')
>> c=10;
>> data = x1; yvals = x1(:,3);
>> [w,b,e] = quadprogsoftsvm(data,yvals,c)
Minimum found that satisfies the constraints.
Optimization completed because the objective function is non-decreasing in
feasible directions, to within the default value of the optimality tolerance,
and constraints are satisfied to within the default value of the constraint tolerance.
<stopping criteria details>
w =
-0.2062
0.0607
0.9515

4Assignment 5
b =
0.4720
e =
1.0e-11 *
0.4566
0.4609
0.4610
0.4576
0.4584
0.4561
0.4733
0.4714
0.4584

5Assignment 5
0.4889
0.4674
0.4713
0.4481
0.4591
0.4706
0.7002
0.4259
0.4578
0.3341
0.4572
Input and output for training data x2:
load('hw5data.mat')
>> c=10;
data = x2; yvals = x2(:,3);
[w,b,e] = quadprogsoftsvm(data,yvals,c)
Minimum found that satisfies the constraints.
Optimization completed because the objective function is non-decreasing in

6Assignment 5
feasible directions, to within the default value of the optimality tolerance,
and constraints are satisfied to within the default value of the constraint tolerance.
<stopping criteria details>
w =
-0.0000
-0.0000
1.0000
b =
5.1297e-08
e =
1.0e-11 *
0.6009
0.6374
0.6263
0.6433
0.6145
0.7037
0.5898
0.6131
0.5796

7Assignment 5
0.6402
0.7133
0.6957
0.5871
0.6591
0.6758
0.5988
0.5804
0.6122
0.5507
0.6205
Input and output for training data x3:
load('hw5data.mat')
>> data = x3; yvals = x3(:,4); c= 10;
>> [w,b,e] = quadprogsoftsvm(data,yvals,c)
Minimum found that satisfies the constraints.
Optimization completed because the objective function is non-decreasing in
feasible directions, to within the default value of the optimality tolerance,
and constraints are satisfied to within the default value of the constraint tolerance.

8Assignment 5
<stopping criteria details>
w =
-0.0720
0.0689
-0.0156
0.9897
b =
0.1261
e =
1.0e-12 *
0.3102
0.3100
0.3106
0.3102
0.3130
0.3109
0.3102
0.3105
0.3234
0.3103

9Assignment 5
0.3104
0.3088
0.3200
0.3105
0.3172
0.3105
0.3105
0.3094
0.3136
0.3103
Input and output for training data x4:
load('hw5data.mat')
data = x4; yvals = x4(:,6); c= 10;
[w,b,e] = quadprogsoftsvm(data,yvals,c)
Minimum found that satisfies the constraints.
Optimization completed because the objective function is non-decreasing in
feasible directions, to within the default value of the optimality tolerance,
and constraints are satisfied to within the default value of the constraint tolerance.
<stopping criteria details>

10Assignment 5
w =
0.0000
-0.0000
-0.0000
-0.0000
0.0000
1.0000
b =
1.0669e-07
e =
1.0e-12 *
0.7957
0.8173
0.7971
0.7979
0.7977
0.8226
0.7978
0.7961

11Assignment 5
0.7978
0.7980
0.7972
0.7960
0.8001
0.7974
0.7970
0.7927
0.8173
0.7976
0.7971
0.8194
0.7967
0.7971
0.8099
0.7960
0.7966
0.7975
0.7962
0.7981