Designing a Linear Support Vector Machine using Supervised Learning Method

Running head: ASSIGNMENT 5
Assignment 5
Name of the Student
Name of the University
Author Note

1Assignment 5
Assignment Introduction:
In this particular assignment the supervised learning method is used for designing a linear
support vector machine which can classify the vectors the d-dimensional feature vectors in
any one of the two classes given by y = +1 or -1. The function of the optimal binary classifier
is given by,
f(x) = sign(wTx + b)
The 4 different training data sets are loaded in MATLAB from hw5data.mat. The labels of
the dataset are x1, x2, x3 and x4.
1.
The given constraint optimization problem is
Min ( 1
2 zT Qz + c∑
i=1
N
εi)
Constraints:
yi ( wT xi+ b ) ≥1−εi for all i
ε i ≥ 0
Here,
∈= [∈ 1∈ 2 ... ..∈ N ]T
z = [ w1 w2 ... . wd b ]T
Now, quadprog in MATLAB is used to solve the problem. The quadprog
objective function in standard form is given by,

2Assignment 5
( 1
2 xT Hx+ f T x)
x
min
such that,
A*x <= b, Aeq∗x =beq, lb≤ x ≤ ub
This problem is a Soft-margin SVM model. The MATLAB function which
solves the problem is shown below.
MATLAB function:
function [w,b,e] = quadprogsoftsvm(data,yvals,c)
rows = size(data,1);
cols = size(data,2);
H = diag([ones(1, cols), zeros(1, rows + 1)]);
f = [zeros(1,cols+1) c*ones(1,rows)]';
p = diag(yvals) * data;
A = -[p yvals eye(rows)];
B = -ones(rows,1);
lb = [-inf * ones(cols+1,1) ;zeros(rows,1)];
z = quadprog(H,f,A,B,[],[],lb);
w = z(1:cols,:);
b = z(cols+1:cols+1,:);
e = z(cols+2:cols+rows+1,:);

3Assignment 5
end
2.
Now, the above MATLAB function is applied to primal form of classifier on the four training
data x1,x2, x3 and x4 and this outputs the w, b and e(error) from original y values. Here, the
value of constant c = 10 is considered.
Input and output for training data x1:
load('hw5data.mat')
>> c=10;
>> data = x1; yvals = x1(:,3);
>> [w,b,e] = quadprogsoftsvm(data,yvals,c)
Minimum found that satisfies the constraints.
Optimization completed because the objective function is non-decreasing in
feasible directions, to within the default value of the optimality tolerance,
and constraints are satisfied to within the default value of the constraint tolerance.
<stopping criteria details>
w =
-0.2062
0.0607
0.9515