machine for linearly separable binary sets Suppose we have this two features, x1 and

x2 here and we want to classify all this elements You can see that we have the class square

and the class rectangle So the goal of the SVM is to design a hyperplane,

here we define this green line as the hyperplane, that classifies all the training vectors in

two classes Here we show two different hyperplanes which

can classify correctly all the instances in this feature set

But the best choice will be the hyperplane that leaves the maximum margin from both classes

The margin is this distance between the hyperplane and the closest elements from this hyperplane

We have the case of the red hyperplane we have this distance, so this is the margin,

which we represent by z1 And in the case of the green hyperplane we

have the margin that we call z2 We can clearly see that the value of z2 is

greater than z1 So the margin is higher in the case of the

green hyperplane, so in this case the best choice will be the green hyperplane

Suppose we have this hyperplane, this hyperplane is defined by one equation, we can state this

equation as this one We have a vector of weights plus omega 0 and

this equation will deliver values greater than 1 for all the input vectors which belongs

to the class 1, in this case the circles And also, we scale this hyperplane so that

it will deliver values smaller than -1 for all values which belongs to class number 2,

the rectangles We can say that this distance to the closest

elements will be at least 1, the modulus is 1

From the geometry we know that the distance between a point and a hyperplane is computed

by this equation So the total margin which is composed by this

distance will be computed by this equation And the aim is that minimizing this term will

maximize the separability When we minimize this weight vector we will

have the biggest margin here that will split this two classes

To minimize this weight vector is a nonlinear optimization task, which can be solved by

this conditions (KKT), which uses Langrange multipliers

The main equations state that the value of omega will be the solution of this sum here

And we also have this other rule. So when we solve these equations, trying to minimize

this omega vector, we will maximize the margin between the two classes which will maximize

the separability the two classes Here we show a simple example

Suppose we have these 2 features, x1 and x2, and we have these 3 values

We want to design, or to find the best hyperplane that will divide this 2 classes

So we know that we can see clearly from this graph that the best division line will be

a parallel line to the line that connects these 2 values here

So we can define this weight vector, which is this point minus this other point. So we

have the constant a and 2 times this constant a

Now we can solve this weight vector and create the hyperplane equations considering this

weight vector We must discover the values of this a here

Since we have this weight vector omega here, we can substitute the values of this point

and also using this point we can substitute these 2 values here

When we place the equation g using the input vector (1,1) we know that we have the value

-1 because this belongs to the class circle So we will have this value here, when we use

the second point, we apply the function and we know that it will deliver the value 1

So we substitute here in the equation also Well, given 2 equations we can isolate the

value of omega 0 in the second equation and we will have omega 0 equal to 1 minus 8 times

a So, using this value, we put the omega 0 in

the first equation and we will reach the value of a, which is 2 divided by 5

Now we discover the value of a and now we substitute the first equation and also discover

the value of omega 0 So by dividing here we will come to the conclusion

that omega 0 is minus 11 divided by 5 and since we know that the weight vector is a

and 2 a we can substitute the value of a here and we will deliver these values of the weight

vector So in this case, these are called the support

vectors because they compose the omega value 2 divided by 5 and 4 divided by 5

And we substitute here the values of omega (2 divided by 5 and 4 divided by 5) and also

the omega 0 value we will deliver the final equation which defines this green hyperplane

which is x1 plus 2 times x2 minus 5.5 And this hyperplane classifies the elements

using support vector machines These are some references that we have used

So this is how SVM algorithm works