Algorithm #5
Support Vector Machine
Maximum Margin Classifier
🎯 Finding the Optimal Boundary
MAX
MARGIN
MARGIN
✨ Support vectors (glowing points) define the widest possible margin
Margin Maximization
SVM finds the hyperplane that creates the widest "street" between classes.
Wider margin = better generalization to new data.
Support Vectors
Only the closest points (support vectors) matter for defining the boundary.
Other points can be moved without changing the decision surface.
The Kernel Trick
Can't separate classes with a straight line? Map to higher dimensions where separation becomes possible
— without actually computing the transformation.
2D mixed
→
Higher-D
separable
separable
Soft Margin (C parameter)
Real data isn't perfectly separable. C controls the tradeoff:
large C = strict margin (may overfit), small C = flexible margin (more tolerant of errors).
🔧 Key Parameters
kernel
'linear', 'rbf', 'poly', 'sigmoid' — shape of decision boundary
C
Regularization: tradeoff between margin width and training errors
gamma
For RBF kernel: how far influence of single point reaches