SVM in Matlab: Meaning of Parameter 'box constraint' in function fitcsvm
I'm new to SVMs in Matlab and need a little bit of help with it.
I want to train a support vector machine using the build in function fitcsvm of the Statistics Toolbox. Of course there are many parameter choices which control how the SVM will be trained.
The Matlab help is a litte bit wage about how the parameters archive a better training result. Especially the parameter 'Box Contraint' seems to have an important influence on the number of chosen support vectors and generalization quality.
The Help (http://de.mathworks.com/help/stats/fitcsvm.html#bt8v_z4-1) says
A parameter that controls the maximum penalty imposed on margin-violating observations, and aids in preventing overfitting (regularization).
If you increase the box constraint, then the SVM classifier assigns fewer support vectors. However, increasing the box constraint can lead to longer training times.
How exactly is this parameter used? Is it the same or something like the soft margin factor C in the Wikipedia reference? Or something completely different?
Thanks for your help.
Answer
You were definitely on the right path. While description in the documentation of fitcsvm (as you posted in the question) is very short, you should have a look at the Understanding Support Vector Machines site in the MATLAB documentation.
In the non-separable case (often called Soft-Margin SVM), one allows misclassifications, at the cost of a penalty factor C. The mathematical formulation of the SVM then becomes:
with the slack variables s_i which cause a penalty term which is weighted by C.
Making C large increases the weight of misclassifications, which leads to a stricter separation.
This factor C is called box constraint.
The reason for this name is, that in the formulation of the dual optimization problem, the Langrange multipliers are bounded to be within the range [0,C].
C thus poses a box constraint on the Lagrange multipliers.
tl;dr your guess was right, it is the C in the soft margin SVM.