How to do multi class classification using Support Vector Machines (SVM)

mlguy picture mlguy · Dec 24, 2009 · Viewed 87.8k times · Source

In every book and example always they show only binary classification (two classes) and new vector can belong to any one class.

Here the problem is I have 4 classes(c1, c2, c3, c4). I've training data for 4 classes.

For new vector the output should be like

C1 80% (the winner)

c2 10%

c3 6%

c4 4%

How to do this? I'm planning to use libsvm (because it most popular). I don't know much about it. If any of you guys used it previously please tell me specific commands I'm supposed to use.

Answer

Oben Sonne picture Oben Sonne · Feb 23, 2011

LibSVM uses the one-against-one approach for multi-class learning problems. From the FAQ:

Q: What method does libsvm use for multi-class SVM ? Why don't you use the "1-against-the rest" method ?

It is one-against-one. We chose it after doing the following comparison: C.-W. Hsu and C.-J. Lin. A comparison of methods for multi-class support vector machines, IEEE Transactions on Neural Networks, 13(2002), 415-425.

"1-against-the rest" is a good method whose performance is comparable to "1-against-1." We do the latter simply because its training time is shorter.