Support Vector Machine vs K Nearest Neighbours

matlab machine-learning libsvm
Mohit Jain picture Mohit Jain · Oct 17, 2013 · Viewed 12k times · Source

I have a data set to classify.By using KNN algo i am getting an accuracy of 90% but whereas by using SVM i just able to get over 70%. Is SVM not better than KNN. I know this might be stupid to ask but, what are the parameters for SVM which will give nearly approximate results as KNN algo. I am using libsvm package on matlab R2008

Answer

Shai picture Shai · Oct 17, 2013

kNN and SVM represent different approaches to learning. Each approach implies different model for the underlying data.

SVM assumes there exist a hyper-plane seperating the data points (quite a restrictive assumption), while kNN attempts to approximate the underlying distribution of the data in a non-parametric fashion (crude approximation of parsen-window estimator).

You'll have to look at the specifics of your scenario to make a better decision as to what algorithm and configuration are best used.

Related questions

Multi-class classification in libsvm

I'm working with libsvm and I must implement the classification for multiclasses with one versus all. How can I do it? Does libsvm version 2011 use this? I think that my question is not very clear. if libsvm don't use automatically …

Read More
How to use libsvm in Matlab?

I am new to matlab and don't know how to use libsvm. Is there any sample code for classifying some data (with 2 features) with a SVM and then visualize the result? How about with kernel (RBF, Polynomial, and Sigmoid )? I …

Read More
Multi-Class SVM( one versus all)

I know that LIBSVM only allows one-vs-one classification when it comes to multi-class SVM. However, I would like to tweak it a bit to perform one-against-all classification. I have tried to perform one-against-all below. Is this the correct approach? The …

Read More