Value of k in k nearest neighbor algorithm

classification knn
user574183 picture user574183 · Jul 19, 2012 · Viewed 46.1k times · Source

I have 7 classes that needs to be classified and I have 10 features. Is there a optimal value for k that I need to use in this case or do I have to run the KNN for values of k between 1 and 10 (around 10) and determine the best value with the help of the algorithm itself?

Answer

NominSim picture NominSim · Jul 19, 2012

In addition to the article I posted in the comments there is this one as well that suggests:

Choice of k is very critical – A small value of k means that noise will have a higher influence on the result. A large value make it computationally expensive and kinda defeats the basic philosophy behind KNN (that points that are near might have similar densities or classes ) .A simple approach to select k is set k = n^(1/2).

It's going to depend a lot on your individual cases, sometimes it is best to run through each possible value for k and decide for yourself.

Related questions

KNN classification with categorical data

I'm busy working on a project involving k-nearest neighbour regression. I have mixed numerical and categorical fields. The categorical values are ordinal (e.g. bank name, account type). Numerical types are, for e.g. salary and age. There are also …

Read More
Finding K-nearest neighbors and its implementation

I am working on classifying simple data using KNN with Euclidean distance. I have seen an example on what I would like to do that is done with the MATLAB knnsearch function as shown below: load fisheriris x = meas(:,3:4); gscatter(…

Read More
kNN: training, testing, and validation

I am extracting image features from 10 classes with 1000 images each. Since there are 50 features that I can extract, I am thinking of finding the best feature combination to use here. Training, validation and test sets are divided as follows: Training …

Read More