How much time does take train SVM classifier?

python numpy machine-learning svm
Il'ya Zhenin picture Il'ya Zhenin · Aug 10, 2013 · Viewed 33.6k times · Source

I wrote following code and test it on small data:

classif = OneVsRestClassifier(svm.SVC(kernel='rbf'))
classif.fit(X, y)

Where X, y (X - 30000x784 matrix, y - 30000x1) are numpy arrays. On small data algorithm works well and give me right results.

But I run my program about 10 hours ago... And it is still in process.

I want to know how long it will take, or it stuck in some way? (Laptop specs 4 GB Memory, Core i5-480M)

Answer

lejlot picture lejlot · Aug 11, 2013

SVM training can be arbitrary long, this depends on dozens of parameters:

  • C parameter - greater the missclassification penalty, slower the process
  • kernel - more complicated the kernel, slower the process (rbf is the most complex from the predefined ones)
  • data size/dimensionality - again, the same rule

in general, basic SMO algorithm is O(n^3), so in case of 30 000 datapoints it has to run number of operations proportional to the2 700 000 000 000which is realy huge number. What are your options?

  • change a kernel to the linear one, 784 features is quite a lot, rbf can be redundant
  • reduce features' dimensionality (PCA?)
  • lower the C parameter
  • train model on the subset of your data to find the good parameters and then train the whole one on some cluster/supercomputer

Related questions

How to implement the Softmax function in Python

From the Udacity's deep learning class, the softmax of y_i is simply the exponential divided by the sum of exponential of the whole Y vector: Where S(y_i) is the softmax function of y_i and e is …

Read More
Convert array of indices to 1-hot encoded numpy array

Let's say I have a 1d numpy array a = array([1,0,3]) I would like to encode this as a 2D one-hot array b = array([[0,1,0,0], [1,0,0,0], [0,0,0,1]]) Is there a quick way to do this? Quicker than just looping over a to set elements …

Read More
gradient descent using python and numpy

def gradient(X_norm,y,theta,alpha,m,n,num_it): temp=np.array(np.zeros_like(theta,float)) for i in range(0,num_it): h=np.dot(X_norm,theta) #temp[j]=theta[j]-(alpha/m)*( np.sum( (…

Read More