How to obtain features' weights

python machine-learning svm scikit-learn libsvm
Paul85 picture Paul85 · Jan 21, 2014 · Viewed 12k times · Source

I am dealing with highly imbalanced data set and my idea is to obtain values of feature weights from my libSVM model. As for now I am OK with the linear kernel, where I can obtain feature weights, but when I am using rbf or poly, I fail to reach my objective.

Here I am using sklearn for my model and it's easy to obtain feature weights for linear kernel using .coef_. Can anyone help me to do same thing for rbf or poly? What I've tried to do so far is given below:

svr = SVC(C=10, cache_size=200, class_weight='auto', coef0=0.0, degree=3.0, gamma=0.12,kernel='rbf', max_iter=-1, probability=True, random_state=0,shrinking=True, tol=0.001, verbose=False)
clf = svr.fit(data_train,target_train)
print clf.coef_

Answer

BartoszKP picture BartoszKP · Jan 21, 2014

This is not only impossible, as stated in the documentation:

Weights asigned to the features (coefficients in the primal problem). This is only available in the case of linear kernel.

but also it doesn't make sense. In linear SVM the resulting separating plane is in the same space as your input features. Therefore its coefficients can be viewed as weights of the input's "dimensions".

In other kernels, the separating plane exists in another space - a result of kernel transformation of the original space. Its coefficients are not directly related to the input space. In fact, for the rbf kernel the transformed space is infinite-dimensional (you can get a starting point on this on Wikipedia of course).

Related questions

Python : How to find Accuracy Result in SVM Text Classifier Algorithm for Multilabel Class

I have used following set of code: And I need to check accuracy of X_train and X_test The following code works for me in my classification problem over multi-labeled class import numpy as np from sklearn.pipeline import …

Read More
Determining the most contributing features for SVM classifier in sklearn

I have a dataset and I want to train my model on that data. After training, I need to know the features that are major contributors in the classification for a SVM classifier. There is something called feature importance for …

Read More
Converting LinearSVC's decision function to probabilities (Scikit learn python )

I use linear SVM from scikit learn (LinearSVC) for binary classification problem. I understand that LinearSVC can give me the predicted labels, and the decision scores but I wanted probability estimates (confidence in the label). I want to continue using …

Read More