Difference between PCA (Principal Component Analysis) and Feature Selection

machine-learning pca feature-selection
AbhinavChoudhury picture AbhinavChoudhury · Apr 27, 2013 · Viewed 14.9k times · Source

What is the difference between Principal Component Analysis (PCA) and Feature Selection in Machine Learning? Is PCA a means of feature selection?

Answer

Roger Rowland picture Roger Rowland · Apr 27, 2013

PCA is a way of finding out which features are important for best describing the variance in a data set. It's most often used for reducing the dimensionality of a large data set so that it becomes more practical to apply machine learning where the original data are inherently high dimensional (e.g. image recognition).

PCA has limitations though, because it relies on linear relationships between feature elements and it's often unclear what the relationships are before you start. As it also "hides" feature elements that contribute little to the variance in the data, it can sometimes eradicate a small but significant differentiator that would affect the performance of a machine learning model.

Related questions

How to use scikit-learn PCA for features reduction and know which features are discarded

I am trying to run a PCA on a matrix of dimensions m x n where m is the number of features and n the number of samples. Suppose I want to preserve the nf features with the maximum variance. …

Read More
Feature/Variable importance after a PCA analysis

I have performed a PCA analysis over my original dataset and from the compressed dataset transformed by the PCA I have also selected the number of PC I want to keep (they explain almost the 94% of the variance). Now I …

Read More
Recovering features names of explained_variance_ratio_ in PCA with sklearn

I'm trying to recover from a PCA done with scikit-learn, which features are selected as relevant. A classic example with IRIS dataset. import pandas as pd import pylab as pl from sklearn import datasets from sklearn.decomposition import PCA # load …

Read More