Supervised Latent Dirichlet Allocation for Document Classification?

snøreven picture snøreven · Nov 25, 2012 · Viewed 14.9k times · Source

I have a bunch of already human-classified documents in some groups.

Is there a modified version of lda which I can use to train a model and then later classify unknown documents with it?

Answer

Ben Allison picture Ben Allison · Nov 26, 2012

For what it's worth, LDA as a classifier is going to be fairly weak because it's a generative model, and classification is a discriminative problem. There is a variant of LDA called supervised LDA which uses a more discriminative criterion to form the topics (you can get source for this in various places), and there's also a paper with a max margin formulation that I don't know the status of source-code-wise. I would avoid the Labelled LDA formulation unless you're sure that's what you want, because it makes a strong assumption about the correspondence between topics and categories in the classification problem.

However, it's worth pointing out that none of these methods use the topic model directly to do the classification. Instead, they take documents, and instead of using word-based features use the posterior over the topics (the vector that results from inference for the document) as its feature representation before feeding it to a classifier, usually a Linear SVM. This gets you a topic model based dimensionality reduction, followed by a strong discriminative classifier, which is probably what you're after. This pipeline is available in most languages using popular toolkits.