Finding topics of an unseen document via Gensim

python nlp latent-semantic-indexing gensim
Peter Kirby picture Peter Kirby · Jul 13, 2012 · Viewed 11.2k times · Source

I am using Gensim to do some large-scale topic modeling. I am having difficulty understanding how to determine predicted topics for an unseen (non-indexed) document. For example: I have 25 million documents which I have converted to vectors in LSA (and LDA) space. I now want to figure out the topics of a new document, lets call it x.

According to the Gensim documentation, I can use:

topics = lsi[doc(x)]

where doc(x) is a function that converts x into a vector.

The problem is, however, that the above variable, topics, returns a vector. The vector is useful if I am comparing x to additional documents because it allows me to find the cosine similarity between them, but I am unable to actually return specific words that are associated with x itself.

Am I missing something, or does Gensim not have this capability?

Thank you,

EDIT

Larsmans has the answer.

I was able to show the topics by using:

for t in topics:
    print lsi.show_topics(t[0])

Answer

Fred Foo picture Fred Foo · Jul 13, 2012

The vector returned by [] on an LSI model is actually a list of (topic, weight) pairs. You can inspect a topic by means of the method LsiModel.show_topic

Related questions

What NLP tools to use to match phrases having similar meaning or semantics

I am working on a project which requires me to match a phrase or keyword with a set of similar keywords. I need to perform semantic analysis for the same. an example: Relevant QT cheap health insurance affordable health insurance …

Read More
How to get rid of punctuation using NLTK tokenizer?

I'm just starting to use NLTK and I don't quite understand how to get a list of words from text. If I use nltk.word_tokenize(), I get a list of words and punctuation. I need only the words instead. …

Read More
How to compute precision, recall, accuracy and f1-score for the multiclass case with scikit learn?

I'm working in a sentiment analysis problem the data looks like this: label instances 5 1190 4 838 3 239 1 204 2 127 So my data is unbalanced since 1190 instances are labeled with 5. For the classification Im using scikit's SVC. The problem is I do not know how to …

Read More