How to use the Scikit learn CountVectorizer?

python-3.x scikit-learn countvectorizer
Sanjeev picture Sanjeev · Dec 12, 2016 · Viewed 8.9k times · Source

I have a set of words for which I have to check whether they are present in the documents.

WordList = [w1, w2, ..., wn]

Another set have list of documents on which I have to check whether these words are present or not.

How to use scikit-learn CountVectorizer so that the features of term-document matrix are only words from WordList and each row represents each particular document with no of times the word from the given list appears in their respective column?

Answer

Sanjeev picture Sanjeev · Dec 12, 2016

Ok. I get it. The code is given below:

from sklearn.feature_extraction.text import CountVectorizer
# Counting the no of times each word(Unigram) appear in document. 
vectorizer = CountVectorizer(input='content',binary=False,ngram_range=(1,1))
# First set the vocab
vectorizer = vectorizer.fit(WordList)
# Now transform the text contained in each document i.e list of text 
Document:list
tfMatrix = vectorizer.transform(Document_List).toarray()

This will output only the term-document matrix with features from wordList only.

Related questions

ValueError: Expected 2D array, got 1D array instead:

While practicing Simple Linear Regression Model I got this error, I think there is something wrong with my data set. Here is my data set: Here is independent variable X: Here is dependent variable Y: Here is X_train Here …

Read More
how to resolve this ValueError: only 2 non-keyword arguments accepted sklearn python

hello i am new to sklearn in python and iam trying to learn it and use this module to predict some numbers based on two features here is the error i am getting: ValueError: only 2 non-keyword arguments accepted and here …

Read More
AttributeError: LinearRegression object has no attribute 'coef_'

I've been attempting to fit this data by a Linear Regression, following a tutorial on bigdataexaminer. Everything was working fine up until this point. I imported LinearRegression from sklearn, and printed the number of coefficients just fine. This was the …

Read More