Is it possible to edit NLTK's vader sentiment lexicon?

noobalert picture noobalert · Nov 8, 2016 · Viewed 7.1k times · Source

I would like to add words to the vader_lexicon.txt to specify polarity scores to a word. What is the right way to do so?

I saw this file in AppData\Roaming\nltk_data\sentiment\vader_lexicon. The file consists of the word, its polarity, intensity, and an array of 10 intensity scores given by "10 independent human raters". [1] However, when I edited it, nothing changed in the results of the following code:

from nltk.sentiment.vader import SentimentIntensityAnalyzer
sia = SentimentIntensityAnalyzer()
s = sia.polarity_scores("my string here")

I think that this text file is accessed by my code when I called SentimentIntensityAnalyzer's constructor. [2] Do you have any ideas on how I can edit a pre-made lexicon?

Sources:

[1] https://github.com/cjhutto/vaderSentiment

[2] http://www.nltk.org/api/nltk.sentiment.html

Answer

Laurie picture Laurie · Jul 25, 2018

For anyone interested, this can also be achieved without having to manually edit the vader lexicon .txt file. Once loaded the lexicon is a normal dictionary with words as keys and scores as values. As provided by repoleved in this post:

from nltk.sentiment.vader import SentimentIntensityAnalyzer

new_words = {
    'foo': 2.0,
    'bar': -3.4,
}

SIA = SentimentIntensityAnalyzer()

SIA.lexicon.update(new_words)

If you wish to remove words, use the '.pop' function:

SIA = SentimentIntensityAnalyzer()

SIA.lexicon.pop('no')