Best Algorithmic Approach to Sentiment Analysis

Question

My requirement is taking in news articles and determining if they are positive or negative about a subject. I am taking the approach outlined below, but I keep reading NLP may be of use here. All that I have read has pointed at NLP detecting opinion from fact, which I don't think would matter much in my case. I'm wondering two things:

1) Why wouldn't my algorithm work and/or how can I improve it? ( I know sarcasm would probably be a pitfall, but again I don't see that occurring much in the type of news we will be getting)

2) How would NLP help, why should I use it?

My algorithmic approach (I have dictionaries of positive, negative, and negation words):

1) Count number of positive and negative words in article

2) If a negation word is found with 2 or 3 words of the positive or negative word, (ie: NOT the best) negate the score.

3) Multiply the scores by weights that have been manually assigned to each word. (1.0 to start)

4) Add up the totals for positive and negative to get the sentiment score.

Answer 1

I don't think there's anything particularly wrong with your algorithm, it's a fairly straightforward and practical way to go, but there are a lot of situations where it will get make mistakes.

1. Ambiguous sentiment words - "This product works terribly" vs. "This product is terribly good"

2. Missed negations - "I would never in a millions years say that this product is worth buying"

3. Quoted/Indirect text - "My dad says this product is terrible, but I disagree"

4. Comparisons - "This product is about as useful as a hole in the head"

5. Anything subtle - "This product is ugly, slow and uninspiring, but it's the only thing on the market that does the job"

I'm using product reviews for examples instead of news stories, but you get the idea. In fact, news articles are probably harder because they will often try to show both sides of an argument and tend to use a certain style to convey a point. The final example is quite common in opinion pieces, for example.

As far as NLP helping you with any of this, word sense disambiguation (or even just part-of-speech tagging) may help with (1), syntactic parsing might help with the long range dependencies in (2), some kind of chunking might help with (3). It's all research level work though, there's nothing that I know of that you can directly use. Issues (4) and (5) are a lot harder, I throw up my hands and give up at this point.

I'd stick with the approach you have and look at the output carefully to see if it is doing what you want. Of course that then raises the issue of what you want you understand the definition of "sentiment" to be in the first place...