"Stop words" list for English?

language-agnostic indexing filtering stop-words nlp
Mark Harrison picture Mark Harrison · Aug 2, 2009 · Viewed 20.2k times · Source

I'm generating some statistics for some English-language text and I would like to skip uninteresting words such as "a" and "the".

  • Where can I find some lists of these uninteresting words?
  • Is a list of these words the same as a list of the most frequently used words in English?

update: these are apparently called "stop words" and not "skip words".

Answer

Thomas picture Thomas · Aug 2, 2009

The magic word to put into Google is "stop words". This turns up a reasonable-looking list.

MySQL also has a built-in list of stop words, but this is far too comprehensive to my tastes. For example, at our university library we had problems because "third" in "third world" was considered a stop word.

Related questions

What and where are the stack and heap?

Programming language books explain that value types are created on the stack, and reference types are created on the heap, without explaining what these two things are. I haven't read a clear explanation of this. I understand what a stack …

Read More
What is dependency injection?

There have been several questions already posted with specific questions about dependency injection, such as when to use it and what frameworks are there for it. However, What is dependency injection and when/why should or shouldn't it be used?

Read More
Regex to match only letters

How can I write a regex that matches only letters?

Read More