metaphone versus soundex versus NYSIIS

machine-learning nlp soundex phonetics metaphone
Slater Victoroff picture Slater Victoroff · Jul 20, 2012 · Viewed 8.2k times · Source

I'm trying to come up with an implicit spell checker that will use the mappings of input words to some kind of more general phonetic representation to account for typos that might occur, basically for a search bar that will automatically correct your spelling to a degree. Two things that I've been looking into are metaphone, nysiis and soundex, but I don't really know which would be better for this application.

I would like there to be preferentially more matches than less matches, and I would like the matching to be a bit more general and so for that reason I was thinking of going with soundex which seems to be a more approximate mapping than the original metaphone, but I don't really know how large the difference in vagueness is. I know that nysiis is pretty similar to soundex, but I don't have a good idea of how similar they are, or how nysiis compares to metaphone.

I am also looking for the solution that is quickest to execute. I know that these phonetic mappers are usually pretty quick, but I'm not sure which would be fastest, considering I would like to be able to check spelling without an increase in search time, speed is a consideration. Thoughts?

Answer

Slater Victoroff picture Slater Victoroff · Jun 4, 2013

I managed to find a wonderful article on this over here:

http://www.informit.com/articles/article.aspx?p=1848528

Not quite everything I was looking for, but a pretty large amount of it.

Related questions

How to compute precision, recall, accuracy and f1-score for the multiclass case with scikit learn?

I'm working in a sentiment analysis problem the data looks like this: label instances 5 1190 4 838 3 239 1 204 2 127 So my data is unbalanced since 1190 instances are labeled with 5. For the classification Im using scikit's SVC. The problem is I do not know how to …

Read More
How does the Google "Did you mean?" Algorithm work?

I've been developing an internal website for a portfolio management tool. There is a lot of text data, company names etc. I've been really impressed with some search engines ability to very quickly respond to queries with "Did you mean: …

Read More
Understanding min_df and max_df in scikit CountVectorizer

I have five text files that I input to a CountVectorizer. When specifying min_df and max_df to the CountVectorizer instance what does the min/max document frequency exactly means? Is it the frequency of a word in its …

Read More