How to measure similarity between strings?

Question

I have a bunch of names, and I want to obtain the unique names. However, due to spelling errors and inconsistencies in the data the names might be written down wrong. I am looking for a way to check in a vector of strings if two of them are similair.

For example:

pres <- c(" Obama, B.","Bush, G.W.","Obama, B.H.","Clinton, W.J.")

I want to find that " Obama, B." and "Obama, B.H." are very similar. Is there a way to do this?

Answer 1

This can be done based on eg the Levenshtein distance. There are multiple implementations of this in different packages. Some solutions and packages can be found in the answers of these questions:

• agrep: only return best match(es)
• In R, how do I replace a string that contains a certain pattern with another string?
• Fast Levenshtein distance in R?

But most often agrep will do what you want :

> sapply(pres,agrep,pres)
$` Obama, B.`
[1] 1 3

$`Bush, G.W.`
[1] 2

$`Obama, B.H.`
[1] 1 3

$`Clinton, W.J.`
[1] 4