Python: Jaccard Distance using word intersection but not character intersection

python set intersection
add-semi-colons picture add-semi-colons · Aug 11, 2012 · Viewed 21.3k times · Source

I didn't realize the that Python set function actually separating string into individual characters. I wrote python function for Jaccard and used python intersection method. I passed two sets into this method and before passing the two sets into my jaccard function I use the set function on the setring.

example: assume I have string NEW Fujifilm 16MP 5x Optical Zoom Point and Shoot CAMERA 2 7 screen.jpg i would call set(NEW Fujifilm 16MP 5x Optical Zoom Point and Shoot CAMERA 2 7 screen.jpg) which will separate string into characters. So when I send it to jaccard function intersection actually look character intersection instead of word to word intersection. How can I do word to word intersection.

#implementing jaccard
def jaccard(a, b):
    c = a.intersection(b)
    return float(len(c)) / (len(a) + len(b) - len(c))

if I don't call set function on my string NEW Fujifilm 16MP 5x Optical Zoom Point and Shoot CAMERA 2 7 screen.jpg I get the following error:

    c = a.intersection(b)
AttributeError: 'str' object has no attribute 'intersection'

Instead of character to character intersection I want to do word to word intersection and get the jaccard similarity.

Answer

Amber picture Amber · Aug 11, 2012

Try splitting your string into words first:

word_set = set(your_string.split())

Example:

>>> word_set = set("NEW Fujifilm 16MP 5x".split())
>>> character_set = set("NEW Fujifilm 16MP 5x")
>>> word_set
set(['NEW', '16MP', '5x', 'Fujifilm'])
>>> character_set
set([' ', 'f', 'E', 'F', 'i', 'M', 'j', 'm', 'l', 'N', '1', 'P', 'u', 'x', 'W', '6', '5'])

Related questions

Python set Union and set Intersection operate differently?

I'm doing some set operations in Python, and I noticed something odd.. >> set([1,2,3]) | set([2,3,4]) set([1, 2, 3, 4]) >> set().union(*[[1,2,3], [2,3,4]]) set([1, 2, 3, 4]) That's good, expected behaviour - but with intersection: >> set([1,2,3]) & set([2,3,4]) set([2, 3]) >> set().intersection(*[[1,2,3], [2,3,4]]) …

Read More
Python -Intersection of multiple lists?

I am playing with python and am able to get the intersection of two lists: result = set(a).intersection(b) Now if d is a list containing a and b and a third element c, is there an built-in function …

Read More
Get difference between two lists

I have two lists in Python, like these: temp1 = ['One', 'Two', 'Three', 'Four'] temp2 = ['One', 'Two'] I need to create a third list with items from the first list which aren't present in the second one. From the example I …

Read More