Tweepy Streaming API returning "None" for coordinates on geo-enabled tweets

user2442175 picture user2442175 · Jun 1, 2013 · Viewed 7.9k times · Source

I am using Tweepy to access the streaming API. I am able to get results with the code below but for tweets where the Geo Enabled value is "True" I am getting a Coordinates returned value of "False". How can this be? Do I need to decode the JSON object being returned for status.coordinates?

from tweepy.streaming import StreamListener
from tweepy import OAuthHandler
from tweepy import Stream
import random
import time
import MySQLdb
import json

consumer_key="XXX"
consumer_secret="XXX"

access_token="XXX"
access_token_secret="XXX"

db=MySQLdb.connect(host='localhost', user='XXX', passwd='XXX', db='twitter')
db.set_character_set('utf8')

Coords = dict()
Place = dict()
PlaceCoords = dict()
XY = []
curr=db.cursor()

class StdOutListener(StreamListener):
    """ A listener handles tweets that are the received from the stream.
    This is a basic listener that inserts tweets into MySQLdb.
    """
    def on_status(self, status):

        print "Tweet Text: ",status.text

        text = status.text

        print "Time Stamp: ",status.created_at

        print "Time Stamp: ",status.created_at

        print "Source: ",status.source

        source = status.source

        print "Author: ",status.user.screen_name

        author = status.user.screen_name

        print "Name: ",status.user.name

        name = status.user.name

        print "Time Zone: ",status.user.time_zone

        time_zone = status.user.time_zone

        print "User Language: ",status.user.lang

        user_language = status.user.lang

        print "Followers: ",status.user.followers_count

        followers = status.user.followers_count

        print "User Description: ",status.user.description

        user_description = status.user.description

        print "Geo Enabled: ",status.user.geo_enabled

        geo_enabled = status.user.geo_enabled

        print "Friends: ",status.user.friends_count

        friends = status.user.friends_count

        print "Retweets: ",status.retweet_count

        retweets = status.retweet_count

        print "Location: ",status.user.location

        location = status.user.location

        print "ID: ",status.user.id_str

        user_id = status.user.id_str

        print "Coordinates: ",status.coordinates

        coordinates = status.coordinates

        print "Place: ",status.place

        place = status.place

Here is a sample result output:

Tweet Text: @aranone aran tu eres el mejor soy tu fanatico 1 me gusta tu musica.hey pana sique asi q vay bn te deseo lo mejor bro)

Time Stamp: 2013-05-30 23:36:38

Time Stamp: 2013-05-30 23:36:38

Source: web

Author: juandvd_96

Name: juan David Romero

Time Zone: Atlantic Time (Canada)

User Language: es

Followers: 365

User Description: hola soy juan david... soy una chico muy enamorado... y soy muy fekiz...

Geo Enabled: True

Friends: 1857

Retweets: 0

Location: veezuela maracaibo

ID: 481513551

Coordinates: None

Place: None

cheers, BD

Thanks for clarifying. I was checking out the listener just now and noticed a tweet where coordinates were populated but as a json object. I am writing tweets to a mysql db as they are streamed and it seems like the one with the coordinates info was not inserted into the database. Not sure if the errors around the SQL statement are for the first or second tweet, both columns where the error occurred are set to 'varchar' values. Here is the streaming result:

Tweet Text: Vi 10 minutos y no pude ver mas. Soy super cagona, dios. Vay a ver otra.

Time Stamp: 2013-06-04 01:08:57

Time Stamp: 2013-06-04 01:08:57

Source: web

Author: ailenvalli

Name: Λili

Time Zone: Santiago

User Language: es

Followers: 384

User Description: Create your reality or it will be created for you

http://instagram.com/ailenvalli

Geo Enabled: True

Friends: 338

Retweets: 0

Location: 704 East Broadway ▲ 1966

ID: 200264965

Coordinates: None

Place: None

firehose_geo.py:87: Warning: Incorrect string value: '\xCE\x9Bili' for column 'Name' at row 1

(text,status.created_at,status.created_at,source,author,name,time_zone,user_language,followers,user_description,geo_enabled,friends,retweets,location,user_id,coordinates,geo)) firehose_geo.py:87: Warning: Incorrect string value: '\xE2\x96\xB2 19...' for column 'Location' at row 1

(text,status.created_at,status.created_at,source,author,name,time_zone,user_language,followers,user_description,geo_enabled,friends,retweets,location,user_id,coordinates,geo))

Tweet Text: I have a feeling WalMart is fixing to take a chunk out of my wallet. Healthy food is so expensive.

Time Stamp: 2013-06-04 01:42:00

Time Stamp: 2013-06-04 01:42:00

Source: Twitter for Android

Author: KaylaRenae21

Name: †Kayla Renae'

Time Zone: Central Time (US & Canada)

User Language: en

Followers: 300

User Description: The things I like to do cannot be found in the city. Hand me a fishing pole & I'll be gone all day.

Geo Enabled: True

Friends: 437

Retweets: 0

Location: Oklahoma

ID: 282414509

Coordinates: {'type': 'Point', 'coordinates': [-96.6623549, 34.7918959]}

Place: {'type': 'Point', 'coordinates': [34.7918959, -96.6623549]}

Answer

alecxe picture alecxe · Jun 3, 2013

The problem is not related to tweepy itself.

For example, see this tweet (https://api.twitter.com/1/statuses/show.json?id=341458303064354817&include_entities=true) - it has geo_enabled set to true while geo, coordinates and place equal to null.

According to twitter docs:

geo_enabled: When true, indicates that the user has enabled the possibility of geotagging their Tweets.

So, it's not a strict rule that there will be location info in the tweet data if geo_enabled is true. Just check if status.geo or status.coordinates are not None in your listener.

Hope that helps.