I am using Tweepy to access the streaming API. I am able to get results with the code below but for tweets where the Geo Enabled value is "True" I am getting a Coordinates returned value of "False". How can this be? Do I need to decode the JSON object being returned for status.coordinates?
from tweepy.streaming import StreamListener
from tweepy import OAuthHandler
from tweepy import Stream
import random
import time
import MySQLdb
import json
consumer_key="XXX"
consumer_secret="XXX"
access_token="XXX"
access_token_secret="XXX"
db=MySQLdb.connect(host='localhost', user='XXX', passwd='XXX', db='twitter')
db.set_character_set('utf8')
Coords = dict()
Place = dict()
PlaceCoords = dict()
XY = []
curr=db.cursor()
class StdOutListener(StreamListener):
""" A listener handles tweets that are the received from the stream.
This is a basic listener that inserts tweets into MySQLdb.
"""
def on_status(self, status):
print "Tweet Text: ",status.text
text = status.text
print "Time Stamp: ",status.created_at
print "Time Stamp: ",status.created_at
print "Source: ",status.source
source = status.source
print "Author: ",status.user.screen_name
author = status.user.screen_name
print "Name: ",status.user.name
name = status.user.name
print "Time Zone: ",status.user.time_zone
time_zone = status.user.time_zone
print "User Language: ",status.user.lang
user_language = status.user.lang
print "Followers: ",status.user.followers_count
followers = status.user.followers_count
print "User Description: ",status.user.description
user_description = status.user.description
print "Geo Enabled: ",status.user.geo_enabled
geo_enabled = status.user.geo_enabled
print "Friends: ",status.user.friends_count
friends = status.user.friends_count
print "Retweets: ",status.retweet_count
retweets = status.retweet_count
print "Location: ",status.user.location
location = status.user.location
print "ID: ",status.user.id_str
user_id = status.user.id_str
print "Coordinates: ",status.coordinates
coordinates = status.coordinates
print "Place: ",status.place
place = status.place
Here is a sample result output:
Tweet Text: @aranone aran tu eres el mejor soy tu fanatico 1 me gusta tu musica.hey pana sique asi q vay bn te deseo lo mejor bro)
Time Stamp: 2013-05-30 23:36:38
Time Stamp: 2013-05-30 23:36:38
Source: web
Author: juandvd_96
Name: juan David Romero
Time Zone: Atlantic Time (Canada)
User Language: es
Followers: 365
User Description: hola soy juan david... soy una chico muy enamorado... y soy muy fekiz...
Geo Enabled: True
Friends: 1857
Retweets: 0
Location: veezuela maracaibo
ID: 481513551
Coordinates: None
Place: None
cheers, BD
Thanks for clarifying. I was checking out the listener just now and noticed a tweet where coordinates were populated but as a json object. I am writing tweets to a mysql db as they are streamed and it seems like the one with the coordinates info was not inserted into the database. Not sure if the errors around the SQL statement are for the first or second tweet, both columns where the error occurred are set to 'varchar' values. Here is the streaming result:
Tweet Text: Vi 10 minutos y no pude ver mas. Soy super cagona, dios. Vay a ver otra.
Time Stamp: 2013-06-04 01:08:57
Time Stamp: 2013-06-04 01:08:57
Source: web
Author: ailenvalli
Name: Λili
Time Zone: Santiago
User Language: es
Followers: 384
User Description: Create your reality or it will be created for you
http://instagram.com/ailenvalli
Geo Enabled: True
Friends: 338
Retweets: 0
Location: 704 East Broadway ▲ 1966
ID: 200264965
Coordinates: None
Place: None
firehose_geo.py:87: Warning: Incorrect string value: '\xCE\x9Bili' for column 'Name' at row 1
(text,status.created_at,status.created_at,source,author,name,time_zone,user_language,followers,user_description,geo_enabled,friends,retweets,location,user_id,coordinates,geo)) firehose_geo.py:87: Warning: Incorrect string value: '\xE2\x96\xB2 19...' for column 'Location' at row 1
(text,status.created_at,status.created_at,source,author,name,time_zone,user_language,followers,user_description,geo_enabled,friends,retweets,location,user_id,coordinates,geo))
Tweet Text: I have a feeling WalMart is fixing to take a chunk out of my wallet. Healthy food is so expensive.
Time Stamp: 2013-06-04 01:42:00
Time Stamp: 2013-06-04 01:42:00
Source: Twitter for Android
Author: KaylaRenae21
Name: †Kayla Renae'
Time Zone: Central Time (US & Canada)
User Language: en
Followers: 300
User Description: The things I like to do cannot be found in the city. Hand me a fishing pole & I'll be gone all day.
Geo Enabled: True
Friends: 437
Retweets: 0
Location: Oklahoma
ID: 282414509
Coordinates: {'type': 'Point', 'coordinates': [-96.6623549, 34.7918959]}
Place: {'type': 'Point', 'coordinates': [34.7918959, -96.6623549]}
The problem is not related to tweepy
itself.
For example, see this tweet (https://api.twitter.com/1/statuses/show.json?id=341458303064354817&include_entities=true) - it has geo_enabled
set to true while geo
, coordinates
and place
equal to null
.
According to twitter docs:
geo_enabled: When true, indicates that the user has enabled the possibility of geotagging their Tweets.
So, it's not a strict rule that there will be location info in the tweet data if geo_enabled
is true. Just check if status.geo
or status.coordinates
are not None
in your listener.
Hope that helps.