How to run a python script like pm2 for nodejs

toy picture toy · Aug 20, 2015 · Viewed 51.4k times · Source

I've used pm2 for my Node.js script and I love it.
Now I have a python script which collect streaming data on EC2. Sometimes the script bombs out and I would like a process manager to restart itself like pm2.

Is there something the same as pm2 for python? I've been searching around and couldn't find anything.

Here's my error

  File "/usr/local/lib/python2.7/dist-packages/tweepy/streaming.py", line 430, in filter
    self._start(async)
  File "/usr/local/lib/python2.7/dist-packages/tweepy/streaming.py", line 346, in _start
    self._run()
  File "/usr/local/lib/python2.7/dist-packages/tweepy/streaming.py", line 286, in _run
    raise exception
AttributeError: 'NoneType' object has no attribute 'strip'
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:90:

It's a simple data collecting script

class StdOutListener(StreamListener):

    def on_data(self, data):
        mydata = json.loads(data)
        db.raw_tweets.insert_one(mydata)
        return True

    def on_error(self, status):
        mydata = json.loads(status)
        db.error_tweets.insert_one(mydata)


if __name__ == '__main__':

    #This handles Twitter authetification and the connection to Twitter Streaming API
    l = StdOutListener()
    auth = OAuthHandler(consumer_key, consumer_secret)
    auth.set_access_token(access_token, access_token_secret)
    stream = Stream(auth, l)

    #This line filter Twitter Streams to capture data by the keywords: 'python', 'javascript', 'ruby'
    stream.filter(follow=[''])

That I would like it to just restart itself in case something happens.

Answer

Simon Smith picture Simon Smith · Dec 9, 2015

You can actually run python scripts from within pm2:

pm2 start echo.py

If the script ends in a .py suffix it will use a python interpreter by default. If your filename doesn't end in .py you can do:

pm2 start echo --interpreter=python

I've found you have to be a little bit careful which python you are using, especially if you are using a virtualenv python with a different version to the 'default' python on your machine.