Long Polling in Python with Flask

Question

I'm trying to do long polling with JQuery and Python under the Flask Framework.

Having done long polling before in PHP, I've tried to go about it in the same way:

A script/function that has a while(true) loop, checking for changes periodically eg.every 0,5 seconds in the database, and returns some data when a change occurs.

So in my ini.py I've created an app.route to /poll for JQuery to call. JQuery gives it some information about the client's current state, and the poll() function compares this with what's currently in the database. The loop is ended and returns information when a change is observed.

Here's the python code:

@app.route('/poll')
def poll():
client_state = request.args.get("state")

    #remove html encoding + whitesapce from client state
    html_parser = HTMLParser.HTMLParser()
    client_state = html_parser.unescape(client_state)
    client_state = "".join(client_state.split())

    #poll the database
    while True:
        time.sleep(0.5)
        data = get_data()
        json_state = to_json(data)
        json_state = "".join(data) #remove whitespace

        if json_state != client_state:
            return "CHANGE"

The problem is that, when the code above starts polling, the server appears to be overloaded and other Ajax calls, and other requests like loading a "loading" image to the html using JQuery are unresponsive and timeout.

For completion's sake I've included the JQuery here:

function poll() {

queryString = "state="+JSON.stringify(currentState);

$.ajax({
    url:"/poll",
    data: queryString,
    timeout: 60000,
    success: function(data) {
        console.log(data);
        if(currentState == null) {
            currentState = JSON.parse(data);
        }
        else {
            console.log("A change has occurred");
        }

        poll();

    },
    error: function(jqXHR, textStatus, errorThrown) {

        console.log(jqXHR.status + "," + textStatus + ", " + errorThrown);

        poll();

    }
});

}

Does this need to multi-threaded or something? Or does anyone have any idea why I'm experiencing this behavior?

Thanks in advance!! :)

Answer 1

Just as the link @Robᵩ mentioned, you flask app is just overload. That's because a flask app is in single threading mode by default when running with app.run(), so it can only serve one request per time.

You can start multi threading with:

if __name__ == '__main__':
    app.run(threaded=True)

Or using a WSGI server like gunicorn or uwsgi to serve flask with multi processing:

gunicorn -w 4 myapp:app

Hopes you're enjoying with Python and Flask!