How to configure nginx + Unicorn to avoid timeout errors?

Question

I have a Rails (v3.2.13, Ruby 2.0.0) application running on nginx + Unicorn (Ubuntu 12.04). All is working well, except when an admin user is uploading users (thousands) via a CVS file. The problem is that I have set timeout to 30 seconds and the import process takes much more time. So, after 30 seconds I get an nginx 502 Bad Gateway page (Unicorn worker is killed).

The obvious solution is to increase timeout, but I don't want this because it'll cause another problems (I guess), because it's not a typical behavior.

Is there a way to handle this kind of problems?

Thanks a lot in advance.

PS: Maybe a solutions is to modify the code. If so, I want to avoid the user to perform another request.

Some ideas (don't know if possible):

• Setup a worker dedicated to this request.
• Send a "work in progress" signal to Unicorn to avoid to be killed.

nginx-app.conf

upstream xxx {
  server unix:/tmp/xxx.socket fail_timeout=0;
}


server {
  listen   80; 

  ...

  location / {

    proxy_pass  http://xxx;
    proxy_redirect     off;
    ...

    proxy_connect_timeout      360;
    proxy_send_timeout         360;
    proxy_read_timeout         360;
  }
}

unicorn.rb

worker_processes 2

listen "/tmp/xxx.socket"

timeout 30

pid "/tmp/unicorn.xxx.pid"

Answer 1

This is a good reason to create a queue. And you will:

• upload csv file (that should be within 30sec)
• your background job that will import user data (that can go for hours…)
• while this job is in progress you can serve some kind of WIP page with job status/percents/etc.

Check https://github.com/resque/resque for example. There is a lot of other queues.