nginx and unicorn bad gatewayconnect() to unix:/tmp/mobile.socket failed AND failed (111: Connection refused)
I am (trying) to set up an ubuntu 11.04 server on rackspace to run a rails 3.2 app with nginx and unicorn. I found this awesome blog http://techbot.me/2010/08/deployment-recipes-deploying-monitoring-and-securing-your-rails-application-to-a-clean-ubuntu-10-04-install-using-nginx-and-unicorn/ that has helped me massively and apart from mysql setup issues I think I have everything nailed except for a bad gateway error
The nginx error log shows
2012/02/25 14:38:34 [crit] 29139#0: *1 connect() to unix:/tmp/mobile.socket failed (2: No such file or directory) while connecting to upstream, client: xx.xx.xxx.xx, server: localhost, request: "GET / HTTP/1.1", upstream: "http://unix:/tmp/mobile.socket:/", host: xx.xx.xxx.xx
(I have x'd out the domains)
I guess this could be a user permissions thing but the file does not actually exist and I'm not sure how it should be created. I am reluctant to create it manually as I feel that doing so would be fixing a symptom rather than fixing the cause
It should also be noted that the user I created on the server has sudo permissions and needs to use sudo to start nginx, not sure if this is right? Any pointers as to what I should be looking for to fix this are greatly appreciated. For completeness my configuration files look like this /etc/init.dunicorn
#! /bin/sh
### BEGIN INIT INFO
# Provides: unicorn
# Required-Start: $local_fs $remote_fs $network $syslog
# Required-Stop: $local_fs $remote_fs $network $syslog
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: starts the unicorn web server
# Description: starts unicorn
### END INIT INFO
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
DAEMON=/usr/bin/unicorn_rails
DAEMON_OPTS="-c /home/testapp/mobile/current/unicorn.rb -E production -D"
NAME=unicorn_rails
DESC=unicorn_rails
PID=/home/testapp/mobile/shared/pids/unicorn.pid
case "$1" in
start)
echo -n "Starting $DESC: "
$DAEMON $DAEMON_OPTS
echo "$NAME."
;;
stop)
echo -n "Stopping $DESC: "
kill -QUIT `cat $PID`
echo "$NAME."
;;
restart)
echo -n "Restarting $DESC: "
kill -QUIT `cat $PID`
sleep 1
$DAEMON $DAEMON_OPTS
echo "$NAME."
;;
and the nginx configuration in /etc/nginx/sites-available/default
# as we are going to use Unicorn as the application server
# we are not going to use common sockets
# but Unix sockets for faster communication
upstream mobile {
# fail_timeout=0 means we always retry an upstream even if it failed
# to return a good HTTP response (in case the Unicorn master nukes a
# single worker for timing out).
# for UNIX domain socket setups:
server unix:/tmp/mobile.socket fail_timeout=0;
}
server {
# if you're running multiple servers, instead of "default" you should
# put your main domain name here
listen 80 default;
# you could put a list of other domain names this application answers
server_name localhost;
root /home/testapp/mobile/current/public;
access_log /var/log/nginx/mobile_access.log;
rewrite_log on;
location / {
#all requests are sent to the UNIX socket
proxy_pass http://mobile;
proxy_redirect off;
proxy_set_header Host $host:$proxy_port;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
client_max_body_size 10m;
client_body_buffer_size 128k;
proxy_connect_timeout 90;
proxy_send_timeout 90;
proxy_read_timeout 90;
proxy_buffer_size 4k;
proxy_buffers 4 32k;
proxy_busy_buffers_size 64k;
proxy_temp_file_write_size 64k;
}
# if the request is for a static resource, nginx should serve it directly
# and add a far future expires header to it, making the browser
# cache the resource and navigate faster over the website
location ~ ^/(images|javascripts|stylesheets|system)/ {
root /home/testapp/mobile/current/public;
expires max;
break;
}
}
UPDATE My unicorn.rb file
# See http://unicorn.bogomips.org/Unicorn/Configurator.html for complete
# documentation.
worker_processes 4
# Help ensure your application will always spawn in the symlinked
# "current" directory that Capistrano sets up.
working_directory "/home/testapp/mobile/current"
# listen on both a Unix domain socket and a TCP port,
# we use a shorter backlog for quicker failover when busy
listen "/tmp/mobile.socket", :backlog => 64
# nuke workers after 30 seconds instead of 60 seconds (the default)
timeout 30
# feel free to point this anywhere accessible on the filesystem
user 'testapp', 'testapp'
shared_path = '/home/testapp/mobile/shared'
pid "#{shared_path}/pids/unicorn.pid"
stderr_path "#{shared_path}/log/unicorn.stderr.log"
stdout_path "#{shared_path}/log/unicorn.stdout.log"
As per suggestion I have manually created the mobile.socket file and I now get the following error
[error] 1083#0: *4 connect() to unix:/tmp/mobile.socket failed (111: Connection refused) while connecting to upstream
Is this just a permissions thing on the mobile.socket file? If so what permissions should I need?
Update 2 nginx and unicorn both seem to be running ok
testapp@airmob:~/mobile/current$ ps aux | grep nginx
root 6761 0.0 0.1 71152 1224 ? Ss 18:36 0:00 nginx: master process /usr/sbin/nginx
testapp 6762 0.0 0.1 71492 1604 ? S 18:36 0:00 nginx: worker process
testapp 6763 0.0 0.1 71492 1604 ? S 18:36 0:00 nginx: worker process
testapp 6764 0.0 0.1 71492 1604 ? S 18:36 0:00 nginx: worker process
testapp 6765 0.0 0.1 71492 1604 ? S 18:36 0:00 nginx: worker process
testapp 13071 0.0 0.0 8036 600 pts/0 R+ 21:21 0:00 grep --color=auto nginx
Answer
I have renamed mobile.socket to mobile.sock in the relevant config files (unicorn.rb and nginx default) and all is good, no need to create any socket files, it just works as expected.
This also happens if the app server is not running (In my case unicorn). Unicorn creates the socket and nginx looks for it. If the socket is not there nginx kicks up a fuss, so if you are reading this looking for a solution make sure your app server (unicorn) is running and make sure all your socket names match in the various configuration files (unicorn.rb and whatever nginx.conf file has the socket mentioned in it)