Realtime server push with Socket IO (or Strophe.js), XMPP and Django

Question

I have a couple of Android and iOS native mobile application that I wrote which connect directly to an XMPP server that I host. They push and pull realtime data through XMPP. I also use some of the XMPP XEP extensions. For other operations, I have a django application running on the same server which all the mobile applications consume through an HTTP REST interface. I use Celery and Redis for the django side to do some operations asynchronously (like doing heavy batched writes to my db).

This all works fine and dandy. Yay.

But now I want to write a web front-end to all of this, so I started researching my options and well - there are so many ways to skin the cat that I wanted to check with the SO community first.

The idea to have a js library that gives me a unified API for socket communications (i.e try different implementations of web sockets or fall back to flash) appeals to me hence why I mention Socket IO. The idea of having to run a nodejs server, well, not so much (one more thing to learn), but if I have to, I definitely will. I know that some people use gevent as a replacement of the node server. Others, decide to write a small nodejs which they connect to the rest of their stack. I would probably do this.

Another option, is to use an js XMPP library like Strophe which I don't think has a flash fallback. Also, I would need to research what this means for my server.

I have read several of the Stackoverflow answer on how to do comet and django - hence why it seems that there are several options.

The question is:

If I want to have the advantage of Socket IO behavior (with the fallbacks) and I want to push realtime data to the web client (which is being fed to the server through XMPP), and use Django what is my best option?

Update: The XMPP server that I use is ejabberd, which also supports BOSH. I realize that I could use Strophe.js and thus my communication would go over a type of long polling http connection instead of websockets. As far as I can tell, there are some XMPP over Websockets open source library, but AFAIK the community is not as active as the SocketIO one.

Update 2: Browsers that I need to support are only modern browsers. I guess this means that Flash fallback will not be that important, which is leaning me towards strophe.js.

Answer 1

I think once you get your hands dirty with some node you'll find that straying from Node for socket.io is going to be much harder. There are very easy to use xmpp modules in node ready to go (see https://github.com/astro/node-xmpp). Remember, node is all javascript, so you're probably familiar with programming in it already.

Personally, I've had some memory leak issues using node 0.6 or higher. Node 0.4 worked without those issues. If you are new to github (as I was before playing with Node) here is how you would get going with a node server.

Getting Node

1. Login to your linux box and favorite directory (I'll assume /)
2. git clone https://github.com/joyent/node.git
3. cd /node
4. git tag -l (this will list all available version of node)
5. git checkout v0.6.16 (this will checkout 0.6.16 version of node, you could replace that with v0.4.12 for example if you have memory issues)
6. ./configure
7. make
8. make install

You'll need certain development tools to build it such as g++, but at this point you'll have a working node command.

Installing Node Modules like xmpp

Node has a nice amount of modules where most things have already been written for you. There is a search facility at http://search.npmjs.org or you can access all modules directly from your shell by using the npm command. NPM is nodes tool for installing and managing node modules. You can type npm search xmpp to search for all xmpp modules, for instance. To install a basic xmpp library for node you would do npm install node-xmpp. By the way, most github node module pages will include instructions on the front page readme file.

Keeping Node Running in Production

This threw me when I first started out. If you have any errors that are not caught node will simply die. So, you can either 1. Make sure there are no errors whatsoever or they are all caught (unlikely because even Node itself will error) 2. Use the uncaughtException handler to catch these problems. You would use code like this in your program

process.addListener("uncaughtException", function (err) {
    util.log("Uncaught exception: " + err);
    console.log(err.stack);
    console.log(typeof(this));
    // maybe email me?

});

Be Extra Safe and Use Forever

Even with the uncaughtException issue your program in production might die. Memory running out, segfaults, who knows what. That's where it pays to use something like the wonderful Node module called "Forever" (see https://github.com/nodejitsu/forever). You can type npm install forever -g to install forever. Note the -g option which puts forever in the GLOBAL node module directory. Without -g it puts the node module in the current working directory. You'll then be able to type something like (assuming your node program was called my_program.js) forever start my_program.js and then the Forever program will make sure that if it dies it gets restarted.