Getting closure-compiler and Node.js to play nice

Question

Are there any projects that used node.js and closure-compiler (CC for short) together?

The official CC recommendation is to compile all code for an application together, but when I compile some simple node.js code which contains a require("./MyLib.js"), that line is put directly into the output, but it doesn't make any sense in that context.

I see a few options:

1. Code the entire application as a single file. This solves the problem by avoiding it, but is bad for maintenance.
2. Assume that all files will be concatenated before execution. Again this avoids the problem, but makes it harder to implement a un-compiled debug mode.
3. I'd like to get CC to "understand" the node.js require() function, but that probably can't be done without editing the compiler itself, can it?

Answer 1

I have been using the Closure Compiler with Node for a project I haven't released yet. It has taken a bit of tooling, but it has helped catch many errors and has a pretty short edit-restart-test cycle.

First, I use plovr (which is a project that I created and maintain) in order to use the Closure Compiler, Library, and Templates together. I write my Node code in the style of the Closure Library, so each file defines its own class or collection of utilities (like goog.array).

The next step is to create a bunch of externs files for the Node functions you want to use. I published some of these publicly at:

https://github.com/bolinfest/node-google-closure-latitude-experiment/tree/master/externs/node/v0.4.8

Though ultimately, I think that this should be a more community driven thing because there are a lot of functions to document. (It's also annoying because some Node functions have optional middle arguments rather than last arguments, making the type annotations complicated.) I haven't started this movement myself because it's possible that we could do some work with the Closure Complier to make this less awkward (see below).

Say you have created the externs file for the Node namespace http. In my system, I have decided that anytime I need http, I will include it via:

var http = require('http');

Though I do not include that require() call in my code. Instead, I use the output-wrapper feature of the Closure Compiler the prepend all of the require()s at the start of the file, which when declared in plovr, in my current project looks like this:

"output-wrapper": [
  // Because the server code depends on goog.net.Cookies, which references the
  // global variable "document" when instantiating goog.net.cookies, we must
  // supply a dummy global object for document.
  "var document = {};\n",

  "var bee = require('beeline');\n",
  "var crypto = require('crypto');\n",
  "var fs = require('fs');\n",
  "var http = require('http');\n",
  "var https = require('https');\n",
  "var mongodb = require('mongodb');\n",
  "var nodePath = require('path');\n",
  "var nodeUrl = require('url');\n",
  "var querystring = require('querystring');\n",
  "var SocketIo = require('socket.io');\n",
  "%output%"
],

In this way, my library code never calls Node's require(), but the Compiler tolerates the uses of things like http in my code because the Compiler recognizes them as externs. As they are not true externs, they have to be prepended as I described.

Ultimately, after talking about this on the discussion list, I think the better solution is to have a new type annotation for namespaces that would look something like:

goog.scope(function() {

    /** @type {~NodeHttpNamesapce} */
    var http = require('http');

    // Use http throughout.

});

In this scenario, an externs file would define the NodeHttpNamespace such that the Closure Compiler would be able to typecheck properties on it using the externs file. The difference here is that you could name the return value of require() whatever you wanted because the type of http would be this special namespace type. (Identifying a "jQuery namespace" for $ is a similar issue.) This approach would eliminate the need to name your local variables for Node namespaces consistently, and would eliminate the need for that giant output-wrapper in the plovr config.

But that was a digression...once I have things set up as described above, I have a shell script that:

1. Uses plovr to build everything in RAW mode.
2. Runs node on the file generated by plovr.

Using RAW mode results in a large concatenation of all the files (though it also takes care of translating Soy templates and even CoffeeScript to JavaScript). Admittedly, this makes debugging a pain because the line numbers are nonsense, but has been working well enough for me so far. All of the checks performed by the Closure Compiler have made it worth it.