What is the proper way to use the node.js postgresql module?

Question

I am writing a node.js app on Heroku and using the pg module. I can't figure out the "right" way to get a client object for each request that I need to query the database.

The documentation uses code like this:

pg.connect(conString, function(err, client) {
  // Use the client to do things here
});

But surely you don't need to call pg.connect inside every function that uses the database right? I've seen other code that does this:

var conString = process.env.DATABASE_URL || "tcp://postgres:1234@localhost/postgres";
var client = new pg.Client(conString);
client.connect();
// client is a global so you can use it anywhere now

I am leaning toward the second option since I believe the free database instance for Heroku is limited to one connection anyway, but are there any drawbacks to doing it this way? Do I need to check if my client object is still connected every time before I use it?

Answer 1

I'm the author of node-postgres. First, I apologize the documentation has failed to make the right option clear: that's my fault. I'll try to improve it. I wrote a Gist just now to explain this because the conversation grew too long for Twitter.

Using pg.connect is the way to go in a web environment.

PostgreSQL server can only handle 1 query at a time per connection. That means if you have 1 global new pg.Client() connected to your backend your entire app is bottleknecked based on how fast postgres can respond to queries. It literally will line everything up, queuing each query. Yeah, it's async and so that's alright...but wouldn't you rather multiply your throughput by 10x? Use pg.connect set the pg.defaults.poolSize to something sane (we do 25-100, not sure the right number yet).

new pg.Client is for when you know what you're doing. When you need a single long lived client for some reason or need to very carefully control the life-cycle. A good example of this is when using LISTEN/NOTIFY. The listening client needs to be around and connected and not shared so it can properly handle NOTIFY messages. Other example would be when opening up a 1-off client to kill some hung stuff or in command line scripts.

One very helpful thing is to centralize all access to your database in your app to one file. Don't litter pg.connect calls or new clients throughout. Have a file like db.js that looks something like this:

module.exports = {
   query: function(text, values, cb) {
      pg.connect(function(err, client, done) {
        client.query(text, values, function(err, result) {
          done();
          cb(err, result);
        })
      });
   }
}

This way you can change out your implementation from pg.connect to a custom pool of clients or whatever and only have to change things in one place.

Have a look at the node-pg-query module that does just this.