Error: connect EADDRNOTAVAIL while processing big async loop

Question

I am experiencing a very strange problem. I am importing some big xml-files and store them into mongoDB. The algorythm is a typical async loop:

doLoop = function( it, callback_loop ) {
    if( it < no_of_items ) {
        storeToMongo( ..., function( err, result ) {
                ...
                doLoop( it+1, callback_loop );
        });
    } else {
        callback_loop();
    }
};
doLoop( 0, function() {
    ...
});

Now (suddenly without any remarkable change in the code) I get the following error while performing the loop:

events.js:72
        throw er; // Unhandled 'error' event
              ^
Error: connect EADDRNOTAVAIL
    at errnoException (net.js:901:11)
    at connect (net.js:764:19)
    at net.js:842:9
    at dns.js:72:18
    at process._tickCallback (node.js:415:13)

The error happens after approximately a minute. The number of items processed in the meantime is always quite the same, but not exactly.

I tried to find out what connect/net causes the error, but I am lost. There is not socket-connection in my code. I have a connection to redis, but that is o.k. Is it the mongoDB-connection? But why does it get lost suddenly?

The only thing that helps to run through the whole loop is to perform the recursive loop call within the mongo-callback like this:

setTimeout( function() {
    doLoop( it+1, callback_loop );
},1);

Anyone out there who has an idea what is going wrong here?

Thanks, heinob

Answer 1

Finally I have found the answer. It is a problem within the default global http agent. See a full description here.