Mongo DB pool connection error

Question

I am using mongoDb with java. I am getting the following error while inserting data. Any help would be appreciated. Thanx

Jul 4, 2012 1:45:32 PM org.xsocket.connection.HandlerAdapter performOnData
WARNING: [6f829b013850ff7914137a5cceC291] closing connection. Error occured by performing onData of com.avaya.onex.hss.requesthandlers.ResponseHandler#18746603 java.lang.OutOfMemoryError: Java heap space
Jul 4, 2012 1:45:25 PM com.mongodb.DBPortPool gotError
WARNING: emptying DBPortPool to localhost:27017 b/c of error
java.net.SocketException: Connection reset by peer: socket write error
    at java.net.SocketOutputStream.socketWrite0(Native Method)
    at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
    at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
    at org.bson.io.PoolOutputBuffer.pipe(PoolOutputBuffer.java:129)
    at com.mongodb.OutMessage.pipe(OutMessage.java:111)
    at com.mongodb.DBPort.go(DBPort.java:119)
    at com.mongodb.DBPort.go(DBPort.java:89)
    at com.mongodb.DBPort.say(DBPort.java:84)
    at com.mongodb.DBTCPConnector.say(DBTCPConnector.java:153)
    at com.mongodb.DBTCPConnector.say(DBTCPConnector.java:138)
    at com.mongodb.DBApiLayer$MyCollection.insert(DBApiLayer.java:261)
    at com.mongodb.DBApiLayer$MyCollection.insert(DBApiLayer.java:211)
    at com.mongodb.DBCollection.insert(DBCollection.java:57)
    at com.mongodb.DBCollection.insert(DBCollection.java:102)
    at com.avaya.onex.hss.requesthandlers.DatabaseConnection.dbWrite(DatabaseConnection.java:50)
    at com.avaya.onex.hss.requesthandlers.ResponseHandler.handleResponse(ResponseHandler.java:232)
    at com.avaya.onex.hss.requesthandlers.ResponseHandler.onData(ResponseHandler.java:104)
    at org.xsocket.connection.HandlerAdapter.performOnData(HandlerAdapter.java:242)
    at org.xsocket.connection.HandlerAdapter.access$200(HandlerAdapter.java:42)
    at org.xsocket.connection.HandlerAdapter$PerformOnDataTask.run(HandlerAdapter.java:210)
    at org.xsocket.SerializedTaskQueue.performPendingTasks(SerializedTaskQueue.java:161)
    at org.xsocket.SerializedTaskQueue.access$100(SerializedTaskQueue.java:40)
    at org.xsocket.SerializedTaskQueue$MultithreadedTaskProcessor.run(SerializedTaskQueue.java:189)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)

This is the class where I have dbWrite function which I call from other classes for inserting data in mongodb.

package com.avaya.onex.hss.requesthandlers;

import com.mongodb.BasicDBObject;
import com.mongodb.DB;
import com.mongodb.DBCollection;
import com.mongodb.Mongo;
import com.mongodb.MongoOptions;
import com.mongodb.WriteConcern;

public class DatabaseConnection {
    private static DatabaseConnection con = null;
    static DB db=null;
    static Mongo m;
    //private static int count=0;

    private DatabaseConnection(){
    }

    public void dbConnect() {

        try{
        // connect to the local database server
        MongoOptions options = new MongoOptions();
        options.connectionsPerHost = 9000000;
        options.maxWaitTime = 1000000;
        options.socketKeepAlive = true;
        options.threadsAllowedToBlockForConnectionMultiplier = 1000;

         m = new Mongo("localhost", options);

         //m.dropDatabase("ClientSimulator");
         db = m.getDB( "ClientSimulator" );

        }catch(Exception e){
            e.printStackTrace();
        }
    }

    public static synchronized DatabaseConnection getInstanceClass(){
        if(con==null)
            con=new DatabaseConnection();
        return con;
    }

    public DB getDatabaseObject(){
        return db;
    }

    public synchronized void dbWrite(DBCollection coll, BasicDBObject obj){
        coll.insert(obj);
        try {
            Thread.currentThread().sleep(100);
        } catch (InterruptedException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }
    }

    public synchronized void dbRead(DBCollection coll){
    }

    public synchronized void dbUpdate(DBCollection coll) {
    }

    public synchronized void dbCollDrop(DBCollection coll){
        coll.drop();
    }
}

Answer 1

Try this.

    options.connectionsPerHost = 100;
    options.maxWaitTime = 2000;
    options.socketKeepAlive = true;
    options.threadsAllowedToBlockForConnectionMultiplier = 50;

connectionsPerHost is the connectionPool. If you put 9000000 it will try to create 90000000 threads and connection object to handle your request. I believe that it is by far more than necessary. You have to reduce that number, and you might also remove the sleep(100) in your insert function.

If java tries to create that many connections. Mongodb server might not be able to handle that much connection or you'd have to change the config to allow that much connections.

Keep in mind that connectionsPerHost is not the maximum quantity of threads. You could have 1000000 threads inserting but only 100 connections will be used and will be shared for all the threads once they are free. maxWaitTime of 2 second should be more than enough. If an insert takes more than 2 seconds, you might consider upgrading your server.