Best Practice to Use HttpClient in Multithreaded Environment

Question

For a while, I have been using HttpClient in a multithreaded environment. For every thread, when it initiates a connection, it will create a completely new HttpClient instance.

Recently, I have discovered that, by using this approach, it can cause the user to have too many ports being opened, and most of the connections are in TIME_WAIT state.

http://www.opensubscriber.com/message/[email protected]/86045.html

Hence, instead of each thread doing :

HttpClient c = new HttpClient();
try {
    c.executeMethod(method);
}
catch(...) {
}
finally {
    method.releaseConnection();
}

We plan to have :

[METHOD A]

// global_c is initialized once through
// HttpClient global_c = new HttpClient(new MultiThreadedHttpConnectionManager());

try {
    global_c.executeMethod(method);
}
catch(...) {
}
finally {
    method.releaseConnection();
}

In a normal situation, global_c will be accessed by 50++ threads concurrently. I was wondering, will this create any performance issues? Is MultiThreadedHttpConnectionManager using a lock-free mechanism to implement its thread safe policy?

If 10 threads are using global_c, will the other 40 threads be locked?

Or would it be better if, in every thread, I create an instance of an HttpClient, but release the connection manager explicitly?

[METHOD B]

MultiThreadedHttpConnectionManager connman = new MultiThreadedHttpConnectionManager();
HttpClient c = new HttpClient(connman);
try {
      c.executeMethod(method);
}
catch(...) {
}
finally {
    method.releaseConnection();
    connman.shutdown();
}

Will connman.shutdown() suffer performance issues?

May I know which method (A or B) is better, for application using an 50++ threads?

Answer 1

Definitely Method A because its pooled and thread safe.

If you are using httpclient 4.x, the connection manager is called ThreadSafeClientConnManager. See this link for further details (scroll down to "Pooling connection manager"). For example:

    HttpParams params = new BasicHttpParams();
    SchemeRegistry registry = new SchemeRegistry();
    registry.register(new Scheme("http", PlainSocketFactory.getSocketFactory(), 80));
    ClientConnectionManager cm = new ThreadSafeClientConnManager(params, registry);
    HttpClient client = new DefaultHttpClient(cm, params);