Wildfly 8.2/undertow read time out

Sannu picture Sannu · Mar 27, 2015 · Viewed 11.6k times · Source

I recently migrated my project from jboss4 to wildfly 8.2 with java1.8. I have a webservice call using SAAJ which runs fine in command line. But when its run from within wildfly8.2, it times out after 60 seconds. I read from jboss forums that read requests have a default timeout of 60 seconds. So i changed my configuration in standalone.xml to

            <ajp-listener name="ajp" socket-binding="ajp" max-parameters="10000"/>
            **<http-listener name="default" socket-binding="http" max-parameters="10000" read-timeout="120000"/>**
            <host name="default-host" alias="localhost">
                <location name="/" handler="welcome-content"/>
                <filter-ref name="server-header"/>
                <filter-ref name="x-powered-by-header"/>
            </host>

But it still times out after 60 seconds with following errors.

Caused by: java.net.SocketTimeoutException: SocketTimeoutException invoking http://test-server/test/v2.0.0/TestService?wsdl: Read timed out
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) [rt.jar:1.8.0_25]
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) [rt.jar:1.8.0_25]
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) [rt.jar:1.8.0_25]
    at java.lang.reflect.Constructor.newInstance(Constructor.java:408) [rt.jar:1.8.0_25]
    at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.mapException(HTTPConduit.java:1347)
    at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.close(HTTPConduit.java:1331)
    at org.apache.cxf.transport.AbstractConduit.close(AbstractConduit.java:56)
    at org.apache.cxf.transport.http.HTTPConduit.close(HTTPConduit.java:632)
    at org.jboss.wsf.stack.cxf.saaj.SOAPConnectionImpl.call(SOAPConnectionImpl.java:120)
    ... 38 more
Caused by: java.net.SocketTimeoutException: Read timed out
    at java.net.SocketInputStream.socketRead0(Native Method) [rt.jar:1.8.0_25]
    at java.net.SocketInputStream.read(SocketInputStream.java:150) [rt.jar:1.8.0_25]
    at java.net.SocketInputStream.read(SocketInputStream.java:121) [rt.jar:1.8.0_25]
    at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) [rt.jar:1.8.0_25]
    at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) [rt.jar:1.8.0_25]
    at java.io.BufferedInputStream.read(BufferedInputStream.java:345) [rt.jar:1.8.0_25]
    at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:703) [rt.jar:1.8.0_25]
    at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:647) [rt.jar:1.8.0_25]
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1534) [rt.jar:1.8.0_25]
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1439) [rt.jar:1.8.0_25]
    at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480) [rt.jar:1.8.0_25]
    at org.apache.cxf.transport.http.URLConnectionHTTPConduit$URLConnectionWrappedOutputStream.getResponseCode(URLConnectionHTTPConduit.java:266)
    at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.handleResponseInternal(HTTPConduit.java:1545)
    at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.handleResponse(HTTPConduit.java:1515)
    at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.close(HTTPConduit.java:1318)

I read here that i could set timeouts but I don't have to as the program runs fine without timing out from command line so its not saaj timeout issue. I am pretty sure wildfly/undertow is timing the socket read out for some reason.

Any help is appreciated.

---More details---

Currently I am using undertow 1.1 Final that came with wildfly8.2. I tried upgrading undertow to 1.2 beta, still same result.

Call that fails:

        responseMsg = soapConn.call(soapMessage, wsdlLoc);

Undertow configuration in wildfly8.2 :

    <subsystem xmlns="urn:jboss:domain:undertow:1.2">
        <buffer-cache name="default"/>
        <server name="default-server">
            <ajp-listener name="ajp" socket-binding="ajp" max-parameters="10000"/>
            <http-listener name="default" socket-binding="http" max-parameters="10000" read-timeout="120000"/>
            <host name="default-host" alias="localhost">
                <location name="/" handler="welcome-content"/>
                <filter-ref name="server-header"/>
                <filter-ref name="x-powered-by-header"/>
            </host>
        </server>
        <servlet-container name="default">
            <jsp-config/>
            <websockets/>
        </servlet-container>
        <handlers>
            <file name="welcome-content" path="${jboss.home.dir}/welcome-content"/>
        </handlers>
        <filters>
            <response-header name="server-header" header-name="Server" header-value="WildFly/8"/>
            <response-header name="x-powered-by-header" header-name="X-Powered-By" header-value="Undertow/1"/>
        </filters>
    </subsystem>

Even more details:

I tried this under wildfly9.0BETA2 and same result. Just want to share more details if it helps. SAAJ webservice call is made from a servlet that is running in wildfly8.2 and target WSDL is on another jboss server. So basically, client webservice call from wildlfy times out in 60 seconds, but If I run same call from a standalone java client and same code works just fine. I have even opened a thread on jboss community and am yet to hear any

Answer

Sannu picture Sannu · Apr 1, 2015

I was able to fix this by changing receive timeout in apache-cxf source and rebuilding it for wildfly8.2

Brief instructions: (versions have to be exactly these otherwise compile fails).

  1. Download CXF2.7.15 src from apache
  2. Download jdk1.6 latest release
  3. Download 3.0.4
  4. Export JAVA_HOME=path-to-jdk1.6
  5. Add jdk1.6/bin and maven/bin to PATH
  6. export MAVEN_OPTS=-Xmx512m to fix permgen errors
  7. Open ./rt/transports/http/src/main/resources/schemas/wsdl/http-conf.xsd and change ReceiveTimeout from 60000 to whatever needed for eg. 600000 (10 minutes)
  8. Run mvn -Pfastinstall

File that has HTTPConduit socket timeout is HTTPClientPolicy.java which is in ./rt/transports/http/target/cxf-rt-transports-http-2.7.15.jar. Copy this jar into wildfly8.2 modules under apache/cxf/impl/main folder. And also, edit module.xml to use this jar.

I also had to change undertow read-timeout settings in standalone.xml to higher value to stop it from reattempting the request.

Hope this helps.