Server returned HTTP response code: 503 for URL

Question

Getting "Exception in thread "main" java.io.IOException: Server returned HTTP response code: 503 for URL" error when I make multiple calls to Amazon Product Advertising API.

Is the reason overloading of the service? One Possible solution is to use Thread.Sleep(milliseconds) method.

But is there any other more sophisticated solution? Like proxies or something?

Here is the code used to make the connection:

URL amazon = new URL(url);
        URLConnection yc = amazon.openConnection();
        BufferedReader in = new BufferedReader(
                                new InputStreamReader(
                                yc.getInputStream()));

I am using Java!

Answer 1

From http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html, which defines these status codes:

10.5.4 503 Service Unavailable

The server is currently unable to handle the request due to a temporary overloading or maintenance of the server. The implication is that this is a temporary condition which will be alleviated after some delay. If known, the length of the delay MAY be indicated in a Retry-After header. If no Retry-After is given, the client SHOULD handle the response as it would for a 500 response.

Note: The existence of the 503 status code does not imply that a server must use it when becoming overloaded. Some servers may wish to simply refuse the connection.

So yes, server overload is a likely cause. To figure out how to deal with this you probably want to discuss it with whoever operates the service, to understand what they can tolerate. Then, if you search for "throttle web requests" on SO, you'll find a number of discussions of how to implement throttling once you know your requirement.

Edit

Now that I see you're talking about Amazon Advertising API, a quick search got me to this page: http://docs.aws.amazon.com/AWSECommerceService/latest/DG/TroubleshootingApplications.html

The Efficiency Guidelines on that page pretty clearly state what will trigger your problem, and the request rates that are acceptable.