Http error 429 java handling?
I have a java program that searches a site for updates. It checks and than sleeps for a period of time set by user.
During some testing I set the sleep time to 0 secs, forgot about it and it over-requested (over-spammed?, don't know the terminology) the site. I wouldn't have noticed this if I didn't go to this site and manually checked for something.
The error I get is 429 : too many requests. I've searched the web and found out that the site should send me when is it ok to try again (retry-after header). The problem is that I either don't get this information or can't find it.
I've viewed the source via chrome and didn't find the information. Then I used getErrorStrem and didn't find the information.
In another question on stackoverflow it is said that
if everything is set up properly, you will also have received a "Retry-after" header along with the 429 response
My knowledge of web technologies is rather limited. The program I wrote looked for patterns that I noticed viewing the source of page. My java knowledge is better, but not great.
So how do I get retry-after info? About 20 hours passed and when I tried again I got the same error.
Answer
As specified in the rfc 6585:
The 429 status code indicates that the user has sent too many requests in a given amount of time ("rate limiting").
The response representations SHOULD include details explaining the condition, and MAY include a Retry-After header indicating how long to wait before making a new request.
This specific header is only optional. So if you don't see it in the headers that's because it isn't there.
Including the condition in the response representation is only a recommandation (and so not mandatory)
There is no rules regarding this limit: it depends on the server configuration.
For exact meaning of verbs commonly used in RFCs : you can read this