How can I handle 503 errors with open-uri?
If a website returns a '503 service unavailable' error then open-uri throws an exception. For example:
require 'open-uri'
open('http://www.google.co.uk/sorry/?continue=http://www.google.co.uk/search%3Fq%3Dhello%26oq%3Dhello%26ie%3DUTF-8')
# OpenURI::HTTPError: 503 Service Unavailable
# ...
However if you then visit it in a web browser, it actually displays a page with a CAPTCHA and not an error.
How can I make sure open-uri doesn't just throw this as an exception but actually handles the response and provides me with the page content?
Answer
OpenURI::HTTPError have an io attribute you can inspect to get what you want. io is a StringIO object with several singleton methods defined on it (status for example):
require 'open-uri'
begin
open('http://www.google.co.uk/sorry/?continue=http://www.google.co.uk/search%3Fq%3Dhello%26oq%3Dhello%26ie%3DUTF-8')
rescue OpenURI::HTTPError => error
response = error.io
response.status
# => ["503", "Service Unavailable"]
response.string
# => <!DOCTYPE html PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\">\n<html DIR=\"LTR\">\n<head><meta http-equiv=\"content-type\" content=\"text/html; charset=utf-8\"><meta name=\"viewport\" content=\"initial-scale=1\">...
end
However for this task the Net::HTTP module is probably a better alternative:
require 'net/http'
response = Net::HTTP.get_response(URI.parse('http://www.google.co.uk/sorry/?continue=http://www.google.co.uk/search%3Fq%3Dhello%26oq%3Dhello%26ie%3DUTF-8'))
response.code
# => "503"
response.body
# => "<!DOCTYPE html PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\">\n<html DIR=\"LTR\">\n<head><meta http-equiv=\"content-type\" content=\"text/html; ...