Is there a workaround to open URLs containing underscores in Ruby?

ruby open-uri
Arty picture Arty · Mar 6, 2011 · Viewed 25.1k times · Source

I'm using open-uri to open URLs. 

resp = open("http://sub_domain.domain.com")

If it contains underscore I get an error:

URI::InvalidURIError: the scheme http does not accept registry part: sub_domain.domain.com (or bad hostname?)

I understand that this is because according to RFC URLs can contain only letters and numbers. Is there any workaround?

Answer

stef picture stef · Mar 6, 2011

This looks like a bug in URI, and uri-open, HTTParty and many other gems make use of URI.parse.

Here's a workaround:

require 'net/http'
require 'open-uri'

def hopen(url)
  begin
    open(url)
  rescue URI::InvalidURIError
    host = url.match(".+\:\/\/([^\/]+)")[1]
    path = url.partition(host)[2] || "/"
    Net::HTTP.get host, path
  end
end

resp = hopen("http://dear_raed.blogspot.com/2009_01_01_archive.html")

Related questions

In Ruby/Rails, how can I encode/escape special characters in URLs?

How do I encode or 'escape' the URL before I use OpenURI to open(url)? We're using OpenURI to open a remote url and return the xml: getresult = open(url).read The problem is the URL contains some user-input text …

Read More
Retrieve contents of URL as string

For tedious reasons to do with Hpricot, I need to write a function that is passed a URL, and returns the whole contents of the page as a single string. I'm close. I know I need to use OpenURI, and …

Read More
Ruby's open-uri and cookies

I would like to store the cookies from one open-uri call and pass them to the next one. I can't seem to find the right docs for doing this. I'd appreciate it if you could tell me the right way …

Read More