SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed ONLY WHEN PROXYING

Question

This post nearly duplicates a number of other posts, including Rails 4 and Ruby 2 Net/HTTP SSL Request: OpenSSL::SSL::SSLError: SSL_connect returned=1 errno=0 state=SSLv2/v3 read server hello A: unknown protocol and SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed, but with one key difference: the "certificate verify failed" error only happens when proxied via an SSL proxy (Charles), and only in the latest version of Ruby.

Specifically, this (non-proxied) code works:

  uri = URI.parse('https://www.ibm.com')
  http = Net::HTTP.new(uri.host, uri.port, nil)
  http.use_ssl = true
  http.start { |agent| agent.get(uri.path) }

and this (proxied) code fails with a "certificate verify failed" error:

  uri = URI.parse('https://www.ibm.com')
  http = Net::HTTP.new(uri.host, uri.port, "localhost", 8888)
  http.use_ssl = true
  http.start { |agent| agent.get(uri.path) }

the puzzle

The error ONLY appears when going through the proxy in the current versions of Ruby/OpenSSL. In particular:

• There's no error when NOT going through the proxy, which leads me to think that the local certs are valid.
• There's no error when going through the proxy with Firefox or curl -v https://www.ibm.com/, which leads me to think that the Charles certs are valid.
• There's no error going through the proxy on an earlier version of Ruby / OpenSSL (see below for the two environments).

All this makes me suspect an actual Ruby bug rather than a problem with my certs, but I'd be happy to be shown otherwise.

the current environment

I'm observing the error with this environment:

$ system_profiler SPSoftwareDataType | grep "System Version"
    System Version: OS X 10.8.5 (12F45)
$ ruby --version ; openssl version ; gem list | grep http
ruby 2.0.0p247 (2013-06-27) [x86_64-darwin12.4.0]
OpenSSL 1.0.1e 11 Feb 2013
http-cookie (1.0.2)
http_router (0.10.2)
net-http-digest_auth (1.4)
net-http-persistent (2.9)
ntlm-http (0.1.1)

the previous environment

The error does not appear with a slightly older version of ruby/OpenSSL:

$ ruby --version ; openssl version ; gem list | grep http
ruby 2.0.0p0 (2013-02-24 revision 39474) [x86_64-darwin10.8.0]
OpenSSL 0.9.8y 5 Feb 2013
http_router (0.11.0, 0.10.2)
httpauth (0.2.0)
net-http-digest_auth (1.2.1)
net-http-persistent (2.8)
ntlm-http (0.1.1)

Answer 1

What is going on here is that Charles cannot provide you with a valid certificate for "https://www.ibm.com", because whatever certificate you set up Charles with (or if Charles creates its own certificate) it cannot be for the "www.ibm.com" name AND certified by a trusted certificate authority. That's why the connection fails if the client verifies the the certificate, and works if you skip verification with

http.verify_mode = OpenSSL::SSL::VERIFY_NONE

It probably worked with the older version of ruby because "net/https" was adjusted to verify certificates by default only recently.