Make timeout work for LWP::UserAgent HTTPS

Question

Solution

As reported by @limulus in the answer I accepted, this was a bug in Net::HTTPS version 6.00. Always be wary of fresh .0 releases. Here's the relevant diff between the buggy and fixed version of that module:

D:\Opt\Perl512.32 :: diff lib\Net\HTTPS.pm site\lib\Net\HTTPS.pm
6c6
< $VERSION = "6.00";
---
> $VERSION = "6.02";
75,78c75,80
< # The underlying SSLeay classes fails to work if the socket is
< # placed in non-blocking mode.  This override of the blocking
< # method makes sure it stays the way it was created.
< sub blocking { }  # noop
---
> if ($SSL_SOCKET_CLASS eq "Net::SSL") {
>     # The underlying SSLeay classes fails to work if the socket is
>     # placed in non-blocking mode.  This override of the blocking
>     # method makes sure it stays the way it was created.
>     *blocking = sub { };
> }

Original question

Relevance: It is annoying to see your HTTPS client block indefinitely because the connection endpoint is unreliable.

This experiment is easy to set up and replay at home. You just need two things, a tarpit to trap an incoming client, and a Perl script. The tarpit can be set up using netcat:

nc -k -l localhost 9999 # on Linux, for multiple requests
nc -l -p 9999 localhost # on Cygwin, for one request only

Then, point the script to this tarpit:

use strict;
use LWP::UserAgent;
use HTTP::Request::Common;

print 'LWP::UserAgent::VERSION  ', $LWP::UserAgent::VERSION, "\n";
print 'IO::Socket::SSL::VERSION ', $IO::Socket::SSL::VERSION, "\n";

my $ua = LWP::UserAgent->new( timeout => 5, keep_alive => 1 );
$ua->ssl_opts( timeout => 5, Timeout => 5 ); # Yes - see note below!
my $rsp = $ua->request( GET 'https://localhost:9999' );
if ( $rsp->is_success ) {
  print $rsp->as_string;
} else {
  die $rsp->status_line;
}

What is this going to do? Well, connect to the port opened by NetCat, and then ... hang. Indefinitely. At least in terms of developer time. I mean it might time out after ten minutes or two hours, but I haven't checked; the specified timeout doesn't take effect, not on Linux, and not on Windows (Win32, haven't checked Cygwin).

Versions used:

LWP::UserAgent::VERSION  6.02
IO::Socket::SSL::VERSION 1.44
# on Linux

LWP::UserAgent::VERSION  6.02
IO::Socket::SSL::VERSION 1.44
# on Win32

Now for the timeout and Timeout parameters. The former is the name of the parameter for LWP::UA, the latter is the name for IO::Socket::SSL, used via LWP::Protocol::https. (Incidentally, why is metacpan HTTPS? Well, at least it's not a tarpit.) I am somehow hoping to have these parameters passed along :)

Just so you know, keep_alive doesn't have anything to do with the timeout not working, I verified that empirically. :)

Anyway, before digging deeper, does anyone know what's going on here and how to make the timeout work with HTTPS? Hard to believe I'm the first person running into this.

Answer 1

This is a result of the Net::HTTPS module overriding the blocking method of IO::Socket with a noop. Upgrading to the latest Net::HTTP package should fix this.