Allow insecure HTTPS connection for Java JDK 11 HttpClient

Question

Sometimes it is needed to allow insecure HTTPS connections, e.g. in some web-crawling applications which should work with any site. I used one such solution with old HttpsURLConnection API which was recently superseded by the new HttpClient API in JDK 11. What is the way to allow insecure HTTPS connections (self-signed or expired certificate) with this new API?

UPD: The code I tried (in Kotlin but maps directly to Java):

    val trustAllCerts = arrayOf<TrustManager>(object: X509TrustManager {
        override fun getAcceptedIssuers(): Array<X509Certificate>? = null
        override fun checkClientTrusted(certs: Array<X509Certificate>, authType: String) {}
        override fun checkServerTrusted(certs: Array<X509Certificate>, authType: String) {}
    })

    val sslContext = SSLContext.getInstance("SSL")
    sslContext.init(null, trustAllCerts, SecureRandom())

    val sslParams = SSLParameters()
    // This should prevent host validation
    sslParams.endpointIdentificationAlgorithm = ""

    httpClient = HttpClient.newBuilder()
        .sslContext(sslContext)
        .sslParameters(sslParams)
        .build()

But on sending I have exception (trying on localhost with self-signed certificate):

java.io.IOException: No name matching localhost found

Using IP address instead of localhost gives "No subject alternative names present" exception.

After some debugging of JDK I found that sslParams are really ignored in the place the exception is thrown and some locally created instance is used. Further debugging revealed that the only way to affect the hostname verification algorithm is setting jdk.internal.httpclient.disableHostnameVerification system property to true. And that's seems to be a solution. SSLParameters in the code above have no effect so this part can be discarded. Making it configurable only globally looks like serious design flaw in new HttpClient API.

Answer 1

As suggested already you need an SSLContext which ignores the bad certificates. The exact code which obtains the SSLContext in one of the links in the question should work by basically creating a null TrustManager which doesn't look at the certs:

private static TrustManager[] trustAllCerts = new TrustManager[]{
    new X509TrustManager() {
        public java.security.cert.X509Certificate[] getAcceptedIssuers() {
            return null;
        }
        public void checkClientTrusted(
            java.security.cert.X509Certificate[] certs, String authType) {
        }
        public void checkServerTrusted(
            java.security.cert.X509Certificate[] certs, String authType) {
        }
    }
};

public static  void main (String[] args) throws Exception {
    SSLContext sslContext = SSLContext.getInstance("TLS");
    sslContext.init(null, trustAllCerts, new SecureRandom());

    HttpClient client = HttpClient.newBuilder()
        .sslContext(sslContext)
        .build();

The problem with the above is obviously that server authentication is disabled completely for all sites. If there were only one bad certificate then you could import it into a keystore with:

keytool -importcert -keystore keystorename -storepass pass -alias cert -file certfile

and then initialize the SSLContext using an InputStream reading the keystore as follows:

char[] passphrase = ..
KeyStore ks = KeyStore.getInstance("PKCS12");
ks.load(i, passphrase); // i is an InputStream reading the keystore

KeyManagerFactory kmf = KeyManagerFactory.getInstance("PKIX");
kmf.init(ks, passphrase);

TrustManagerFactory tmf = TrustManagerFactory.getInstance("PKIX");
tmf.init(ks);

sslContext = SSLContext.getInstance("TLS");
sslContext.init(kmf.getKeyManagers(), tmf.getTrustManagers(), null);

Either of the above solutions will work for a self-signed certificate. A third option is in the case where the server provides a valid, non self-signed certificate but for a host which does not match any of the names in the certificate it provides, then a system property "jdk.internal.httpclient.disableHostnameVerification" can be set to "true" and this will force the certificate to be accepted in the same way that the HostnameVerifier API was used previously. Note, that in normal deployments it isn't expected that any of these mechanisms would be used, as it should be possible to automatically verify the certificate supplied by any correctly configured HTTPS server.