HttpClient 403 error when using valid client certificate

Question

I'm trying to automate some tasks on a website using Java. I have a valid client for that website (works when I login using firefox), but I keep getting a 403 error when I try to login using http client. Note that I want my trust store to trust anything (I know it's not safe, but at this point I am not worried about that).

Here's my code:

    KeyStore keystore = getKeyStore();//Implemented somewhere else and working ok
    String password = "changeme";

    SSLContext context = SSLContext.getInstance("SSL");
    KeyManagerFactory kmfactory = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
    kmfactory.init(keystore, password.toCharArray());
    context.init(kmfactory.getKeyManagers(), new TrustManager[] { new X509TrustManager() {
        @Override
        public void checkClientTrusted(java.security.cert.X509Certificate[] arg0, String arg1)
                throws CertificateException {
        }
        @Override
        public void checkServerTrusted(java.security.cert.X509Certificate[] arg0, String arg1)
                throws CertificateException {
        }

        @Override
        public java.security.cert.X509Certificate[] getAcceptedIssuers() {
            return null;
        }
    } }, new SecureRandom());
    SSLSocketFactory sf = new SSLSocketFactory(context);

    Scheme httpsScheme = new Scheme("https", 443, sf);
    SchemeRegistry schemeRegistry = new SchemeRegistry();
    schemeRegistry.register(httpsScheme);
    ClientConnectionManager cm = new SingleClientConnManager(schemeRegistry);
    HttpClient client = new DefaultHttpClient(cm);

    HttpGet get = new HttpGet("https://theurl.com");
    HttpResponse response = client.execute(get);
    System.out.println(EntityUtils.toString(response.getEntity(), "UTF-8"));

The last statement prints out a 403 error. What am I missing here?

Answer 1

Figured it out own my own. The 403 error was getting was because Java SSL was not selecting my client certificate.

I debugged the SSL handshake and found that the server asked for a client certificate issued by a list of authorities and the issuer of my client certificate wasn't on that list. So Java SSL simply couldn't find an appropriate certificate on my keystore. It looks like web browsers and Java implement SSL a little differently since my browser actually asks me which certificate to use, no matter what the server certificate asks for in terms of issuers of client certificates.

In this case, the server certificate is to blame. It is self-signed and the list of issuers it informs as acceptable is incomplete. And that doesn't mix well with the Java SSL implementation. But the server isn't mine and there is nothing I can do about it, except for complaining about the brazilian government (their server). Without further due, here's my work around:

First, I used a TrustManager that trusts anything (like I did in my question):

public class MyTrustManager implements X509TrustManager {
    public void checkClientTrusted(X509Certificate[] chain, String authType) throws CertificateException {
    }
    public void checkServerTrusted(X509Certificate[] chain, String authType) throws CertificateException {
    }
    public X509Certificate[] getAcceptedIssuers() {
        return null;
    }
}

Then I implemented a key manager that always uses the key I want from a PKCS12 (.pfx) certificate:

public class MyKeyManager extends X509ExtendedKeyManager {

KeyStore keystore = null;
String password = null;
public MyKeyManager(KeyStore keystore, String password) {
        this.keystore = keystore;
        this.password = password;
}

@Override
public String chooseClientAlias(String[] arg0, Principal[] arg1, Socket arg2) {
    return "";//can't be null
}

@Override
public String chooseServerAlias(String arg0, Principal[] arg1, Socket arg2) {
    return null;
}

@Override
public X509Certificate[] getCertificateChain(String arg0) {
    try {
        X509Certificate[] result = new X509Certificate[keystore.getCertificateChain(keystore.aliases().nextElement()).length];
        for (int i=0; i < result.length; i++){
            result[i] = (X509Certificate) keystore.getCertificateChain(keystore.aliases().nextElement())[i];
        }
        return result ;
    } catch (Exception e) {
    }
    return null;
}

@Override
public String[] getClientAliases(String arg0, Principal[] arg1) {
    try {
        return new String[] { keystore.aliases().nextElement() };
    } catch (Exception e) {
        return null;
    }
}

@Override
public PrivateKey getPrivateKey(String arg0) {
    try {
        return ((KeyStore.PrivateKeyEntry) keystore.getEntry(keystore.aliases().nextElement(),
                new KeyStore.PasswordProtection(password.toCharArray()))).getPrivateKey();
    } catch (Exception e) {
    }
    return null;
}

@Override
public String[] getServerAliases(String arg0, Principal[] arg1) {
    return null;
}

}

This would work if my pfx also contained its issuer certificate. But it doesn't (yay!). So when I used the key manager as above, I got a SSL handshake error (peer not authenticated). The server only authenticates the client if the client sends a certificate chain that the server trusts. Since my certificate (issued by a Brazilian agency) doesn't contain its issuer, its certificate chain contains only itself. The server doesn't like that and denies to authenticate the client. The work around is to create manually the certificate chain:

...
@Override
    //The order matters, your certificate should be the first one in the chain, its issuer the second, its issuer's issuer the third and so on.
public X509Certificate[] getCertificateChain(String arg0) {
            X509Certificate[] result = new X509Certificate[2];
            //The certificate chain contains only one entry in my case
            result[0] = (X509Certificate) keystore.getCertificateChain(keystore.aliases().nextElement())[0];
            //Implement getMyCertificateIssuer() according to your needs. In my case, I read it from a JKS keystore from my database
            result[1] = getMyCertificateIssuer();
            return result;
}
...

After that, it was just a matter of putting my custom key and trust managers to good use:

            InputStream keystoreContents = null;//Read it from a file, a byte array or whatever floats your boat
            KeyStore keystore = KeyStore.getInstance("PKCS12");
            keystore.load(keystoreContetns, "changeme".toCharArray());
            SSLContext context = SSLContext.getInstance("TLSv1");
            context.init(new KeyManager[] { new MyKeyManager(keystore, "changeme") },
                            new TrustManager[] { new MyTrustManager() }, new SecureRandom());
            SSLSocketFactory sf = new SSLSocketFactory(context);
            Scheme httpsScheme = new Scheme("https", 443, sf);
            SchemeRegistry schemeRegistry = new SchemeRegistry();
            schemeRegistry.register(httpsScheme);
            ClientConnectionManager cm = new SingleClientConnManager(schemeRegistry);
            HttpClient client = new DefaultHttpClient(cm);
            HttpPost post = new HttpPost("https://www.someserver.com");