Solving sslv3 alert handshake failure when trying to use a client certificate

Question

I'm trying to connect to a service that requires a certificate for authorization. The process is that I send the service a CSR file. The service signs the CSR and sends me a certificate that I use for connection.

1. I generated the CSR by the following command line:

   openssl req -new -nodes -newkey rsa:2048 -keyout cert.key -out cert.csr
   

2. I took the content of the cert.csr and sent to them. They generate the client certificate and I got a PEM file back.

3. I now try to connect using their certificate file in SSLCERT for curl() and providing the private key from cert.key as CURLOPT_SSLKEY - (which I got at step 1).

4. Fails with: error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure

What am I doing wrong in this process?

It works when I try with a received a test certificate including a private key from the service (self signed certificate). But when I use a certificate they generated from my CSR and then use my private key as key, it errors with handshake failure.

So I know it does not have something to do with that openssl / curl doesn't support v3/TLS etc. that others when researching for a solution found out their problem was.

Here is what I run:

  curl -i -v --request POST https://service.com/ --cert clientcert.pem --key private_key.pem --cert-type pem --tlsv1.1 --insecure
* Connected to service.com (1xx.xxx.xxx.xx) port 443 (#0)
* successfully set certificate verify locations:
*   CAfile: none
  CApath: /etc/ssl/certs
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server key exchange (12):
* SSLv3, TLS handshake, Request CERT (13):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS handshake, CERT verify (15):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS alert, Server hello (2):
* error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure
* Closing connection 0

Running following versions: curl 7.35.0 (x86_64-pc-linux-gnu) libcurl/7.35.0 OpenSSL/1.0.1f zlib/1.2.8 libidn/1.28 librtmp/2.3

Answer 1

Not a definite answer but too much to fit in comments:

I hypothesize they gave you a cert that either has a wrong issuer (although their server could use a more specific alert code for that) or a wrong subject. We know the cert matches your privatekey -- because both curl and openssl client paired them without complaining about a mismatch; but we don't actually know it matches their desired CA(s) -- because your curl uses openssl and openssl SSL client does NOT enforce that a configured client cert matches certreq.CAs.

Do openssl x509 <clientcert.pem -noout -subject -issuer and the same on the cert from the test P12 that works. Do openssl s_client (or check the one you did) and look under Acceptable client certificate CA names; the name there or one of them should match (exactly!) the issuer(s) of your certs. If not, that's most likely your problem and you need to check with them you submitted your CSR to the correct place and in the correct way. Perhaps they have different regimes in different regions, or business lines, or test vs prod, or active vs pending, etc.

If the issuer of your cert does match desiredCAs, compare its subject to the working (test-P12) one: are they in similar format? are there any components in the working one not present in yours? If they allow it, try generating and submitting a new CSR with a subject name exactly the same as the test-P12 one, or as close as you can get, and see if that produces a cert that works better. (You don't have to generate a new key to do this, but if you choose to, keep track of which certs match which keys so you don't get them mixed up.) If that doesn't help look at the certificate extensions with openssl x509 <cert -noout -text for any difference(s) that might reasonably be related to subject authorization, like KeyUsage, ExtendedKeyUsage, maybe Policy, maybe Constraints, maybe even something nonstandard.

If all else fails, ask the server operator(s) what their logs say about the problem, or if you have access look at the logs yourself.