Let me explain quickly what I'm trying to do. I'm trying to build my own Apple's Push Notification service in java (for testing purposes). This service works thanks to TLS socket.
I have a java client to create a TLS socket to send push notifications to the APNs. I changed the host url to redirect the socket to localhost:2195. Now I'm trying to write a java socket server to get the notification request.
However, I get an exception during the handshake and can't find how to fix it.
Note : I'm using the same certificate on both sides, it's a standard .p12 file that works to send push notifications to the APNs.
Here is the client (simplified) :
KeyStore ks = KeyStore.getInstance("PKCS12");
ks.load(new FileInputStream(certificatePath), password.toCharArray());
KeyManagerFactory kmf = KeyManagerFactory.getInstance("sunx509");
kmf.init(ks, password.toCharArray());
TrustManagerFactory tmf = TrustManagerFactory.getInstance("sunx509");
tmf.init((KeyStore)null);
SSLContext sc = SSLContext.getInstance("TLS");
sc.init(kmf.getKeyManagers(), tmf.getTrustManagers(), null);
SSLSocketFactory ssf = sc.getSocketFactory();
SSLSocket socket = (SSLSocket) ssf.createSocket(InetAddress.getLocalHost(), 2195);
socket.startHandshake();
Here is the server :
KeyStore ks = KeyStore.getInstance("PKCS12");
ks.load(new FileInputStream(certificatePath), password.toCharArray());
KeyManagerFactory kmf = KeyManagerFactory.getInstance("sunx509");
kmf.init(ks, password.toCharArray());
SSLContext context = SSLContext.getInstance("TLS");
context.init(kmf.getKeyManagers(), null, null);
SSLServerSocketFactory ssf = context.getServerSocketFactory();
serverSocket = (SSLServerSocket) ssf.createServerSocket(2195);
And here is the exception :
javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: No trusted certificate found
I guess the client isn't trusting the server's certificate. I tryed to set the client's TrustManager to accept the server's p12 and it worked, however I need this to work without editing the client (since it's working that way with the real APNs).
What kind of certificate needs the server to be trusted by the client ?
Thanks in advance.
EDIT: I WAS WRONG! tmf.init(null) DOES use the default keystore just like sslctx.init(,null,) ! That default is normally the cacerts file in JRE/lib/security which DOES trust many established CAs so now I think we can be confident the real server is using a cert under an established CA (and so is trusted by your client) while the cert in your p12 apparently does not; but there are two possibilities here:
it is selfsigned, or issued by an unknown, obscure, or unproven CA
it is issued by a 'real' CA under an 'intermediate' CA that needs a chain cert (or several) and you do not have the chain cert(s) in your p12. Note this could still work for client auth to the real server, because the real server can easily have the chain cert(s) 'preloaded' in its truststore even though they aren't in Java's.
To distinguish these, look at keytool -keystore file -storetype pkcs12 -list -v
and see what cert or sequence of certs you have there.
Then there may be several approaches to solution:
if you are only missing chain cert(s) for an established CA get them and add them. keytool only allows you to replace the whole chain so you must get all needed certs; openssl (if you have or get it) can break out the key and cert(s) from a pkcs12, replace or add individual certs, and join them back together.
create a different store and key for the server and get it a cert (chain) from an established CA. Usually costs some money and requires you prove control of the server's domain name. (Your client can and should still use this p12. The two sides needn't be the same.)
locate the trust anchor (from the p12, or from somewhere else like the CA) and have it in a truststore the client explicitly loads. You effectively tried this by using the p12 as the truststore and say you don't want that.
put the trust anchor in the client's default truststore, so the client continues using the default. If you don't mind modifying your JRE (and no other user or application on your system is bothered) just add to JRE/lib/security/cacerts. Or, assuming you can set system properties, put the anchor in a store or just leave it in the p12 and set javax.net.ssl.trustStore{,Password,Type} to point to that store. (If you copy you should take only the cert; a p12 is a KEY AND cert not just a cert. Don't just -importkeystore; -importcert a cert file, created with -exportcert if necessary.) (You can System.setProperty in your code, but that's changing your code. If you run from commandline you can use 'java -Dname=value...'. For other cases YMMV.)
There is one possible 'type' issue: if the cert was issued with ExtendedKeyUsage extension and that value specifies only TLSclient and not TLSserver (which the CA can choose to do) then using it for server probably won't work -- it appears JSSE enforces EKU restrictions. But if that is a problem you'll get a very different Exception. And you can see this also in the keytool -list -v above.
Since you (rightly) want to use this p12 for your client, your server logic similarly needs to trust it. (Using it for outgoing auth does NOT automatically make it trusted for incoming auth.) But only if/when clientAuth is actually done, which is not the default; does your server code .setNeedClientAuth(true) on the SSLServerSocket before accepting the connection? Possible approaches are equivalent to the above except skipping #2 as inapplicable. If both client and server use the same JRE, that makes the cacerts way a little easier.
Finally, yes TrustManager 'PKIX' is newer and generally more featureful than 'SunX509'. But for the basic test 'is the trust anchor in our truststore' they are equivalent.
Sorry again for the mislead.