Client-side encryption over HTTP with Diffie-Hellman Key Exchange and AES

Question

After watching a YouTube video on the Diffie-Hellman Key Exchange, I wanted to try an implementation in JavaScript (Atwood's law).

I sketched up an cipher on Node.js with the following rules:

• Step 1: Client and server agree on a shared key:

  • Client & server start with a 512bit prime public key pK

  • Client generates a 512bit prime private key kC and sends powMod(3, kC, pK)

  • Server generates a 512bit prime private key kS and sends powMod(3, kS, pK)

  • Client & Server use powMod(response, privatekey, pK) as the shared key

• Step 2: Communication

  • Before a client sends data it is encrypted with the shared key using the Stanford Javascript Crypto Library (256bit AES, HMAC authentication, PBKDF2 password strengthening, and CCM authenticated-encryption.)

  • Once the server decrypts the data with the shared key, it generates a new 512bit prime private key and sends it as a SJCL encrypted response.

  • The client and server switch to a new shared key using powMod(3, prevSharedKey, newPrivKey)

Now I have a few questions..

How secure would such a system be in comparison with HTTPS or other algorithms? What are the weakest points of such a system?

In terms of security / practicality, would it be better to use 1024 bit keys for stronger security? Are the HMAC/PBKDF2/CCM options overkill? Is it worth modulating the shared key? Thanks for reading!

Answer 1

Your system is massively insecure, but I'm not trying to dissuade you or anyone from playing around with stuff like this. You should continue to. But it is vital that you consider anything you create to be a "toy" system that should never be considered or advertised as "secure".

Let's break down the security question into two parts.

1. How secure is the key exchange?
2. How secure is the encryption you use once you've got a shared key?

Let me answer (2) first as that will be the simplest. It will be terribly insecure unless you are smarter than all of the people who've worked on and studied TLS over the years. TLS before version 1.2 (which few sites use) is vulnerable to Chosen Ciphertext Attacks (CCAs) in principle and to the BEAST attack in practice depending on cipher suit choice. And SSL 2.0 is more badly broken.

The point is that very very smart people, working on these protocols over years, got some things wrong. There is every reason to believe that you are I working on these sorts of things on our own will make huge mistakes. The basic encryption algorithms are fine. They aren't broken. But the protocols are.

So if you haven't studied and fully understood all of the details of SSL, why they are there and how they have gone wrong in some cases, then it is almost certain that any protocol you devise will be terrible.

Now to question (2). There are two issues with this. (a) Diffie-Hellman is not designed to provide the sorts of security you probably need; and (b) I don't think that you've implemented DH correctly.

2.a:

Diffie-Hellman Key exchange, when done right, is secure for key exchange, but it does nothing for authentication. This is why the question "is it secure" is often the wrong question. It is secure for some purposes, but massively insecure for others as it isn't designed for those other purposes.

As Josh3737 pointed out, there is no way for the client and the server to know that they are talking to the right party. If Sam is the server and Charlie is the Client, there is nothing that stops Mallory from setting up her own server that masquerades as Sam. So Cathy can go through the key exchange with Mallory, thinking that she is talking to Sam. Mallory can pretend to be Charlie when talking to Sam.

Once set up this way, Mallory can act as a Man In The Middle between Sam and Charlie. When Charlie sends data intended to Sam, Mallory will decrypt it using the shared key between C and M, read it (and possibly change it), and then re-encrypt it the the shared key between M and S and send that off to S.

To solve the authentication problem, you need some sort of Public Key Infrastructure (PKI) and these are really a pain. The system of Certificate Authorities and such that we have with SSL/TLS is fraught with problems, but it remains the best system out there.

2.b:

A 512 bit public modulus along with 512 bit private keys are not strong enough. DH keys need to be bigger. I wouldn't go with anything less than 2048 bits. You might get away with 1024 bits you aren't worried about someone being able to break today's secrets five years from now.

You didn't give enough information on how your primes were selected. Not every prime will work. You need to use a "safe prime" for your modulus, otherwise there are shortcuts available for an attacker to compute the discrete logarithm.