I'm working on an app I'd like to sell some day -- sooner rather than later! I'd like to develop a reasonably simple serial number scheme to protect it.
I've been thinking about this a (very little) bit, and I think public key cryptography is a good place to start. I could generate a string that identifies the license (like SKU + plain ole' integral serial number), hash it, encrypt it, and encode the serial number + identifier into a 25 digit (or so) alphanumeric key. The app would then decode the key into a serial number and "signature", generate an identifier hash, decrypt the "signature" using a corresponding public key and compare it against the generated identifier hash.
Essentially, the product key carries two pieces of data: the serial number the user claims to own plus a signature of sorts the program can use to verify that claim. I don't know if 25 alphanumeric characters (which encode 5 bits each for a realistic total of 120 bits) is enough for all this. But, it doesn't have to be cryptographically secure, just enough that the codes aren't easily guessable. I'm OK with short key lengths and short hashes.
As far as implementation goes, the app is written in Objective-C for Mac OS X, but given how easy it is to inject code into Cocoa apps, I'll probably write the verification code in straight C.
I would not use any strong cryptography, since you have to decrypt it in program anyways, making keygens or at least cracks easy to do.
I would do the following - take a, say, 25 digit number. Now add some rules, such as: - number must be divisible by 31 - it must start and end with the last letter ...
Always generate keys using these rules. Use 20 rules or more (more the better). When deploying the app, use smaller number of rules, e.g. 10 to check if key is valid. These rules will then be disassemled and used to create keygen.
On every update enable one of the rules you didn't use before. If rules are selected correctly, you will disable most of keys generated by keygens.