Digital Signature SunMSCAPI provider & MS Crypto API

Question

I want to sign file with the SunMSCAPI provider. As public key and signatures needs to be imported using MS Crypto API.

Generally generating signatures with SHA1withRSA, ends up with big-endian to little-endian (byte order) conversion.

//generate keystore with java keytool
$Keytool -genkey -alias tsign -keystore c:\test\tsignjks.p12 - keyalg rsa -storetype  pkcs12

In Java application:

//for signing and getting keystore, assuming windows certificate is installed
..ks = KeyStore.getInstance("Windows-MY","SunMSCAPI"); 
PrivateKey priv = ks.getKey("tsign",password); 
Signature rsa = Signature.getInstance("SHA1withRSA","SunMSCAPI"); 
rsa.initSign(priv);
.. 
rsa.update(buffer, 0, len);
..
byte[] realSig = rsa.sign();

//for writing public key for ms crypto api or exporting it from windows certificate store
Certificate cert = ks.getCertificate("tsign");
byte[] encodedCert = cert.getEncoded();
FileOutputStream certfos = new FileOutputStream("tsigncer.cer");
certfos.write(encodedCert);

//for writing signatures for ms crypto api
FileOutputStream sigfos = new FileOutputStream(targetPath + "/"
                + signatureName);
sigfos.write(realSig);

I believe that SunMSCAPI can resolve my problem, but I don't know when i import public key using MS Crypto API, It never import at at first stage (unless i change big endian to little endian byte order) below is my code for crypto API.

LPCSTR file = "tsigncer.cer";
//LPCSTR file = "omsign.p12";
BOOL crypt_res = FALSE;

HCRYPTPROV crypt_prov_hndl = NULL;
 crypt_res = CryptAcquireContext(&crypt_prov_hndl, NULL, NULL, PROV_RSA_FULL, 0/*CRYPT_NEWKEYSET*/);
//crypt_res = CryptAcquireContext(&crypt_prov_hndl, NULL, NULL, PROV_DSS, CRYPT_VERIFYCONTEXT/*CRYPT_NEWKEYSET*/);

    if (!crypt_res) {
        HRESULT decode_hr = __HRESULT_FROM_WIN32(GetLastError());
        return decode_hr;
    }

    // Load key file
    HANDLE fileHandle = CreateFile(file, // name of the write
                       GENERIC_READ,          // open for writing
                       0,                      // do not share
                       NULL,                   // default security
                       OPEN_EXISTING,             // create new file only
                       FILE_ATTRIBUTE_NORMAL,  // normal file
                       NULL);                  // no attr. template

    if (fileHandle == INVALID_HANDLE_VALUE)
    {
        DWORD d = GetLastError();
        return -1;
    }

    BYTE buffer[2056];
    DWORD fileSize = 0;
    DWORD fileSizeResult = GetFileSize(fileHandle, &fileSize);

    DWORD numBytesRead = 0;
    BOOL fileLoadResult = ReadFile(fileHandle, (PVOID)buffer, fileSizeResult, &numBytesRead, NULL);

    // Import key
    BOOL result = ImportKey(crypt_prov_hndl, (LPBYTE)buffer, numBytesRead);
//result is always false..

Answer 1

If you work with MSCAPI, it is assumed that you've added your key to the Microsoft Certificate store. You can check if the key is present by going to "Internet Properties" > "Content" > "Certificates" which gives you a list of certificates that are available. If your certificate isn't there, you can't use it. If it's there, you need this code:

SunMSCAPI providerMSCAPI = new SunMSCAPI();
Security.addProvider(providerMSCAPI);
KeyStore ks = KeyStore.getInstance("Windows-MY");
ks.load(null, null);

From there on, the code is pretty standard. Please consult my book on digital signatures for more info (the book is free).

IMPORTANT ADDITION: I forgot to mention that SunMSCAPI isn't present in the 64-bit version of Java 6 (I don't know about Java 7). You can fix this by installing the 32-bit version.