vb6 winhhtp: Error Occurred in the Secure Channel Support

Question

I wrote a VB6 program which uses winhttp.dll to send and receive messages to/from a remote server. It has been working fine from various operating systems: Windows 2000, WinXP, Win7, Win8.

Recently the server provider informed me that they will "phase out support of SHA-1 security certificates" and i need to "check that the thumbprint is aligned to SHA-2 new SSL cert.".

What is happening now is that when my program runs on WinXP,7,8 - it's still OK. But when running on Windows 2000, when I call the winhttp Send() method, I get an exception "Error Occurred in the Secure Channel Support".

I am failey clueless on certificates etc and I don't really know what to do.

My Vb6 code looks as follows:

' Connect to server     
Set m_ServerObj = New WinHttpRequest
m_ServerObj.Open "GET", m_ServerURL_SC

' Send request to server
m_ServerObj.Option(WinHttpRequestOption_SslErrorIgnoreFlags) = &H3300 
            'Unknown certification authority (CA) or untrusted root   0x0100
            'Wrong usage                                              0x0200
            'Invalid common name (CN)                                 0x1000
            'Invalid date or certificate expired                      0x2000

m_ServerObj.Send xml

The error occurres when Send is called

Answer 1

ERROR_WINHTTP_SECURE_CHANNEL_ERROR: This is a generic error thrown whenever Schannel.dll encounters a SSL version / feature / cipher / key-length / etc. it doesn't understand or support, or when the server (in the case of a client) will not negotiate for anything the client supports. WinHttp relies on it (which is part of the OS) to provide support for SSL / TLS secure channels.

Now, in your case, a few reasons could explain this error, but since connecting to the server in question still works under Windows XP when it fails on Windows 2000, I don't think this failure is related (yet) to those SHA-2 signatures. [more details at the end]

1) SSL 3.0 or TLS 1.0 disabled

First thing I would check: open "Internet Options" from the Control Panel, switch to the "Advanced" tab, and in the options make sure SSL 3.0 & TLS 1.0 are checked. If you had to tick any of the two, a reboot will be needed. Then check again. (afterwards, if it still fails, you're free to disable SSL 3.0 if you wish i.e. because of POODLE)

2) Disabled / Missing CSP or no CSP provides support for the required cipher(s)

Schannel.dll makes use of the available cipher suites as provided by the different Cryptographic Service Providers (CSP) installed on the machine. If none of the CSPs provide support for the cipher(s) required by the server, then the HTTPS connection will fail.

I don't know in details the difference between the exact list of ciphers available on Windows 2000 versus those on Windows XP. Well, I know AES support was introduced with Windows XP & Windows Server 2003, and SHA-2 support (hashing using the CryptoAPI, not those SHA-2 certificates for TLS) was introduced as part of Windows XP SP3. As for Windows 2000, the MSDN page Cipher Suites in Schannel doesn't say anything about Windows 2000, but the list of CSPs included with Windows 2000 can be found in the page Microsoft CryptoAPI and Cryptographic Service Providers. More research would probably give you more details on what is supported on Windows 2000 and what's not.

If the error is indeed related to the installed/available CSPs, then the discrepancy between Windows 2000 and Windows XP would mean one of three things: a) a cipher or cipher suite, available on both Windows 2000 and Windows XP, has been disabled for Schannel on that/those Windows 2000 machines; read How to restrict the use of certain cryptographic algorithms and protocols in Schannel.dll to find out which registry keys are used to disable cipher/cipher suites, so you can check if it's the case. b) a cipher or cipher suite, available on both Windows 2000 and Windows XP, is missing/not registered on that/those Windows 2000 machines; there is no easy way to check this, but if you get that connection error on every Windows 2000 machine you try, then c) would make more sense. c) a cipher suite, bringing support for at least one of the ciphers (e.g. AES) required by the server, was only introduced in Windows XP and is not available for Windows 2000; you're then out of luck, if you stick with WinHttp anyway.

By now you should have a few cues on what could cause the error (my bet: AES is the minimum being required)

About those SHA-2 certificates

Support for SHA-2 signatures (superseding SHA-1) was introduced with TLS 1.2. If you didn't know yet, the only Windows OSes supporting TLS 1.2 (and thus certificates with SHA-2 signatures) are Windows 7, Windows Server 2008 R2, Windows 8+ and Windows Server 2012. This of course should not affect most applications (e.g. Chrome uses BoringSSL, a fork of OpenSSL) unless they depend on Schannel.dll, like WinHttp or most other Windows components.

SHA-1 certificates are expected to be gone by January 1st, 2017. By then, more and more servers will have ported their certificates to the SHA-2 variety, so expect more HTTPS connection failures for WinHttp under Windows XP & Windows Vista. It makes sense, after all; support for Windows XP finally came to an end on April 8, 2014 (Microsoft had already dropped support for Windows Vista) and the last Windows OS not supporting TLS 1.1 & 1.2, Windows Server 2008, reached end of mainstream support this January.

So, what should you do?: Either stop supporting Windows 2000 & XP, or find a replacement for WinHttp that uses a SSL/TLS library, e.g. OpenSSL, being regularly updated. Maybe libcurl?

Final note: Unless you use a self-signed certificate or a certificate without a valid chain of trust for the server, you really should leave WinHttpRequestOption_SslErrorIgnoreFlags to &H0& (otherwise, what's the point of using SSL/TLS?)