What does the Python InsecureRequestWarning really mean?
I'm getting the warning:
/.../local/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:734: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
InsecureRequestWarning)
I'm reading the doc.
I'm seeing lots of posts on how to disable it if I know what I'm doing, like this one.
But I'm still having trouble figuring out what the error means. I gather that it means that I'm missing a certificate (because it only happens on my VPS, not on my Mac running the same version of a script), but I don't understand why I need a certificate to make a secure request to a third-party API.
A helpful summary (or just a point in the right direction) would be much appreciated so I can decide whether or not to disable it. My gut is that I shouldn't disable it, so I'd like to figure out how to address the problem properly.
Answer
I am glad that you did not simply disable the warning. Great question, actually! What's required here is basic understanding of how the "chain of trust" is working. That is not a shame, many do not have knowledge about this. However, as a developer one should know the basics! Go ahead, and maybe read about how the whole thing works.
In short, TLS is meant to guarantee secrecy, authenticity, and integrity. Common sense in the security community is (*): without certificate verification you get NONE of these three items, because you are vulnerable to man in the middle attacks. That is, verify the certificate, or you might just as well stop using HTTPS. That is what the warning is about.
A little more context: part of this security architecture is that the remote host claims to have a certificate signed by someone higher in the chain of trust, a so-called certificate authority (CA). The client needs to verify that this CA actually did sign that certificate in question. For this verification to work, the client needs a local database with the public keys of many CAs (think of these as "trust anchors", the collection of which can be called "certificate bundle").
I don't understand why I need a certificate to make a secure request to a third-party API
Please, read about the details elsewhere. But, for completeness of this answer, this is a high-level abstraction that should clarify why some external source of information is required:
- Your client does not trust the remote end.
- So, your client needs to involve a third party and ask about the credibility of the remote end.
- That third party is your local database of public keys of many CAs, a so-called "certificate bundle".
You can use the requests library instead of urllib3, it performs certificate verification by default (and ships its own CA database).
(*) unverified HTTPS connections can be "better" than plain HTTP, but this needs to be evaluated on a case-to-case basis.