python httplib2 certificate verify failed

python ssl httplib2
brainstormtrooper picture brainstormtrooper · Jan 7, 2015 · Viewed 8.8k times · Source

I have tried everything I can find to get this to work...

I'm working on a plugin for a python-based task program (called GTG). I'm running Gnome on Opensuse Linux.

Code (Python 2.7):

def initialize(self):
    """
    Intialize backend: try to authenticate. If it fails, request an authorization.
    """
    super(Backend, self).initialize()
    path = os.path.join(CoreConfig().get_data_dir(), 'backends/gtask', 'storage_file-%s' % self.get_id())
    # Try to create leading directories that path
    path_dir = os.path.dirname(path)
    if not os.path.isdir(path_dir):
        os.makedirs(path_dir)

    self.storage = Storage(path)
    self.authenticate()

def authenticate(self):
    """ Try to authenticate by already existing credences or request an authorization """
    self.authenticated = False

    credentials = self.storage.get()
    if credentials is None or credentials.invalid == True:
        self.request_authorization()
    else:
        self.apply_credentials(credentials)
        # Request periodic import, avoid waiting a long time
        # self.start_get_tasks()

def apply_credentials(self, credentials):
    """ Finish authentication or request for an authorization by applying the credentials """


    http = httplib2.Http(ca_certs = '/etc/ssl/certs/ca_certs.pem', disable_ssl_certificate_validation=True)

    http = credentials.authorize(http)

    # Build a service object for interacting with the API.
    self.service = build_service(serviceName='tasks', version='v1', http=http, developerKey='AIzaSyAmUlk8_iv-rYDEcJ2NyeC_KVPNkrsGcqU')
    # self.service = build_service(serviceName='tasks', version='v1')
    self.authenticated = True

def _authorization_step2(self, code):
    credentials = self.flow.step2_exchange(code)
    # credential = self.flow.step2_exchange(code)

    self.storage.put(credentials)
    credentials.set_store(self.storage)

    return credentials

def request_authorization(self):
    """ Make the first step of authorization and open URL for allowing the access """

    self.flow = OAuth2WebServerFlow(client_id=self.CLIENT_ID,
        client_secret=self.CLIENT_SECRET,
        scope='https://www.googleapis.com/auth/tasks',
        redirect_uri='http://localhost:8080',
        user_agent='GTG')


    oauth_callback = 'oob'
    auth_uri = self.flow.step1_get_authorize_url(oauth_callback)
    # credentials = self.flow.step2_exchange(code)



    # url = self.flow.step1_get_authorize_url(oauth_callback)
    browser_thread = threading.Thread(target=lambda: webbrowser.open_new(auth_uri))
    browser_thread.daemon = True
    browser_thread.start()

    # Request the code from user
    BackendSignals().interaction_requested(self.get_id(), _(
        "You need to <b>authorize GTG</b> to access your tasks on <b>Google</b>.\n"
        "<b>Check your browser</b>, and follow the steps there.\n"
        "When you are done, press 'Continue'."),
        BackendSignals().INTERACTION_TEXT,
        "on_authentication_step")


def on_authentication_step(self, step_type="", code=""):

    if step_type == "get_ui_dialog_text":
        return _("Code request"), _("Paste the code Google has given you"
                "here")
    elif step_type == "set_text":
        try:

            credentials = self._authorization_step2(code)
        except FlowExchangeError, e:
            # Show an error to user and end
            self.quit(disable = True)
            BackendSignals().backend_failed(self.get_id(), 
                        BackendSignals.ERRNO_AUTHENTICATION)
            return

        self.apply_credentials(credentials)
        # Request periodic import, avoid waiting a long time
        self.start_get_tasks()

The browser window opens up and I am presented with a code from Google. The program opens a small window where I can enter the code from Google.When that happens I get this in the console : 

 No handlers could be found for logger "oauth2client.util"
Created new window in existing browser session.
[522:549:0108/063825:ERROR:nss_util.cc(821)] After loading Root Certs, loaded==false: NSS error code: -8018

but the SSL icon is green in Chrome...

then when I submit the code, I get :

    Exception in thread Thread-10:
Traceback (most recent call last):
  File "/usr/lib64/python2.7/threading.py", line 810, in __bootstrap_inner
    self.run()
  File "/usr/lib64/python2.7/threading.py", line 763, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/usr/lib/python2.7/site-packages/GTG/backends/backend_gtask.py", line 204, in on_authentication_step
    credentials = self._authorization_step2(code)
  File "/usr/lib/python2.7/site-packages/GTG/backends/backend_gtask.py", line 151, in _authorization_step2
    credentials = self.flow.step2_exchange(code)
  File "/usr/lib/python2.7/site-packages/oauth2client/util.py", line 132, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/oauth2client/client.py", line 1283, in step2_exchange
    headers=headers)
  File "/usr/lib/python2.7/site-packages/httplib2/__init__.py", line 1586, in request
    (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
  File "/usr/lib/python2.7/site-packages/httplib2/__init__.py", line 1328, in _request
    (response, content) = self._conn_request(conn, request_uri, method, body, headers)
  File "/usr/lib/python2.7/site-packages/httplib2/__init__.py", line 1250, in _conn_request
    conn.connect()
  File "/usr/lib/python2.7/site-packages/httplib2/__init__.py", line 1037, in connect
    raise SSLHandshakeError(e)
SSLHandshakeError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)

The file is called backend_gtask.py...

I have tried importing the certificate as stated here : How to update cacerts.txt of httplib2 for Github?

I have tried to disable verification (httplib2.Http(disable_ssl_certificate_validation=True)) as stated all over the web,

I have updated the python packages (which seemed to make things worse)

I have copied ca_certs.pem back and forth between /etc/ssl... and /usr/lib/python2.7/...

When I visit the auth page in a browser, it says the certificate is verified...

What else can I possibly check?

SHORT TEST CODE :

from oauth2client.client import OAuth2WebServerFlow
from oauth2client.tools import run
from oauth2client.file import Storage

CLIENT_ID = 'id'
CLIENT_SECRET = 'secret'

flow = OAuth2WebServerFlow(client_id=CLIENT_ID,
                           client_secret=CLIENT_SECRET,
                           scope='https://www.googleapis.com/auth/tasks',
                           redirect_uri='http://localhost:8080')

storage = Storage('creds.data')

credentials = run(flow, storage)

print "access_token: %s" % credentials.access_token

Found that here: https://github.com/burnash/gspread/wiki/How-to-get-OAuth-access-token-in-console%3F

Answer

brainstormtrooper picture brainstormtrooper · Jan 9, 2015

OK... Big thanks to Steffen Ullrich.

httplib2 version 0.9 tries to use the system certificates and not the certs.txt file that used to be shipped with it. It also enforces verification.

httplib2 can take a couple of useful parameters - notably ca_certs. Use it to point to the actual *.pem file in you ssl installation. I cannot be a folder, must be a real file.

I use the following in the initialization of the plugin :

self.http = httplib2.Http(ca_certs = '/etc/ssl/ca-bundle.pem')

Then, for all subsequent calls to httplib or google client libraries, I pass my pre-built http object as a parameter like this:

credentials = self.flow.step2_exchange(code, self.http)

self.http = credentials.authorize(self.http)

Now ssl connections work with the new httplib2...

I will eventually have to make sure the plugin can find certificates on any system, but at least I know what the problem was.

Thanks again to Steffen Ullrich for walking me through this.

Related questions

pip install fails with "connection error: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:598)"

I am very new to Python and trying to > pip install linkchecker on Windows 7. Some notes: pip install is failing no matter the package. For example, > pip install scrapy also results in the SSL error. Vanilla install of …

Read More
urllib and "SSL: CERTIFICATE_VERIFY_FAILED" Error

I am getting the following error: Exception in thread Thread-3: Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 810, in __bootstrap_inner self.run() File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.…

Read More
Python Requests throwing SSLError

I'm working on a simple script that involves CAS, jspring security check, redirection, etc. I would like to use Kenneth Reitz's python requests because it's a great piece of work! However, CAS requires getting validated via SSL so I have …

Read More