Calling __enter__ and __exit__ manually

Vyktor picture Vyktor · Oct 29, 2014 · Viewed 10.7k times · Source

I've googled calling __enter__ manually but with no luck. So let's imagine I have MySQL connector class that uses __enter__ and __exit__ functions (originally used with with statement) to connect/disconnect from a database.

And let's have a class that uses 2 of these connections (for example for data sync). Note: this is not my real-life scenario, but it seems to be the simplest example.

Easiest way to make it all work together is class like this:

class DataSync(object):

    def __init__(self):
        self.master_connection = MySQLConnection(param_set_1)
        self.slave_connection = MySQLConnection(param_set_2)

    def __enter__(self):
            self.master_connection.__enter__()
            self.slave_connection.__enter__()
            return self

    def __exit__(self, exc_type, exc, traceback):
            self.master_connection.__exit__(exc_type, exc, traceback)
            self.slave_connection.__exit__(exc_type, exc, traceback)

    # Some real operation functions

# Simple usage example
with DataSync() as sync:
    records = sync.master_connection.fetch_records()
    sync.slave_connection.push_records(records)

Q: Is it okay (is there anything wrong) to call __enter__/__exit__ manually like this?

Pylint 1.1.0 didn't issue any warnings on this, nor have I found any article about it (google link in the beggining).

And what about calling:

try:
    # Db query
except MySQL.ServerDisconnectedException:
    self.master_connection.__exit__(None, None, None)
    self.master_connection.__enter__()
    # Retry

Is this a good/bad practice? Why?

Answer

Eric picture Eric · Aug 26, 2016

Your first example is not a good idea:

  1. What happens if slave_connection.__enter__ throws an exception:

    • master_connection acquires its resource
    • slave_connection fails
    • DataSync.__enter__ propogates the exception
    • DataSync.__exit__ does not run
    • master_connection is never cleaned up!
    • Potential for Bad Things
  2. What happens if master_connection.__exit__ throws an exception?

    • DataSync.__exit__ finished early
    • slave_connection is never cleaned up!
    • Potential for Bad Things

contextlib.ExitStack can help here:

def __enter__(self):
    with ExitStack() as stack:
        stack.enter_context(self.master_connection)
        stack.enter_context(self.slave_connection)
        self._stack = stack.pop_all()
    return self

def __exit__(self, exc_type, exc, traceback):
    self._stack.__exit__(self, exc_type, exc, traceback)

Asking the same questions:

  1. What happens if slave_connection.__enter__ throws an exception:

    • The with block is exited, and stack cleans up master_connection
    • Everything is ok!
  2. What happens if master_connection.__exit__ throws an exception?

    • Doesn't matter, slave_connection gets cleaned up before this is called
    • Everything is ok!
  3. Ok, what happens if slave_connection.__exit__ throws an exception?

    • ExitStack makes sure to call master_connection.__exit__ whatever happens to the slave connection
    • Everything is ok!

There's nothing wrong with calling __enter__ directly, but if you need to call it on more than one object, make sure you clean up properly!