Handle an exception thrown in a generator

Question

I've got a generator and a function that consumes it:

def read():
    while something():
        yield something_else()

def process():
    for item in read():
        do stuff

If the generator throws an exception, I want to process that in the consumer function and then continue consuming the iterator until it's exhausted. Note that I don't want to have any exception handling code in the generator.

I thought about something like:

reader = read()
while True:
    try:
        item = next(reader)
    except StopIteration:
        break
    except Exception as e:
        log error
        continue
    do_stuff(item)

but this looks rather awkward to me.

Answer 1

When a generator throws an exception, it exits. You can't continue consuming the items it generates.

Example:

>>> def f():
...     yield 1
...     raise Exception
...     yield 2
... 
>>> g = f()
>>> next(g)
1
>>> next(g)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in f
Exception
>>> next(g)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration

If you control the generator code, you can handle the exception inside the generator; if not, you should try to avoid an exception occurring.