Try/catch or validation for speed?

Blender picture Blender · Apr 8, 2011 · Viewed 7.9k times · Source

I'm working with Python and whenever I've had to validate function input, I assumed that the input worked, and then caught errors.

In my case, I had a universal Vector() class which I used for a few different things, one of which is addition. It functioned both as a Color() class and as a Vector(), so when I add a scalar to the Color(), it should add that constant to each individual component. Vector() and Vector() addition required component-wise addition.

This code is being used for a raytracer so any speed boosts are great.

Here's a simplified version of my Vector() class:

class Vector:
  def __init__(self, x, y, z):
    self.x = x
    self.y = y
    self.z = z

  def __add__(self, other):
    try:
      return Vector(self.x + other.x, self.y + other.y, self.z + other.z)
    except AttributeError:
      return Vector(self.x + other, self.y + other, self.z + other)

I'm currently using the try...except method. Does anybody know of a faster method?


EDIT: Thanks to the answers, I tried and tested the following solution, which checks specifically for a class name before adding the Vector() objects:

class Vector:
  def __init__(self, x, y, z):
    self.x = x
    self.y = y
    self.z = z

  def __add__(self, other):
    if type(self) == type(other):
      return Vector(self.x + other.x, self.y + other.y, self.z + other.z)
    else:
      return Vector(self.x + other, self.y + other, self.z + other)

I ran a speed test with these two blocks of code using timeit, and the results were pretty significant:

 1.0528049469 usec/pass for Try...Except
 0.732456922531 usec/pass for If...Else
 Ratio (first / second): 1.43736090753

I haven't tested the Vector() class with no input validation whatsoever (i.e. moving the checking out of the class and into the actual code), but I'd imagine that it's even faster than the if...else method.


Late update: Looking back at this code, this is not an optimal solution.

OOP makes this even faster:

class Vector:
  def __init__(self, x, y, z):
    self.x = x
    self.y = y
    self.z = z

  def __add__(self, other):
    return Vector(self.x + other.x, self.y + other.y, self.z + other.z)

class Color(Vector):
  def __add__(self, other):
    if type(self) == type(other):
      return Color(self.x + other.x, self.y + other.y, self.z + other.z)
    else:
      return Color(self.x + other, self.y + other, self.z + other)

Answer

ncoghlan picture ncoghlan · Apr 8, 2011

I upvoted Matt Joiner's answer, but wanted to include some additional observations to make it clear that, along with a couple of other factors, there are 4 times that matter when choosing between pre-checking conditions (known as LBYL or "Look Before You Leap") and just handling exceptions (known as EAFP or "Easier to Ask Forgiveness than Permission").

Those timings are:

  • Timing when the check succeeds with LBYL
  • Timing when the check fails with LBYL
  • Timing when an exception is not thrown with EAFP
  • Timing when an exception is thrown with EAFP

The additional factors are:

  • The typical ratio of check success/failure or exception thrown/not thrown cases
  • Whether or not there is a race condition that prevents the use of LBYL

That last point is the one that needs to be addressed first: if there is a potential for a race condition, then you have no choice, you must use exception handling. A classic example is:

if <dir does not exist>:
    <create dir> # May still fail if another process creates the target dir

Since LBYL doesn't rule out the exception is such cases, it offers no real benefit and there's no judgement call to be made: EAFP is the only approach that will handle the race condition correctly.

But if there's no race condition, either approach is potentially viable. They offer different trade-offs:

  • if no exception is raised, then EAFP is close to free
  • however, it is comparatively expensive if an exception occurs, as there is quite a lot of processing involved in unwinding the stack, creating the exception and comparing it to the exception handling clauses
  • LBYL, by contrast, incurs a potentially high fixed cost: the additional check is always performed, regardless of success or failure

That then leads to the following decision criteria:

  • Is this piece of code known to be critical to the speed of the application? If not, then don't worry about which of the two is faster, worry about which of the two is easier to read.
  • Is the pre-check more expensive than the cost of raising and catching an exception? If yes, then EAFP is always faster and should be used.
  • Things get more interesting if the answer is "no". In that case, which is faster will depend on whether the success or the error case is more common, and the relative speeds of the pre-check and the exception handling. Answering this definitively requires real timing measurements.

As a rough rule of thumb:

  • if there is a potential race condition, use EAFP
  • if speed isn't critical, just use whichever you consider easier to read
  • if the pre-check is expensive, use EAFP
  • if you expect the operation to succeed most of the time*, use EAFP
  • if you expect the operation to fail more than half the time, use LBYL
  • if in doubt, measure it

*People will vary as to what they consider "most of the time" in this context. For me, if I expect the operation to succeed more than half the time, I would just use EAFP as a matter of course, until I had reason to suspect this piece of code was an actual performance bottleneck.