Writing and Using Custom Exceptions in Python

This tutorial will go through the "what" and "why" of exceptions in Python, and then walk you through the process of creating and using your own types of exceptions. And more importantly, when not to. Enjoy!

What is an Exception?

If you have been coding in Python for any length of time, no doubt you have seen a traceback. Just in case you haven't, here we'll make one happen. You can open up a Python console and type in the statements that follow, or just read along:

>>> l = [1,2,3]               #1
>>> l['apples']               #2
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: list indices must be integers, not str
>>> l[4]                      #3
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IndexError: list index out of range
>>> l[1]                      #4
2

So here is what we just did. In line 1 we made a list with three elements in it. Line 2 tries to access the element at the index 'apples'. Now, since that's just really not cool, Python complains. It does so by raising a TypeError. TypeError is a kind of Exception. When an Exception gets raised and but does not get caught (we'll get to that later), then it ends up printing a traceback to the error output. In the case of the Python console, the error output is just the console. A traceback message gives some information about the actual error and gives some detail about how we got to the point where the error actually happened. If this last sentence was confusing don't worry, it should become clear with the next example.

The TypeError complains that it was expecting an integer and not a string. This seems like a pretty reasonable reaction. So in line 3 we give it an integer. Unfortunately the only indices available for l are 0,1 and 2, but we're trying to access l[4]. Naturally this also isn't cool. So Python complains again by raising an IndexError and printing an appropriate traceback.

Finally, we do something sensible and access l[1]. This, as compared to our other attempts, is cool. l[1] has the value 2.

Great. So what have we learned?

Python uses Exceptions to tell on bad code. Exceptions are raised when something doesn't work according to plan, where the program cannot proceed. And there are different types of exceptions for different situations.

The Traceback

Here's a more interesting example, just to demonstrate a bit more about the traceback and why this exception is cool as well:

>>> def f1(x):
...     assert x == 1
... 
>>> def f2(x):
...     f1(x)
...  
>>> def f3(x):
...     f2(x)
...     not_cool
... 

So far nothing amazing has happened in the code above. We made three functions such that f3 calls f2 calls f1. Now let's do something with them:

>>> f1(1)
>>> f1(2)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 2, in f1
AssertionError

Here we meet the AssertionError. This exception gets raised every time x == 1 evaluates to False. Pretty straight-forward. As a side note, assertion statements like the one above are pretty useful for 'sanity checking' while running code.

>>> f2(1)
>>> f2(2)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 2, in f2
  File "<stdin>", line 2, in f1
AssertionError

This time the traceback is a little longer. From this traceback, you can see that the AssertionError was raised in f1, and that f1 was called by f2. It even has line numbers. The line numbers aren't too useful right now because we are just entering things into the console. However, writing and executing complex Python programs entirely in the console is not common practice; usually you'll be calling functions that are stored in files. If such a function raises an Exception, then the traceback will help you find exactly what line of what file raised the error.

>>> f3(1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in f3
NameError: global name 'not_cool' is not defined
>>> 

This time we are calling f3. The value of 1 gets passed to f1 and the assertion statement causes no error. The program then proceeds back to f3 where the statement not_cool caused an error. The traceback does not include any information about f1 and f2 because those functions had already executed without error.

>>> f3(2)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 2, in f3
  File "<stdin>", line 2, in f2
  File "<stdin>", line 2, in f1
AssertionError

This time we execute f3 and give it a value that will cause an AsserionError to be raised by f1. The traceback describes the process flow.

If you are having any trouble understanding how the program flow can be determined from a traceback message, then it will likely be worth your while to do some reading about 'call stacks'.

Catching Exceptions

The power of exceptions is that we can write code to react to them. To do this, we catch exceptions. Continuing from our previous code:

>>> try:
...     f1(1)
... except:
...     print "caught an exception"
... 

Aaaand nothing happens. Great, there was no exception.

>>> try:
...     f1(2)
... except:
...     print "caught an exception"
... 
caught an exception

This time there was one and we caught the bugger. Whatever you put inside an except block will only execute if it catches an exception. But there's more to it than that:

>>> try:
...     f1(1)
... except:
...     print "caught an exception"
... else:
...    print "no exception"
... finally:
...    print "the end"
no exception
the end
>>>
>>> try:
...     f1(2)
... except:
...     print "caught an exception"
... else:
...    print "no exception"
... finally:
...    print "the end"
caught an exception
the end

The else block only gets executed if no exception gets caught. And the finally block gets executed no matter what.

But not all exceptions are created equal. The reason we have different types of exceptions is because we might want to react to them differently.

For example:

>>> def foo(i):
...     l = [1,2,3]
...     try:
...         assert i >= 1
...         return l[i]
...     except TypeError,e:                                  ####A
...         print "dealing with TypeError"
...     except IndexError, e:                                ####B
...         print "dealing with IndexError"
...     except:                                                         ####C
...         print "oh dear"
...     finally:                                                           ####D
...         print "the end"
... 
>>> 
>>> foo('apples')
dealing with TypeError
the end
>>> 
>>> foo(-1)
oh dear
the end
>>> 
>>> foo(4)
dealing with IndexError
the end
>>> 
>>> foo(1)
the end
2

Whenever we call foo, we try to return the requested element of the list if it is positive. We have 3 different ways of catching exceptions. If an exception gets raised, then execution proceeds to the first except block that matches the exception. In other words, if an exception is raised, then Python first checks if it is a TypeError (A). If it is a TypeError, then it will process that except block before it proceeds to the finally block. If it is not a TypeError, then Python checks if it is an IndexError (B), etc.

Note that that means that the order of the except blocks matters. Let's edit foo to look like this:

>>> def foo(i):
...     l = [1,2,3]
...     try:
...         assert i >= 1
...         return l[i]
...     except:                                                         ####C
...         print "oh dear"
...     except TypeError,e:                                  ####A
...         print "dealing with TypeError"
...     except IndexError, e:                                ####B
...         print "dealing with IndexError"
...     finally:                                                           ####D
...         print "the end"
... 

A good rule of thumb is to only catch exceptions you are willing to handle.

Raising Exceptions On Purpose

You can explicitly raise Exceptions in two ways (making an error in your code is more of an implicit method).

The first way is to reraise an exception you caught. For example:

try:
    do_important_stuff()
except:
    import traceback
    s = traceback.format_exc()
    send_error_message_to_responsible_adult(s)
    raise 

Or, you can construct an Exception object and raise it yourself. Since Exceptions have different types, they sometimes expect different arguments. Here's a really basic example:

def greet_person(sPersonName):
    """
    says hello
    """
    if sPersonName == "Robert":
        raise Exception("we don't like you, Robert")
    print "Hi there {0}".format(sPersonName)

Try to greet a few people and see what happens.

Subclassing Exceptions and Other Fancy Things

Since Exceptions are objects and can be constructed, it makes sense that we can subclass the Exception class. Or even subclass subclasses of the Exception class.

class MyException(Exception):
    def __init__(self,*args,**kwargs):
        Exception.__init__(self,*args,**kwargs)


class MyIndexError(IndexError):
def __init__(self,*args,**kwargs):
        IndexError.__init__(self,*args,**kwargs)

Now you can raise your own exceptions, just like any other exception. Note that when creating your except statements, inheritance matters. That is, an except statement aimed at IndexError will also catch MyIndexError. This is because a MyIndexError IS an IndexError. Conversely, if you have an except block aimed at MyIndexError, then it will NOT catch IndexError. This is because an IndexError IS NOT a MyIndexError.

So that is simple enough. But why would you want to do that? Well, best practice is really to avoid doing that sort of thing. modern versions of Python have a rich set of Exceptions already, so it often isn't worth creating more stuff. If you wanted to create MyIndexError, then ask yourself if a regular IndexError would do the trick. If it won't do the trick, then it's possibly worthwhile.

For example, if the context in which the exception was raised is extra meaningful, then it might be worth storing that context in the Exception:

class MyExceptionWithContext(Exception):
    def __init___(self,dErrorArguments):
        Exception.__init__(self,"my exception was raised with arguments {0}".format(dErrArguments))
        self.dErrorArguments = dErrorArguements
        
def do_stuff(a,b,c):
   if some_complicated_thing(a,b,c):
       raise MyExceptionWithContext({
                'a' : a,
                'b' : b,
                'c' : c,
       })
   else:
       return life_goes_on(a,b,c)

A lot of the time, the context of an exception is simple enough that it can be passed as a message to one of the built-in Exception types. In other cases, this sort of thing really makes sense. The urllib package demonstrates a good use of this method. If you ask urllib to access a specific URL it may just succeed, but if it doesn't, then it tries to give you as much information as possible to debug the problem. For example, it generates different kinds exceptions for a timeouts and 404s.

Sometimes you want an Exception that is very much like one of the built-in exceptions in every way, but it has some pre-determined message. Like so:

>>> class OhMyGoodnessExc(Exception):
...    def __init__(self):
...        Exception.__init__(self,"well, that rather badly didnt it?") 
... 
>>> raise OhMyGoodnessExc()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
__main__.OhMyGoodnessExc: well, that rather badly didnt it?
>>>
>>> raise OhMyGoodnessExc()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
__main__.OhMyGoodnessExc: well, that rather badly didnt it?

Every time you raise an OhMyGoodnessExc, the same thing happens. The fact that it is anOhMyGoodnessExc doesn't matter much─what we care about is the message.

This is more easily and neatly achieved by just constructing a suitable exception beforehand and raising it when you need to:

>>> oh_my_goodness = Exception("well, that rather badly didnt it?")
>>>
>>> raise oh_my_goodness
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
Exception: well, that rather badly didnt it?
>>>
>>> raise oh_my_goodness
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
Exception: well, that rather badly didnt it?

The thing to note here is that an Exception doesn't need to be raised as soon as it is constructed. And when in doubt, KISS (Keep It Simple, Stupid).

Conclusion

We've covered a lot of ground here, specifically what exceptions do and how and when to catch them; how to make them happen and how to make your own exception classes. We also covered quite a bit of good practice regarding exceptions. Here's a little summary:

1. Not all exceptions are created equal: if you know what class of exception you are dealing with, then be specific about what you catch
2. Don't catch anything you can't deal with
3. If you need to deal with multiple exception types, then have multiple except blocks in the right order
4. custom exceptions can be very useful if complex or specific information needs to be stored in exception instances
5. don't make new exception classes when the built-in ones have all the functionality you need
6. you don't need to raise an Exception as soon as it is constructed, this makes it easy to pre-define a set of allowed errors in one place

That's all folks.