Errors... you'll see a lot of them as a programmer, especially when you're just getting started. Most of the time, errors can be pretty annoying, but all of the time, they are illuminating.

In this article, we'll be going over the major kinds of errors that stop Python in it's tracks. And once you have a good idea about what an error is, we'll talk about how to deal with them sensibly and how to leverage them for the greater good.

Python Syntax Errors

The first kind of error we'll cover is the humble Syntax Error. Syntax errors are also known as "parsing errors". Basically, parsing errors stop the program from executing. Take a look at this:

# parse_errors.py

print("program start")
print "program middle"
print("program end")

Now if we run this script using Python2, it will work just fine:

$ python2 parse_errors.py

program start
program middle
program end

But, of course, Python3 gives us a syntax error:

$ python3 parse_errors.py

  File "parse_errors.py", line 2
    print "program middle"
                         ^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print("program middle")?

In this case, none of the calls to print executed. Python 3 doesn't recognise parse_errors.py as valid Python code, so it doesn't execute anything. And the error message is pretty self explanitary.

Let's look at another example:

# parse_errors2.py

while 1
    print("winning")

Notice the conspicuous lack of :. This will cause a syntax error in both Python 2 and 3:

$ python3 parse_errors2.py

  File "parse_errors2.py", line 1
    while 1
          ^
SyntaxError: invalid syntax

Another interesting behaviour is what happens when we import modules with syntax errors:

# error_importer.py

print('importer start')
from parse_errors import *
print('importer end')

Now let's run it with Python2:

$ python2 error_importer.py

importer start
program start
program middle
program end
importer end

This is all as expected because the syntax is all valid in Python2. But running it in Python 3 is a little different:

$  python3 error_importer.py

importer start
Traceback (most recent call last):
  File "error_importer.py", line 2, in <module>
    from parse_errors import *
  File "/home/sheena/workspace/codementor/parse_errors.py", line 2
    print "program middle"
                         ^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print("program middle")?

Python 3 starts off by printing importer start then breaks down on the import statement. The error message is a bit more verbose this time. It also has a special name: Traceback. But before we delve into Tracebacks, it's important to know a bit about something called "Call Stack".

Exception Bubbling and the Traceback

Take a look at this code (there aren't any errors in it):

def func1():
    return 1

def func2():
    return func1()

def func3():
    return func2()

func3() ####

Once the various functions are defined, the marked line pushes func3 to the call stack, func3 then pushes func2, and func2 pushes func1. Now func1 is popped off the call stack as it returns 1 to func2, then func2 is popped as it returns 1 to func3.

If all that pushing and popping sounds confusing to you, take a look at this explanation of the Python call stack

Ok, now let's introduce an error:

# bubble.py

def func1():
    return 1/0   ##### ERROR

def func2():
    return func1()

def func3():
    return func2()

func3() ####

If you run this code it will raise an Exception:

$ python3 bubble.py

Traceback (most recent call last):
  File "bubble.py", line 10, in <module>
    func3() ####
  File "bubble.py", line 8, in func3
    return func2()
  File "bubble.py", line 5, in func2
    return func1()
  File "bubble.py", line 2, in func1
    return 1/0   ##### ERROR
ZeroDivisionError: division by zero

How does this work?

Well, it starts off just like before, with a func3, func2 and func1 being pushed to the call stack. Then there is a runtime error in func1 as it tries to divide by zero. This *raises anException*. AnExceptionis a special kind of Python object that stores information about what went wrong. Now thatException*bubbles* through the call stack. TheTraceback` is a message that describes the call stack as it caused the error, every stack frame is described briefly.

In a way, an Exception can be thought of as a special kind of return. Take a look at these two functions:

def func_a():
    return  "no error"
    print("this line never executes")

def func_b():
    a = 1/0
    print("this line never executes")

Calling func_a will get you a nice string. Calling func_b will raise an Exception and stop execution. And neither of those print statements will ever get executed.

Fixing Python Errors

Reading a Traceback is fairly straight-forward (even though the error itself is not always very easy to fix). Here is our Traceback from before:

Traceback (most recent call last):
  File "bubble.py", line 10, in <module>
    func3() ####
  File "bubble.py", line 8, in func3
    return func2()
  File "bubble.py", line 5, in func2
    return func1()
  File "bubble.py", line 2, in func1
    return 1/0   ##### ERROR
ZeroDivisionError: division by zero

The last part of the Traceback describes the actual Exception that was raised. Moving up slightly, you see that the Exception was raised by func1 in line 2 of our script. Moving up some more, you can see that func2 was called func1 in line 5 of the script, and so on.

The example I gave above was quite simple - the error was hard coded into the script and there were no arguments sent to the various functions. An Exception is usually raised because something unexpected went wrong. Most of the time, useful code is a lot more complicated, and sometimes Traceback isn't quite enough to reveal the error on its own. In that case, you have a few different tools that can help. I won't explain these tools in depth, as it's a bit beyond the scope of this article, but since we are on the topic of fixing errors...

• print: You can print out variable values and such, to get insight into what happened. This is commonly called "print debugging". It's quick and dirty and there are usually better ways to get things done. But if you find yourself print debugging, please remember to remove the print statements once you are done.
• logging.debug: Python has a special logging module that is a lot more intelligent than print and is generally considered a better practice, but there is also a learning curve. If you don't know how to use the logging module yet, don't worry too much as it's not strictly necesary (but it is rather nice). You can learn about it here
• Python also has a built-in debugger that you can use to walk through your code line by line. You can find the latest documentation here. It lets you explore and interact with your program as it runs. This can be immensely useful.
• There are also a few libraries that make Traceback more informative. One example is TBVaccine.

There is, of course, a lot to be said about preventing errors in the first place. But again, that's a bit out of scope. Things worth looking into here are:

• Testing your code. I suggest using pytest and occasionally, doctest
• Type hinting. Using type hints in your code can eliminate entire categories of error completely, while being useful as documentation. Take a look at mypy for more details.

Exceptions as objects

Everything in Python is an object. That includes Exceptions. When an Exception is raised, that means an instance of the Exception class is created. And since the Exception class is, in fact, a class, it can be subclassed.

Here we see that KeyError is a subclass of Exception:

>>> Exception
<class 'Exception'>
>>> Exception.__doc__
'Common base class for all non-exit exceptions.'
>>> KeyError
<class 'KeyError'>
>>> KeyError.__doc__
'Mapping key not found.'
>>> issubclass(KeyError,Exception)
True

Note that we didn't need to import anything to execute the above code. The basic built in Exception is always in the scope. If you were to create your own Exception classes, you would need to import them whenever you refer to them, just like regular classes.

We can do the same thing with IndexError

>>> IndexError
<class 'IndexError'>
>>> IndexError.__doc__
'Sequence index out of range.'
>>> issubclass(IndexError,Exception)
True

Surviving errors

Take a look at this:

some_function()
another_function()

If some_function raises an Exception, then another_function will never get called when you run this script. The program will simply crash. This can be...annoying. Errors are a part of life. When you become aware of an error you have made, you handle it, then you move on to whatever comes next. Python can be similarly responsible.

There is syntax built into Python to allow it to recover from error. This syntax empowers you to make your code robust. It also empowers you to make your code into something brittle and opaque, so it should be handelled with care.

In this section, we'll handle the syntax and the error handling mechanisms that Python exposes. Once we are done with the basic syntax, we'll move on to a discussion on best practices.

Python Try Except

I'll be sticking to Python3 syntax. It's a little different from Python2, but it basically functions the same way.

Consider the following program:

# divider.py

print("welcome to the number divider program")
x = float(input("please input a number:\n"))
y = float(input("please input another number:\n"))
answer = x/y
print(f"The answer is {answer}")

Let's play with it a bit. First, let's enter some integers:

$ python3 divider.py

welcome to the number divider program
please input a number:
1
please input another number:
2
The answer is 0.5

Now for some floats:

$ python3 divider.py

welcome to the number divider program
please input a number:
12.3
please input another number:
45.6
The answer is 0.26973684210526316

Let's break it by entering some english text:

$ python3 divider.py

welcome to the number divider program
please input a number:
a number
Traceback (most recent call last):
  File "divider.py", line 11, in <module>
    x = float(input("please input a number:\n"))
ValueError: could not convert string to float: 'a number'

That's an Exception we'll need to be able to recover from. This is an illustration of one of the golden rules of user interface development: Always expect your users to do weird things! Your program should not crash every time a user makes a mistake!

Now let's divide by zero:

$ python3 divider.py

welcome to the number divider program
please input a number:
1
please input another number:
0
Traceback (most recent call last):
  File "divider.py", line 13, in <module>
    answer = x/y
ZeroDivisionError: float division by zero

Last but not least, run the program and press Ctrl+C part way through:

$ python3 divider.py
welcome to the number divider program
please input a number:
4
please input another number:
^CTraceback (most recent call last):
  File "divider.py", line 12, in <module>
    y = float(input("please input another number:\n"))
KeyboardInterrupt

This last Exception isn't really an error. Maybe the user wants to quit the program. That's totally fine and acceptable.

Python Catch Exception

Now let's make our code a little bit more robust. We'll start off by introducing a function that is all about getting valid info from users.

# divider2.py

def get_user_input(message:str) -> float:
     while True:
         try:
             return float(input(f"{message}:\n"))
         except:
             print("Oops!  That was no valid number.  Try again...")


print("welcome to the number divider program")
x = get_user_input("please input a number")
y = get_user_input("please input another number")
answer = x/y
print(f"The answer is {answer}")

Let's take a closer look at get_user_input function.

As we know from the last version of the code, float(input(f"{message}:\n")) could raise an Exception. We would like to be able to recover from that Exception. So we'll put it in a try block. Then the except block says what to do if an Exception happens.

Now let's try running the program and entering some invalid numbers:

python3 divider2.py

welcome to the number divider program
please input a number:
12,3
Oops!  That was no valid number.  Try again...
please input a number:
12.3
please input another number:
foooooooooooo
Oops!  That was no valid number.  Try again...
please input another number:
baaaaaaaa
Oops!  That was no valid number.  Try again...
please input another number:
2
The answer is 6.15

Wonderful! Now we can deal with the invalid inputs. But there is a serious problem in this code, can you spot it?

Catching the right Exception

Let's try exiting the program before it has finished runnning (Ctrl+C):

welcome to the number divider program
please input a number:
^COops!  That was no valid number.  Try again...
please input a number:
1
please input another number:
^COops!  That was no valid number.  Try again...
please input another number:
2
The answer is 0.5

The problem is that the except block doesn't care about what type of Exception it catches. We need it to only catch ValueErrors while letting every other Exception bubble through the call stack.

We'll change our function to look like this:

def get_user_input(message:str) -> float:
     while True:
         try:
             return float(input(f"{message}:\n"))
         except ValueError:
             print("Oops!  That was no valid number.  Try again...")


print("welcome to the number divider program")
x = get_user_input("please input a number")
y = get_user_input("please input another number")
answer = x/y
print(f"The answer is {answer}")

And then we run the script again. This time we'll enter an invalid number before trying to exit.

welcome to the number divider program
please input a number:
1..2
Oops!  That was no valid number.  Try again...
please input a number:
^CTraceback (most recent call last):
  File "divider.py", line 10, in <module>
    x = get_user_input("please input a number")
  File "divider.py", line 4, in get_user_input
    return float(input(f"{message}:\n"))
KeyboardInterrupt

Looking good! Now we can recover from some user errors! But...the program output is pretty ugly when the user tries to exit so let's pretty it up:

def get_user_input(message:str) -> float:
     while True:
         try:
             return float(input(f"{message}:\n"))
         except ValueError:
             print("Oops!  That was no valid number.  Try again...")


try:
    print("welcome to the number divider program")
    x = get_user_input("please input a number")
    y = get_user_input("please input another number")
    answer = x/y
    print(f"The answer is {answer}")
except KeyboardInterrupt:
    print("We bid you farewell!")
    print("Please rate us on the app store")

Now if the user presses Ctrl+C, they'll get a nice message. Take note of the fact that the try block is multiple lines long. This means that any keyboard interrupt issued at any point during the execution of any of those lines will get caught and dealt with by the except block.

Working with Exception objects

Next up, let's make our ValueError error message a bit more informative:

def get_user_input(message:str) -> float:
     while True:
         try:
            return float(input(f"{message}:\n"))
         except ValueError as e:   ###
            print(f"Oops! {e}. Try again...")

If you use an as like above, you will have access to the Exception object. In this case, we'll just be showing the actual error message to the user. Note, this isn't a full Traceback, and that e is a ValueError object, not a string! It just has a very friendly string representation.

Run the program again and give it some invalid input:

welcome to the number divider program
please input a number:
qw
Oops! could not convert string to float: 'qw'. Try again...

Multiple except blocks, and the raise statement

Let's add some unnecessary complications to our code:

def get_user_input(message:str) -> float:
     while True:
        try:
            return float(input(f"{message}:\n"))
        except KeyboardInterrupt:
            print("Interrupted the process when an input was required...")
            raise
        except ValueError:
            print(f"Oops!. Try again...")

Now try pressing Ctrl+C at different points in the program. What we've demonstrated here is that you can have multiple except blocks for a single try if you need to. This way you can handle different errors in different ways. That's handy.

Then there is that raise statment, which re-raises the Exception that was caught. A more verbose version of this code can be seen below:

def get_user_input(message:str) -> float:
     while True:
        try:
            return float(input(f"{message}:\n"))
        except KeyboardInterrupt as e:
            print("Interrupted the process when an input was required...")
            raise e
        except ValueError:
            print(f"Oops!. Try again...")

In fact, you can raise Exceptions whenever you want with raise. For example:

>>> raise Exception("Oh Noes!")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
Exception: Oh Noes!

I'll leave it up to you, dear reader, to put in some error handling for the ZeroDivisionError.

Which except block?

Consider the following:

class Error1(Exception): pass

class Error2(Error1): pass

class Error3(Error2): pass

try:
    raise Error2()  ###
except Error1:
    print("1")
except Error2:
    print("2")
except Error3:
    print("3")

print('done')

Now Error2 is a subclass of Error1. That means our first except block will be executed. Once it has finished, none of the other except blocks will be considered. In other words, Python only executes the first matching except block and is aware of inheritance.

If you run the above code you'll get:

1
done

If you were to raise Error1 or Error2 instead, the output would be exactly the same. Try it yourself and see what happens if you rearrange the except blocks.

For example, what happens if you run this script?

class Error1(Exception): pass

class Error2(Error1): pass

class Error3(Error2): pass

try:
    raise Error2()  ###
except Error2:
    print("2")
except Error1:
    print("1")
except Error3:
    print("3")

print('done')

One except, multiple Exception types

You can also have one except statement with multiple error types, like so:

try:
    foo()
except (KeyError, ZeroDivisionError , NameError):
    handle_error()

You'll get the error object, just like before:

try:
    foo()
except (KeyError, ZeroDivisionError , NameError) as e:
    handle_error(e)

So if foo() raises a KeyError, then e would be an instance of KeyError and if foo() raises a NameError, then e could be an instance of NameError.

Else

You all know about if...else right? Well, else can also be used in the context of Exception handling. Consider the following:

def useless_function(key):
    d = {1:1}
    try:
        print(d[key])
    except KeyError:
        print("KeyError handled")
    else:
        print("else")

Now let's call the useless function:

>>> useless_function(1)
1
else
>>> useless_function(2)
KeyError handled

So if there is a KeyError, it is handled by the except block. Otherwise, the else block is executed.

Here is the useless function as pseudocode:

def useless_function(key):
    d = {1:1}
    try:
        print(d[key])
    if there is a KeyError:
        print("KeyError handled")
    else:
        print("else")

Let's add another except block:

def useless_function(key):
    d = {1:1}
    try:
        print(d[key])
    except KeyError:
        print("KeyError handled")
    except NameError:
        handle_name_error()
    else:
        print("else")

Now the pseudocode looks like this:

def useless_function(key):
    d = {1:1}
    try:
        print(d[key])
    if there is a KeyError:
        print("KeyError handled")
    elif there is a NameError:
        handle_name_error()
    else:
        print("else")

Finally

Sometimes it is necessary to perform certain actions, regardless if an Exception is present. For example, when you are writing to a file, which should close when your code is finished. Or when you are dealing with a connection to a database and the connection should close when your code is finished. Or maybe there is some other cleanup process that your code needs to perform. You could do something like this:

try:
    raise KeyError()
except KeyError:
    print("KeyError")
else:
    print("No error")
finally:
    print("cleanup finally")


print("afterwards")

If you run the above code, you'll get this output:

KeyError
cleanup finally
afterwards

Now let's edit the code so no Exception gets raised:

try:
    pass ######
except KeyError:
    print("KeyError")
else:
    print("No error")
finally:
    print("cleanup finally")


print("afterwards")

The output is now:

No error
cleanup finally
afterwards

Let's put our error back in and raise it in the except block

try:
    raise KeyError()    ######
except KeyError:
    print("KeyError")
    raise               ######
else:
    print("No error")
finally:
    print("cleanup finally")


print("afterwards")

Now this happens:

KeyError
cleanup finally
Traceback (most recent call last):
  File "animal_errors.py", line 4, in <module>
    raise KeyError()
KeyError

Let's raise a NameError instead:

try:
    raise NameError()
except KeyError:
    print("KeyError")
    raise
else:
    print("No error")
finally:
    print("cleanup finally")


print("afterwards")

The output now looks like this:

cleanup finally
Traceback (most recent call last):
  File "animal_errors.py", line 4, in <module>
    raise NameError()
NameError

In this pattern, "cleanup finally" is always printed. It is present in every single output above. This can be very useful, especially if you need to perform some kind of cleanup action NO MATTER WHAT, as you can put that action inside a finally block.

Notice that when an Exception is re-raised or not handled for any reason, the finally block is executed before the Exception is allowed to bubble through the call stack.

Another thing worth knowing is that you don't need any except blocks for a finally to work. The following code is completely valid:

try:
    raise NameError()
finally:
    print("cleanup finally")


print("afterwards")

The output:

cleanup finally
Traceback (most recent call last):
  File "animal_errors.py", line 4, in <module>
    raise NameError()
NameError

That's it for the syntax. Well done for getting this far Next up, we'll talk about how this stuff can be dangerous.

With great power...

This try...except stuff is pretty powerful. It gives you the power to make user friendly and resilient code. It also gives you the power to build brittle and opaque code, so it should be handled with care. There are a few tricks I've learnt over the years that's saved me a lot of time:

Be specific about the Exception you are catching!!!!!

You see all those exclamation marks up there? They are there for a very good reason. There is an anti-pattern I've seen again and again and I've made this painful mistake before.

def get_user_input(message:str) -> float:
    while True:try:
        return float(inpit(f"{message}:\n"))
    except:
        print(f"Oops! Try again...")

This should look somewhat familiar. We have an "except all" block that messes with our keyboard interrupt. We've already seen that that can be annoying. But it can be much more sinister than that.

I've added an extra bug into the code above, can you see it? I used the word inpit instead of input. Now run the code...if you dare.

This is what the output looks like:

Oops! Try again...
Oops! Try again...
Oops! Try again...
Oops! Try again...
Oops! Try again...
Oops! Try again...
Oops! Try again..
etc

And when you try to escape with a keyboard interrupt, it just says Oops! Try again.... This is terrible.

In this case, the error was a pretty simple one: a misspelling. Chances are your editor would have warned you about it before you even ran the code. But a lot of runtime errors are a lot more complicated. Sometimes there are edge cases, logic errors, and misconceptions in the code that really should raise an Exception. Exceptions are great because they tell us that there is something that needs to be fixed.

When your code fails, as it will, it should fail loudly and informatively. That way you can jump in and quickly fix the bug. Silencing and misrepresenting errors is a bad idea.

Imagine writing some code to operate a nuclear power plant. Imagine misrepresenting bugs in your code. There is a word that comes to mind: BOOM!

If you want to learn more about this particular anti-pattern, take a look at this entertaining blog post.

Be specific about where the Exceptions can be raised

Consider...

try:
    this()
    that()
    other_stuff()
    this_other_thing()
    a_complicated function() ####
    some_heavy_stuff()
    etc()
except SomeException:
    handle_exception()

Now let's say that the only function that should be able to raise an instance of SomeException in this code is a_complicated_function. Then there are a few problems here:

• The above fact simply isn't clear from this code. You'd have to do some serious digging if you needed to figure that out.
• If some other line that wasn't meant to raise the SomeException did raise SomeException, then you wouldn't know there was a problem because the except clause would just handle it.

This would be a better way to write your code:

this()
that()
other_stuff()
this_other_thing()

try:
    a_complicated function() ####
except SomeException:
    handle_exception()
else:
    some_heavy_stuff()
    etc()

This version of the code is clear and doesn't hide any errors.

Put your except blocks in a useful order

Here's some code we covered before:

class Error1(Exception): pass

class Error2(Error1): pass

class Error3(Error2): pass

try:
    raise Error2()  ###
except Error1:
    print("1")
except Error2:
    print("2")
except Error3:
    print("3")

print('done')

Running this gives you:

1
done

If you were to raise Error1 or Error3 instead of Error2, then the output would be exactly the same. The first except block is general enough to catch all the errors.

In other words, this code does the exact same thing no matter which of the three errors are raised:

class Error1(Exception): pass

class Error2(Error1): pass

class Error3(Error2): pass

try:
    raise Error2()  # or Error1() or Error3
except Error1:
    print("1")

print('done')

The rule here is if you have multiple except blocks that are catching Exceptions inheriting from each other, then be sure to order your except blocks from most to least specific:

class Error1(Exception): pass

class Error2(Error1): pass

class Error3(Error2): pass

try:
    raise Error2()  ###
except Error3:
    print("3")
except Error2:
    print("2")
except Error1:
    print("1")

print('done')

Try running the code and raising different errors. Now the error messages describe what actually happened. This is a good thing.

In general

This section can be summaried as:

• Only handle the Exceptions that you explicitly want to handle.
• Bugs in your code should look like bugs.
• It's best to know about bugs as early as possible.
• If a bug happens, you want to have enough information to fix it.

Conclusion

Well done! You should be a pro when it comes to handelling Exceptions now. You know what they are, how they interact with the call stack, and how to handle them. Most importantly, you know how to handle Exception well! Your knowledge of best practices will reduce the number of bugs in your code and will likely help you debug actual errors.

To be honest, there is a lot more to be said about Exception. This article dealt with handling Exception, but they can also be leveraged in different ways. If you really want to be a pro, then the next thing I would suggest you learn is how to use AssertionErrors effectively. You can also learn about creating and raising custom Exception.