Exception handing and debugging in Python

You may have already seen Python’s exception handling if you have tried illegal operations and Python interpreter catches them, for instance

>> 1/0
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ZeroDivisionError: division by zero

The first part in the last line says ZeroDivisionError. This is one of the Python’s built-in Exceptions of arithematic errors.

Exceptions are raised by different kinds of errors arising when executing Python code. In your own code, you may also catch errors, or define custom error types. Please see a more complete list of Python’s built-in Exceptions at https://docs.python.org/3/library/exceptions.html.

In the below, we shall take a look at some examples which will teach us to how to use exception handlings in Python programs. In addition, we are going to study a few debugging strategies at the end.

Exceptions

Exceptions are raised by errors in Python:

  • TypeError: unsupported operation

    >>> 1+'apple'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for +: 'int' and 'str'

  • KeyError: invalid use of key

>>> eng2kor = {'three': 'set', 'two': 'dool', 'one': 'haha'}
eng2kor[0]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
KeyError: 0

  • indexError: invalid use of index

    >>> a = [1, 2, 3]
>>> a[4]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IndexError: list index out of range

  • AttributeError: attribute reference or assignment failure

    >>> eng2kor.append('foo')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'dict' object has no attribute 'append'

Catching exceptions

try/except

The try statement works as follows:

  1. First, the statement(s) between the try and except is executed,
  2. If no exception occurs, the except clause is skipped and execution of the try statement is finished,
  3. If an exception occurs during the execution of the try clause, the rest of the clause is skipped. Then if its type matches the exception named after the except keyword, the except clause is executed, and then execution continues after the try-except satement.
  4. If an exception occurs which does not match the exception named in the except clause, it is passed on to outer try statement; if no handler is found, it is an unhandled exception and execution stops with a message as shown above Exceptions.

Now let’s take a look at few examples. In the first example, you will see the error message if you enter an input that is not a number so that it cannot be converted to an interger:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
"""

lectureNote/chapters/chapt05/codes/try_except1.py

try-except example, originally from https://docs.python.org/3/tutorial/errors.html

"""


while True:
    try:
        x = int(input("Please enter a number: "))
        break
    except ValueError:
        print("Oops!  Could not invert your input to an integer.  Try again...")

Exception handlers don’t just handle exceptions if they occur immediately in the try clause, but also if they occur inside functions that are called (even indirectly) in the try clause. For example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
"""

lectureNote/chapters/chapt05/codes/try_except2.py

try-except example, originally from https://docs.python.org/3/tutorial/errors.html

"""

def this_fails():
    print('try-except example')
    print('1. valid division 1/2')
    print('2. invalid division 1/0')
    option = int(input("Please enter an option number -- 1 or 2: "))

    if option is 1:
        x = 1./2
    elif option is 2:
        x = 1./0
    return x

try:
    x = this_fails()
    print(x)
except ZeroDivisionError:
    print('Handling run-time error')

try/finally

The try statement can be used with another optional statement finally which is intended to define clean-up actions that must be executed under all circumstances. For instance,

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
"""

lectureNote/chapters/chapt05/codes/try_finally.py

try-finally example, originally from https://docs.python.org/3/tutorial/errors.html

"""

def divide(x, y):
     try:
         result = x / y
     except ZeroDivisionError:
         print("Division by zero!")
     else:
         print("Result is", result)
     finally:
         print("Executing finally clause, Thanks for testing me!")

if __name__ == '__main__':
    divide(1.0,2.0)
    divide(1.0,0.0)

Such a clean-up step can be useful for resource management, e.g., closing a file.

Raising exceptions

Let’s now take a look at how to raise an exception in your code. In the following example, we run an iteration using our old example of the Newton’s root finding algorithm. An idea is to raise an exception of StopIteration (see line 33 in the following example) when the exit condition is satisfied, and use the exception to determine a proper convergence criterion:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
"""

lectureNote/chapters/chapt04/codes/try_except3.py

try-except example, written by Prof. Dongwook Lee for AMS 209.

"""

import numpy as np
MaxIter = 100


def funct(x):
    # f(x) = x + exp(x) + 10/(1+x^2) - 5
    f  = x  + np.exp(x) + 10/(1+x**2)-5
    df = 1. + np.exp(x) - 20.*x/(1.+x**2)**2
    fdf = f/df

    return fdf



def root_finder(threshold):

    # declare global variables
    global error, soln, MaxIter

    # append the newest error in the error list
    error.append(funct(soln[-1]))

    # Raise StopIteration exception if stop condition is met
    if (abs(error[-1]) < threshold or len(soln) > MaxIter):
        raise StopIteration

    # Newton's iteration
    x = soln[-1] - error[-1]

    # append the newest search result to the soln list
    soln.append(x)

    # tuple output
    return soln,error


if __name__ == '__main__':

    # define a list for the error placeholder
    error = list()

    # take an initial value as an input from users
    x = float(input("Please enter an initial search value: "))

    # define a list for the soln placeholder
    soln = list()

    # append the last result to the soln list
    soln.append(x)

    # take a threshold value as an input from users
    threshold = float(input("Please enter a threshold value: "))

    while True:
        try:
            (soln,error) = root_finder(threshold)
        except StopIteration:
            break

    print(len(soln))
    print(soln[-1])
    print(error[-1])

    for i in range(len(soln)):
        print(i, soln[i], error[i])

Python debugging

pdb debugger

Inserting print statements may work best in some situations, but it is often better to use a debugger. The Python debugger pdb is very easy to use, often even easier than inserting print statements and well worth learning. See the pdb documentation for more information.

You can insert breakpoints in your code (see line 17 in the below) where control should pass back to the user, at which point you can query the value of any variable, or step through the program line by line from this point on:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
"""
lectureNote/chapters/chapt05/codes/debugging2.py

Originally from
http://faculty.washington.edu/rjl/classes/am583s2014/notes/python_debugging.html

Debugging demo using print statements

Debugging demo using pdb.
"""

x = 3.
y = -22.

def f(z):
    x = z+10
    import pdb;pdb.set_trace()
    return x

y = f(x)

print("x = ",x)
print("y = ",y)

Of course one could set multiple breakpoints with other pdb.set_trace() commands. For the above example we might do this as below. Upon running the above example, we get the prompt for the pdb shell when we hit the breakpoint

$ python3 debugging2.py

> /Users/ylee/Documents/ucsc/18_spring/ams129/lecture_notes/source/chapters/chapt05/codes/debugging2.py(18)f()
-> return x
(Pdb) p x
13.0
(Pdb) p y
-22.0
(Pdb) p z
3.0

Note that p is short for print which is the same as gdb command. You could also type print x but this would then execute the Python print command instead of the debugger command (though in this case it would print the same thing).

There are many other pdb commands, such as next to execute the next line, continue to continue executing until the next breakpoint, etc. (See the pdb documentation for more details.)

You can also run the code as a script from the script mode and again you will be put into the pdb shell when the breakpoint is reached

$ python3 debugging2.py

> /Users/ylee/Documents/ucsc/18_spring/ams129/lecture_notes/source/chapters/chapt05/codes/debugging2.py(18)f()
-> return x
(Pdb) p z
3.0
(Pdb) continue
x =  3.0
y =  13.0

Debugging after an exception occurs

Often code has bugs that cause an exception to be raised, resulting the program to halt execution. Consider the following example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
"""

lectureNote/chapters/chapt05/codes/pdb_example.py

"""
        
def division_by_zero(x):
    x/=x-1
    return x

# import pdb
for i in range(5,0,-1):
    # pdb.set_trace()
    soln = division_by_zero(float(i))

Executing this will show

>>> exec(open('./pdb_example.py').read())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<string>", line 14, in <module>
  File "<string>", line 8, in division_by_zero
ZeroDivisionError: float division by zero

At some point there must be a division by zero. To figure out when this happens, we could insert a pdb.set_trace() command in the loop and step through it until the error occurs and then look at i, but we can do so even more easily using a post-mortem analysis after it dies, using pdb.pm()

>>> import pdb
>>> pdb.pm()
> <string>(8)division_by_zero()
(Pdb) p i
1
(Pdb) p soln
2.0
(Pdb) p x
1.0

This starts up pdb at exactly the point where the exception is about to occur. We see that the divide by zero happens when i = 1.

Using pdb from IPython

In IPython it’s even easier to do this post-mortem analysis. Let’s rerun the example in IPython again

In [1]: run pdb_example.py
---------------------------------------------------------------------------
ZeroDivisionError                         Traceback (most recent call last)
~/Documents/ucsc/18_spring/ams129/lecture_notes/source/chapters/chapt05/codes/pdb_example.py in <module>()
     12 for i in range(5,0,-1):
     13     #pdb.set_trace()
---> 14     soln = division_by_zero(float(i))

~/Documents/ucsc/18_spring/ams129/lecture_notes/source/chapters/chapt05/codes/pdb_example.py in division_by_zero(x)
      6
      7 def division_by_zero(x):
----> 8     x/=x-1
      9     return x
     10

ZeroDivisionError: float division by zero

In order to start pdb just type

In [2]: pdb
Automatic pdb calling has been turned ON

and then running pdb will be automatically invoked if an exception occurs

In [3]: run pdb_example.py
---------------------------------------------------------------------------
ZeroDivisionError                         Traceback (most recent call last)
~/Documents/ucsc/18_spring/ams129/lecture_notes/source/chapters/chapt05/codes/pdb_example.py in <module>()
     12 for i in range(5,0,-1):
     13     #pdb.set_trace()
---> 14     soln = division_by_zero(float(i))

~/Documents/ucsc/18_spring/ams129/lecture_notes/source/chapters/chapt05/codes/pdb_example.py in division_by_zero(x)
      6
      7 def division_by_zero(x):
----> 8     x/=x-1
      9     return x
     10

ZeroDivisionError: float division by zero
> /Users/ylee/Documents/ucsc/18_spring/ams129/lecture_notes/source/chapters/chapt05/codes/pdb_example.py(8)division_by_zero()
      6
      7 def division_by_zero(x):
----> 8     x/=x-1
      9     return x
     10

ipdb> p i
1
ipdb> p soln
2.0
ipdb> p x
1.0
ipdb> q

In [4]: pdb
Automatic pdb calling has been turned OFF

As just shown, typing pdb again will turn the pdb session off.