.. _ch04-python-exception-debugging: =============================================================== Exception handing and debugging in Python =============================================================== You may have already seen Python's exception handling if you have tried illegal operations and Python interpreter catches them, for instance:: >> 1/0 Traceback (most recent call last): File "", line 1, in ZeroDivisionError: integer division or modulo by zero The first part in the last line says ``ZeroDivisionError``. This is one of the Python's built-in Exceptions of arithematic errors. Exceptions are raised by different kinds of errors arising when executing Python code. In your own code, you may also catch errors, or define custom error types. Please see a more complete list of Python's built-in Exceptions at https://docs.python.org/2/library/exceptions.html. In the below, we shall take a look at some examples which will teach us to how to use exception handlings in Python programs. In addition, we are going to study a few debugging strategies at the end. .. _ch04-python-exceptions: -------------------------------------------- Exceptions -------------------------------------------- Exceptions are raised by errors in Python: * ``TypeError``: unsupported operation:: >>> 1+'apple' Traceback (most recent call last): File "", line 1, in TypeError: unsupported operand type(s) for +: 'int' and 'str' * ``KeyError``: invalid use of key:: >>> eng2kor = {'three': 'set', 'two': 'dool', 'one': 'haha'} eng2kor[0] Traceback (most recent call last): File "", line 1, in KeyError: 0 * ``indexError``: invalid use of index:: >>> eng2kor.values()[4] Traceback (most recent call last): File "", line 1, in IndexError: list index out of range * ``AttributeError``: attribute reference or assignment failure:: >>> eng2kor.append('foo') Traceback (most recent call last): File "", line 1, in AttributeError: 'dict' object has no attribute 'append' -------------------------------------------- Catching exceptions -------------------------------------------- ^^^^^^^^^^^^^^^^^^^^^^^ try/except ^^^^^^^^^^^^^^^^^^^^^^^ The ``try`` statement works as follows: 1. First, the statement(s) between the ``try`` and ``except`` is executed, 2. If no exception occurs, the ``except`` clause is skipped and execution of the ``try`` statement is finished, 3. If an exception occurs during the execution of the ``try`` clause, the rest of the clause is skipped. Then if its type matches the exception named after the ``except`` keyword, the except clause is executed, and then execution continues after the ``try-except`` satement. 4. If an exception occurs which does not match the exception named in the except clause, it is passed on to outer ``try`` statement; if no handler is found, it is an *unhandled exception* and execution stops with a message as shown above :ref:`ch04-python-exceptions`. Now let's take a look at few examples. In the first example, you will see the error message if you enter an input that is not a number so that it cannot be converted to an interger: .. literalinclude:: ./codes/try_except1.py :language: python :linenos: Exception handlers don’t just handle exceptions if they occur immediately in the try clause, but also if they occur inside functions that are called (even indirectly) in the try clause. For example: .. literalinclude:: ./codes/try_except2.py :language: python :linenos: ^^^^^^^^^^^^^^^^^^^^^^^ try/finally ^^^^^^^^^^^^^^^^^^^^^^^ The ``try`` statement can be used with another optional statement ``finally`` which is intended to define clean-up actions that must be executed under all circumstances. For instance, .. literalinclude:: ./codes/try_finally.py :language: python :linenos: Such a clean-up step can be useful for resource management, e.g., closing a file. ^^^^^^^^^^^^^^^^^^^^^^^ Raising exceptions ^^^^^^^^^^^^^^^^^^^^^^^ Let's now take a look at how to *raise* an exception in your code. In the following example, we run an iteration using our old example of the Newton's root finding algorithm. An idea is to raise an exception of ``StopIteration`` (see line 34 in the following example) when the exit condition is satisfied, and use the exception to determine a proper convergence criterion: .. literalinclude:: ./codes/try_except3.py :language: python :linenos: .. _ch04-python-debugging: -------------------------------------------- Python debugging -------------------------------------------- ---------------------------------------------------------------------------- Print statements ---------------------------------------------------------------------------- Print statements can be added almost anywhere in a Python code to print things out to the terminal window as it goes along. You might want to put some special symbols in debugging statements to flag them as such, which makes it easier to see what output is your debug output and also makes it easier to find them again later to remove from the code, e.g. you might use "+++" or "DEBUG". As an example, suppose you are trying to better understand Python namespaces and the difference between local and global variables. Then this code might be useful: .. literalinclude:: ./codes/debugging1.py :language: python :linenos: .. note:: In the above example, we see two tokens, ``%s`` for strings and ``%d`` for numbers, as placeholders of what comes after the ``%`` sign, e.g., ``% (x, y, z)``. See more examples `here `_. Here the print function in the definition of `f(x)` is being used for debugging purposes. Executing this code gives:: $ python debugging1.py +++ before calling f: x = 3.0, y = -22.0 +++ in function f: x = 13.0, y = -22.0, z = 3.0 +++ after calling f: x = 3.0, y = 13.0 If you are printing large amounts you might want to write to a file rather than to the terminal. .. _ch04-python-pdb: ------------------------------------------------------------------------ pdb debugger ------------------------------------------------------------------------ Inserting print statements may work best in some situations, but it is often better to use a *debugger*. The Python debugger pdb is very easy to use, often even easier than inserting print statements and well worth learning. See the `pdb documentation `_ for more information. You can insert *breakpoints* in your code (see line 17 in the below) where control should pass back to the user, at which point you can query the value of any variable, or step through the program line by line from this point on: .. literalinclude:: ./codes/debugging2.py :language: python :linenos: Of course one could set multiple breakpoints with other ``pdb.set_trace()`` commands. For the above example we might do this as below. Upon running the above example, we get the prompt for the ``pdb`` shell when we hit the breakpoint:: $ python debugging2.py > /Users/dongwook/Repos/ucsc/soe/teaching/2015-2016/Fall/AMS209/lectureNote/chapters/chapt04/codes/debugging2.py(18)f() -> return x (Pdb) p x 13.0 (Pdb) p y -22.0 (Pdb) p z 3.0 Note that ``p`` is short for ``print`` which is the same as ``gdb`` command. You could also type ``print x`` but this would then execute the Python print command instead of the debugger command (though in this case it would print the same thing). There are many other ``pdb`` commands, such as ``next`` to execute the next line, ``continue`` to continue executing until the next breakpoint, etc. (See `the pdb documentation `_ for more details.) You can also run the code as a script from the script mode and again you will be put into the pdb shell when the breakpoint is reached:: $ python debugging2.py > /Users/dongwook/Repos/ucsc/soe/teaching/2015-2016/Fall/AMS209/lectureNote/chapters/chapt04/codes/debugging2.py(18)f() -> return x (Pdb) p z 3.0 (Pdb) continue x = 3.0 y = 13.0 ----------------------------------------------------------------------- Debugging after an exception occurs ----------------------------------------------------------------------- Often code has bugs that cause an exception to be raised, resulting the program to halt execution. Consider the following example: .. literalinclude:: ./codes/pdb_example.py :language: python :linenos: Executing this will show:: >>> execfile('pdb_example.py') Traceback (most recent call last): File "", line 1, in File "pdb_example.py", line 14, in soln = division_by_zero(float(i)) File "pdb_example.py", line 10, in division_by_zero x/=x-1 ZeroDivisionError: float division by zero At some point there must be a division by zero. To figure out when this happens, we could insert a `pdb.set_trace()` command in the loop and step through it until the error occurs and then look at `i`, but we can do so even more easily using a *post-mortem* analysis after it dies, using `pdb.pm()`:: >>> import pdb >>> pdb.pm() >/Users/dongwook/Repos/ucsc/soe/teaching/2015-2016/Fall/AMS209/lectureNote/chapters/chapt04/codes/pdb_example.py(10)division_by_zero() -> x/=x-1 (Pdb) p i 1 (Pdb) p soln 2.0 (Pdb) p x 1.0 This starts up ``pdb`` at exactly the point where the exception is about to occur. We see that the divide by zero happens when `i = 1`. ---------------------------------------------------------- Using pdb from IPython ---------------------------------------------------------- In IPython it's even easier to do this post-mortem analysis. Let's rerun the example in IPython again:: In [1]: run pdb_example.py --------------------------------------------------------------------------- ZeroDivisionError Traceback (most recent call last) /Users/dongwook/Repos/ucsc/soe/teaching/2015-2016/Fall/AMS209/lectureNote/chapters/chapt04/codes/pdb_example.py in () 12 13 for i in range(5,0,-1): ---> 14 soln = division_by_zero(float(i)) /Users/dongwook/Repos/ucsc/soe/teaching/2015-2016/Fall/AMS209/lectureNote/chapters/chapt04/codes/pdb_example.py in division_by_zero(x) 8 9 def division_by_zero(x): ---> 10 x/=x-1 11 return x 12 ZeroDivisionError: float division by zero In order to start ``pdb`` just type:: In [2]: pdb Automatic pdb calling has been turned ON and then running ``pdb`` will be automatically invoked if an exception occurs:: In [3]: run pdb_example.py --------------------------------------------------------------------------- ZeroDivisionError Traceback (most recent call last) /Users/dongwook/Repos/ucsc/soe/teaching/2015-2016/Fall/AMS209/lectureNote/chapters/chapt04/codes/pdb_example.py in () 12 13 for i in range(5,0,-1): ---> 14 soln = division_by_zero(float(i)) /Users/dongwook/Repos/ucsc/soe/teaching/2015-2016/Fall/AMS209/lectureNote/chapters/chapt04/codes/pdb_example.py in division_by_zero(x) 8 9 def division_by_zero(x): ---> 10 x/=x-1 11 return x 12 ZeroDivisionError: float division by zero > /Users/dongwook/Repos/ucsc/soe/teaching/2015-2016/Fall/AMS209/lectureNote/chapters/chapt04/codes/pdb_example.py(10)division_by_zero() 9 def division_by_zero(x): ---> 10 x/=x-1 11 return x ipdb> p i 1 ipdb> p soln 2.0 ipdb> p x 1.0 ipdb> q In [4]: pdb Automatic pdb calling has been turned OFF As just shown, typing ``pdb`` again will turn the pdb session off.