Added sections 2-4
This commit is contained in:
David Beazley
393
Notes/03_Program_organization/03_Error_checking.md
Normal file
393
Notes/03_Program_organization/03_Error_checking.md
Normal file
@@ -0,0 +1,393 @@
|
||||
# 3.3 Error Checking
|
||||
|
||||
This section discusses some aspects of error checking and exception handling.
|
||||
|
||||
### How programs fail
|
||||
|
||||
Python performs no checking or validation of function argument types or values.
|
||||
A function will work on any data that is compatible with the statements in the function.
|
||||
|
||||
```python
|
||||
def add(x, y):
|
||||
return x + y
|
||||
|
||||
add(3, 4) # 7
|
||||
add('Hello', 'World') # 'HelloWorld'
|
||||
add('3', '4') # '34'
|
||||
```
|
||||
|
||||
If there are errors in a function, they will show up at run time (as an exception).
|
||||
|
||||
```python
|
||||
def add(x, y):
|
||||
return x + y
|
||||
|
||||
>>> add(3, '4')
|
||||
Traceback (most recent call last):
|
||||
...
|
||||
TypeError: unsupported operand type(s) for +:
|
||||
'int' and 'str'
|
||||
>>>
|
||||
```
|
||||
|
||||
To verify code, there is a strong emphasis on testing (covered later).
|
||||
|
||||
### Exceptions
|
||||
|
||||
Exceptions are used to signal errors.
|
||||
To raise an exception yourself, use `raise` statement.
|
||||
|
||||
```python
|
||||
if name not in names:
|
||||
raise RuntimeError('Name not found')
|
||||
```
|
||||
|
||||
To catch an exception use `try-except`.
|
||||
|
||||
```python
|
||||
try:
|
||||
authenticate(username)
|
||||
except RuntimeError as e:
|
||||
print(e)
|
||||
```
|
||||
|
||||
### Exception Handling
|
||||
|
||||
Exceptions propagate to the first matching `except`.
|
||||
|
||||
```python
|
||||
def grok():
|
||||
...
|
||||
raise RuntimeError('Whoa!') # Exception raised here
|
||||
|
||||
def spam():
|
||||
grok() # Call that will raise exception
|
||||
|
||||
def bar():
|
||||
try:
|
||||
spam()
|
||||
except RuntimeError as e: # Exception caught here
|
||||
...
|
||||
|
||||
def foo():
|
||||
try:
|
||||
bar()
|
||||
except RuntimeError as e: # Exception does NOT arrive here
|
||||
...
|
||||
|
||||
foo()
|
||||
```
|
||||
|
||||
To handle the exception, use the `except` block. You can add any statements you want to handle the error.
|
||||
|
||||
```python
|
||||
def grok(): ...
|
||||
raise RuntimeError('Whoa!')
|
||||
|
||||
def bar():
|
||||
try:
|
||||
grok()
|
||||
except RuntimeError as e: # Exception caught here
|
||||
statements # Use this statements
|
||||
statements
|
||||
...
|
||||
|
||||
bar()
|
||||
```
|
||||
|
||||
After handling, execution resumes with the first statement after the `try-except`.
|
||||
|
||||
```python
|
||||
def grok(): ...
|
||||
raise RuntimeError('Whoa!')
|
||||
|
||||
def bar():
|
||||
try:
|
||||
grok()
|
||||
except RuntimeError as e: # Exception caught here
|
||||
statements
|
||||
statements
|
||||
...
|
||||
statements # Resumes execution here
|
||||
statements # And continues here
|
||||
...
|
||||
|
||||
bar()
|
||||
```
|
||||
|
||||
### Built-in Exceptions
|
||||
|
||||
There are about two-dozen built-in exceptions.
|
||||
This is not an exhaustive list. Check the documentation for more.
|
||||
|
||||
```python
|
||||
ArithmeticError
|
||||
AssertionError
|
||||
EnvironmentError
|
||||
EOFError
|
||||
ImportError
|
||||
IndexError
|
||||
KeyboardInterrupt
|
||||
KeyError
|
||||
MemoryError
|
||||
NameError
|
||||
ReferenceError
|
||||
RuntimeError
|
||||
SyntaxError
|
||||
SystemError
|
||||
TypeError
|
||||
ValueError
|
||||
```
|
||||
|
||||
### Exception Values
|
||||
|
||||
Most exceptions have an associated value. It contains more information about what's wrong.
|
||||
|
||||
```python
|
||||
raise RuntimeError('Invalid user name')
|
||||
```
|
||||
|
||||
This value is passed to the variable supplied in `except`.
|
||||
|
||||
```python
|
||||
try:
|
||||
...
|
||||
except RuntimeError as e: # `e` holds the value raised
|
||||
...
|
||||
```
|
||||
|
||||
The value is an instance of the exception type. However, it often looks like a string when
|
||||
printed.
|
||||
|
||||
```python
|
||||
except RuntimeError as e:
|
||||
print('Failed : Reason', e)
|
||||
```
|
||||
|
||||
### Catching Multiple Errors
|
||||
|
||||
You can catch different kinds of exceptions with multiple `except` blocks.
|
||||
|
||||
```python
|
||||
try:
|
||||
...
|
||||
except LookupError as e:
|
||||
...
|
||||
except RuntimeError as e:
|
||||
...
|
||||
except IOError as e:
|
||||
...
|
||||
except KeyboardInterrupt as e:
|
||||
...
|
||||
```
|
||||
|
||||
Alternatively, if the block to handle them is the same, you can group them:
|
||||
|
||||
```python
|
||||
try:
|
||||
...
|
||||
except (IOError,LookupError,RuntimeError) as e:
|
||||
...
|
||||
```
|
||||
|
||||
### Catching All Errors
|
||||
|
||||
To catch any exception, use `Exception` like this:
|
||||
|
||||
```python
|
||||
try:
|
||||
...
|
||||
except Exception:
|
||||
print('An error occurred')
|
||||
```
|
||||
|
||||
In general, writing code like that is a bad idea because you'll have no idea
|
||||
why it failed.
|
||||
|
||||
### Wrong Way to Catch Errors
|
||||
|
||||
Here is the wrong way to use exceptions.
|
||||
|
||||
```python
|
||||
try:
|
||||
go_do_something()
|
||||
except Exception:
|
||||
print('Computer says no')
|
||||
```
|
||||
|
||||
This swallows all possible errors. It may make it impossible to debug
|
||||
when the code is failing for some reason you didn't expect at all
|
||||
(e.g. uninstalled Python module, etc.).
|
||||
|
||||
### Somewhat Better Approach
|
||||
|
||||
This is a more sane approach.
|
||||
|
||||
```python
|
||||
try:
|
||||
go_do_something()
|
||||
except Exception as e:
|
||||
print('Computer says no. Reason :', e)
|
||||
```
|
||||
|
||||
It reports a specific reason for failure. It is almost always a good
|
||||
idea to have some mechanism for viewing/reporting errors when you
|
||||
write code that catches all possible exceptions.
|
||||
|
||||
In general though, it's better to catch the error more narrowly. Only
|
||||
catch the errors you can actually deal with. Let other errors pass to
|
||||
other code.
|
||||
|
||||
### Reraising an Exception
|
||||
|
||||
Use `raise` to propagate a caught error.
|
||||
|
||||
```python
|
||||
try:
|
||||
go_do_something()
|
||||
except Exception as e:
|
||||
print('Computer says no. Reason :', e)
|
||||
raise
|
||||
```
|
||||
|
||||
It allows you to take action (e.g. logging) and pass the error on to the caller.
|
||||
|
||||
### Exception Best Practices
|
||||
|
||||
Don't catch exceptions. Fail fast and loud. If it's important, someone
|
||||
else will take care of the problem. Only catch an exception if you
|
||||
are *that* someone. That is, only catch errors where you can recover
|
||||
and sanely keep going.
|
||||
|
||||
### `finally` statement
|
||||
|
||||
It specifies code that must fun regardless of whether or not an exception occurs.
|
||||
|
||||
```python
|
||||
lock = Lock()
|
||||
...
|
||||
lock.acquire()
|
||||
try:
|
||||
...
|
||||
finally:
|
||||
lock.release() # this will ALWAYS be executed. With and without exception.
|
||||
```
|
||||
|
||||
Comonly used to properly manage resources (especially locks, files, etc.).
|
||||
|
||||
### `with` statement
|
||||
|
||||
In modern code, `try-finally` often replaced with the `with` statement.
|
||||
|
||||
```python
|
||||
lock = Lock()
|
||||
with lock:
|
||||
# lock acquired
|
||||
...
|
||||
# lock released
|
||||
```
|
||||
|
||||
A more familiar example:
|
||||
|
||||
```python
|
||||
with open(filename) as f:
|
||||
# Use the file
|
||||
...
|
||||
# File closed
|
||||
```
|
||||
|
||||
It defines a usage *context* for a resource. When execution leaves that context,
|
||||
resources are released. `with` only works with certain objects.
|
||||
|
||||
## Exercises
|
||||
|
||||
### (a) Raising exceptions
|
||||
|
||||
The `parse_csv()` function you wrote in the last section allows
|
||||
user-specified columns to be selected, but that only works if the
|
||||
input data file has column headers.
|
||||
|
||||
Modify the code so that an exception gets raised if both the `select`
|
||||
and `has_headers=False` arguments are passed.
|
||||
For example:
|
||||
|
||||
```python
|
||||
>>> parse_csv('Data/prices.csv', select=['name','price'], has_headers=False)
|
||||
Traceback (most recent call last):
|
||||
File "<stdin>", line 1, in <module>
|
||||
File "fileparse.py", line 9, in parse_csv
|
||||
raise RuntimeError("select argument requires column headers")
|
||||
RuntimeError: select argument requires column headers
|
||||
>>>
|
||||
```
|
||||
|
||||
Having added this one check, you might ask if you should be performing
|
||||
other kinds of sanity checks in the function. For example, should you
|
||||
check that the filename is a string, that types is a list, or anything
|
||||
of that nature?
|
||||
|
||||
As a general rule, it’s usually best to skip such tests and to just
|
||||
let the program fail on bad inputs. The traceback message will point
|
||||
at the source of the problem and can assist in debugging.
|
||||
|
||||
The main reason for adding the above check to avoid running the code
|
||||
in a non-sensical mode (e.g., using a feature that requires column
|
||||
headers, but simultaneously specifying that there are no headers).
|
||||
|
||||
This indicates a programming error on the part of the calling code.
|
||||
|
||||
### (b) Catching exceptions
|
||||
|
||||
The `parse_csv()` function you wrote is used to process the entire
|
||||
contents of a file. However, in the real-world, it’s possible that
|
||||
input files might have corrupted, missing, or dirty data. Try this
|
||||
experiment:
|
||||
|
||||
```python
|
||||
>>> portfolio = parse_csv('Data/missing.csv', types=[str, int, float])
|
||||
Traceback (most recent call last):
|
||||
File "<stdin>", line 1, in <module>
|
||||
File "fileparse.py", line 36, in parse_csv
|
||||
row = [func(val) for func, val in zip(types, row)]
|
||||
ValueError: invalid literal for int() with base 10: ''
|
||||
>>>
|
||||
```
|
||||
|
||||
Modify the `parse_csv()` function to catch all `ValueError` exceptions
|
||||
generated during record creation and print a warning message for rows
|
||||
that can’t be converted.
|
||||
|
||||
The message should include the row number and information about the reason why it failed.
|
||||
To test your function, try reading the file `Data/missing.csv` above.
|
||||
For example:
|
||||
|
||||
```python
|
||||
>>> portfolio = parse_csv('Data/missing.csv', types=[str, int, float])
|
||||
Row 4: Couldn't convert ['MSFT', '', '51.23']
|
||||
Row 4: Reason invalid literal for int() with base 10: ''
|
||||
Row 7: Couldn't convert ['IBM', '', '70.44']
|
||||
Row 7: Reason invalid literal for int() with base 10: ''
|
||||
>>>
|
||||
>>> portfolio
|
||||
[{'price': 32.2, 'name': 'AA', 'shares': 100}, {'price': 91.1, 'name': 'IBM', 'shares': 50}, {'price': 83.44, 'name': 'CAT', 'shares': 150}, {'price': 40.37, 'name': 'GE', 'shares': 95}, {'price': 65.1, 'name': 'MSFT', 'shares': 50}]
|
||||
>>>
|
||||
```
|
||||
|
||||
### (c) Silencing Errors
|
||||
|
||||
Modify the `parse_csv()` function so that parsing error messages can be silenced if explicitly desired by the user.
|
||||
For example:
|
||||
|
||||
```python
|
||||
>>> portfolio = parse_csv('Data/missing.csv', types=[str,int,float], silence_errors=True)
|
||||
>>> portfolio
|
||||
[{'price': 32.2, 'name': 'AA', 'shares': 100}, {'price': 91.1, 'name': 'IBM', 'shares': 50}, {'price': 83.44, 'name': 'CAT', 'shares': 150}, {'price': 40.37, 'name': 'GE', 'shares': 95}, {'price': 65.1, 'name': 'MSFT', 'shares': 50}]
|
||||
>>>
|
||||
```
|
||||
|
||||
Error handling is one of the most difficult things to get right in
|
||||
most programs. As a general rule, you shouldn’t silently ignore
|
||||
errors. Instead, it’s better to report problems and to give the user
|
||||
an option to the silence the error message if they choose to do so.
|
||||
|
||||
[Next](04_Modules)
|
||||
Reference in New Issue
Block a user