zev/practical-python
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Editing

Browse Source
This commit is contained in:
David Beazley
2020-05-28 10:22:32 -05:00
parent a5cae9cdc2
commit 7a1cccb847
6 changed files with 268 additions and 170 deletions
Download Patch File Download Diff File Expand all files Collapse all files

77
Notes/03_Program_organization/01_Script.md
View File

@@ -1,3 +1,5 @@
[Contents](../Contents) \| [Previous (2.7 Object Model)](../02_Working_with_data/07_Objects) \| [Next (3.2 More on Functions)](02_More_functions)
# 3.1 Scripting
In this part we look more closely at the practice of writing Python
@@ -16,7 +18,7 @@ statement3
...
```
We have been writing scripts to this point.
We have mostly been writing scripts to this point.
### A Problem
@@ -28,7 +30,7 @@ organized.
### Defining Things
You must always define things before they get used later on in a program.
Names must always be defined before they get used later.
```python
def square(x):
@@ -41,11 +43,12 @@ z = square(b) # Requires `square` and `b` to be defined
```
**The order is important.**
You almost always put the definitions of variables an functions near the beginning.
You almost always put the definitions of variables and functions near the top.
### Defining Functions
It is a good idea to put all of the code related to a single *task* all in one place.
Use a function.
```python
def read_prices(filename):
@@ -85,7 +88,7 @@ def foo():
help(math)
```
There are no *special* statements in Python.
There are no *special* statements in Python (which makes it easy to remember).
### Function Definition
@@ -106,13 +109,14 @@ def foo(x):
bar(x)
```
Functions must only be defined before they are actually *used* (or called) during program execution.
Functions must only be defined prior to actually being *used* (or called) during program execution.
```python
foo(3) # foo must be defined already
```
Stylistically, it is probably more common to see functions defined in a *bottom-up* fashion.
Stylistically, it is probably more common to see functions defined in
a *bottom-up* fashion.
### Bottom-up Style
@@ -137,24 +141,26 @@ def spam(x):
spam(42) # Code that uses the functions appears at the end
```
Later functions build upon earlier functions.
Later functions build upon earlier functions. Again, this is only
a point of style. The only thing that matters in the above program
is that the call to `spam(42)` go last.
### Function Design
Ideally, functions should be a *black box*.
They should only operate on passed inputs and avoid global variables
and mysterious side-effects. Main goals: *Modularity* and *Predictability*.
and mysterious side-effects. Your main goals: *Modularity* and *Predictability*.
### Doc Strings
A good practice is to include documentations in the form of
doc-strings. Doc-strings are strings written immediately after the
It's good practice to include documentation in the form of a
doc-string. Doc-strings are strings written immediately after the
name of the function. They feed `help()`, IDEs and other tools.
```python
def read_prices(filename):
'''
Read prices from a CSV file of name,price
Read prices from a CSV file of name,price data
'''
prices = {}
with open(filename) as f:
@@ -164,14 +170,19 @@ def read_prices(filename):
return prices
```
A good practice for doc strings is to write a short one sentence
summary of what the function does. If more information is needed,
include a short example of usage along with a more detailed
description of the arguments.
### Type Annotations
You can also add some optional type annotations to your function definitions.
You can also add optional type hints to function definitions.
```python
def read_prices(filename: str) -> dict:
'''
Read prices from a CSV file of name,price
Read prices from a CSV file of name,price data
'''
prices = {}
with open(filename) as f:
@@ -181,13 +192,15 @@ def read_prices(filename: str) -> dict:
return prices
```
These do nothing. It is purely informational.
They may be used by IDEs, code checkers, etc.
The hints do nothing operationally. They are purely informational.
However, they may be used by IDEs, code checkers, and other tools
to do more.
## Exercises
In section 2, you wrote a program called `report.py` that printed out a report showing the performance of a stock portfolio.
This program consisted of some functions. For example:
In section 2, you wrote a program called `report.py` that printed out
a report showing the performance of a stock portfolio. This program
consisted of some functions. For example:
```python
# report.py
@@ -215,8 +228,9 @@ def read_portfolio(filename):
...
```
However, there were also portions of the program that just performed a series of scripted calculations.
This code appeared near the end of the program. For example:
However, there were also portions of the program that just performed a
series of scripted calculations. This code appeared near the end of
the program. For example:
```python
...
@@ -231,7 +245,8 @@ for row in report:
...
```
In this exercise, we’re going take this program and organize it a little more strongly around the use of functions.
In this exercise, we’re going take this program and organize it a
little more strongly around the use of functions.
### Exercise 3.1: Structuring a program as a collection of functions
@@ -242,10 +257,12 @@ functions. Specifically:
* Create a function `print_report(report)` that prints out the report.
* Change the last part of the program so that it is nothing more than a series of function calls and no other computation.
### Exercise 3.2: Creating a function for program execution
### Exercise 3.2: Creating a top-level function for program execution
Take the last part of your program and package it into a single function `portfolio_report(portfolio_filename, prices_filename)`.
Have the function work so that the following function call creates the report as before:
Take the last part of your program and package it into a single
function `portfolio_report(portfolio_filename, prices_filename)`.
Have the function work so that the following function call creates the
report as before:
```python
portfolio_report('Data/portfolio.csv', 'Data/prices.csv')
@@ -256,8 +273,9 @@ of function definitions followed by a single function call to
`portfolio_report()` at the very end (which executes all of the steps
involved in the program).
By turning your program into a single function, it becomes easy to run it on different inputs.
For example, try these statements interactively after running your program:
By turning your program into a single function, it becomes easy to run
it on different inputs. For example, try these statements
interactively after running your program:
```python
>>> portfolio_report('Data/portfolio2.csv', 'Data/prices.csv')
@@ -272,4 +290,13 @@ For example, try these statements interactively after running your program:
>>>
```
### Commentary
Python makes it very easy to write relatively unstructured scripting code
where you just have a file with a sequence of statements in it. In the
big picture, it's almost always better to utilize functions whenever
you can. At some point, that script is going to grow and you'll wish
you had a bit more organization. Also, a little known fact is that Python
runs a bit faster if you use functions.
[Contents](../Contents) \| [Previous (2.7 Object Model)](../02_Working_with_data/07_Objects) \| [Next (3.2 More on Functions)](02_More_functions)

135
Notes/03_Program_organization/02_More_functions.md
View File

@@ -1,6 +1,10 @@
[Contents](../Contents) \| [Previous (3.1 Scripting)](01_Script) \| [Next (3.3 Error Checking)](03_Error_checking)
# 3.2 More on Functions
This section fills in a few more details about how functions work and are defined.
Although functions were introduced earlier, very few details were provided on how
they actually work at a deeper level. This section aims to fill in some gaps
and discuss matters such as calling conventions, scoping rules, and more.
### Calling a Function
@@ -25,7 +29,8 @@ prices = read_prices(filename='prices.csv', debug=True)
### Default Arguments
Sometimes you want an optional argument.
Sometimes you want an argument to be optional. If so, assign a default value
in the function definition.
```python
def read_prices(filename, debug=False):
@@ -53,7 +58,8 @@ parse_data(data, debug=True)
parse_data(data, debug=True, ignore_errors=True)
```
Keyword arguments improve code clarity.
In most cases, keyword arguments improve code clarity--especially for arguments that
serve as flags or which are related to optional features.
### Design Best Practices
@@ -67,7 +73,7 @@ d = read_prices('prices.csv', debug=True)
Python development tools will show the names in help features and documentation.
### Return Values
### Returning Values
The `return` statement returns a value
@@ -76,7 +82,7 @@ def square(x):
return x * x
```
If no return value or `return` not specified, `None` is returned.
If no return value is given or `return` is missing, `None` is returned.
```python
def bar(x):
@@ -94,8 +100,8 @@ b = foo(4) # b = None
### Multiple Return Values
Functions can only return one value.
However, a function may return multiple values by returning a tuple.
Functions can only return one value. However, a function may return
multiple values by returning them in a tuple.
```python
def divide(a,b):
@@ -124,18 +130,19 @@ def foo():
```
Variables assignments occur outside and inside function definitions.
Variables defined outside are "global". Variables inside a function are "local".
Variables defined outside are "global". Variables inside a function
are "local".
### Local Variables
Variables inside functions are private.
Variables assigned inside functions are private.
```python
def read_portfolio(filename):
portfolio = []
for line in open(filename):
fields = line.split()
s = (fields[0],int(fields[1]),float(fields[2]))
fields = line.split(',')
s = (fields[0], int(fields[1]), float(fields[2]))
portfolio.append(s)
return portfolio
```
@@ -143,8 +150,8 @@ def read_portfolio(filename):
In this example, `filename`, `portfolio`, `line`, `fields` and `s` are local variables.
Those variables are not retained or accessible after the function call.
```pycon
>>> stocks = read_portfolio('stocks.dat')
```python
>>> stocks = read_portfolio('portfolio.csv')
>>> fields
Traceback (most recent call last):
File "<stdin>", line 1, in ?
@@ -152,11 +159,12 @@ NameError: name 'fields' is not defined
>>>
```
They also can't conflict with variables found elsewhere.
Locals also can't conflict with variables found elsewhere.
### Global Variables
Functions can freely access the values of globals.
Functions can freely access the values of globals defined in the same
file.
```python
name = 'Dave'
@@ -185,20 +193,24 @@ If you must modify a global variable you must declare it as such.
```python
name = 'Dave'
def spam():
global name
name = 'Guido' # Changes the global name above
```
The global declaration must appear before its use. Having seen this,
know that it is considered poor form. In fact, try to avoid entirely
The global declaration must appear before its use and the corresponding
variable must exist in the same file as the function. Having seen this,
know that it is considered poor form. In fact, try to avoid `global` entirely
if you can. If you need a function to modify some kind of state outside
of the function, it's better to use a class instead (more on this later).
### Argument Passing
When you call a function, the argument variables are names for passed values.
If mutable data types are passed (e.g. lists, dicts), they can be modified *in-place*.
When you call a function, the argument variables are names that refer
to the passed values. These values are NOT copies (see [section
2.7](../02_Working_with_data/07_Objects)). If mutable data types are
passed (e.g. lists, dicts), they can be modified *in-place*.
```python
def foo(items):
@@ -213,7 +225,8 @@ print(a) # [1, 2, 3, 42]
### Reassignment vs Modifying
Make sure you understand the subtle difference between modifying a value and reassigning a variable name.
Make sure you understand the subtle difference between modifying a
value and reassigning a variable name.
```python
def foo(items):
@@ -225,19 +238,22 @@ print(a) # [1, 2, 3, 42]
# VS
def bar(items):
items = [4,5,6] # Reassigns `items` variable
items = [4,5,6] # Changes local `items` variable to point to a different object
b = [1, 2, 3]
bar(b)
print(b) # [1, 2, 3]
```
*Reminder: Variable assignment never overwrites memory. The name is simply bound to a new value.*
*Reminder: Variable assignment never overwrites memory. The name is merely bound to a new value.*
## Exercises
This exercise involves a lot of steps and putting concepts together from past exercises.
The final solution is only about 25 lines of code, but take your time and make sure you understand each part.
This set of exercises have you implement what is, perhaps, the most
powerful and difficult part of the course. There are a lot of steps
and many concepts from past exercises are put together all at once.
The final solution is only about 25 lines of code, but take your time
and make sure you understand each part.
A central part of your `report.py` program focuses on the reading of
CSV files. For example, the function `read_portfolio()` reads a file
@@ -251,12 +267,13 @@ If you were doing a lot of file parsing for real, you’d probably want
to clean some of this up and make it more general purpose. That's
our goal.
Start this exercise by creating a new file called `fileparse.py`. This is where we will be doing our work.
Start this exercise by creating a new file called
`Work/fileparse.py`. This is where we will be doing our work.
### Exercise 3.3: Reading CSV Files
To start, let’s just focus on the problem of reading a CSV file into a
list of dictionaries. In the file `fileparse.py`, define a simple
list of dictionaries. In the file `fileparse.py`, define a
function that looks like this:
```python
@@ -290,20 +307,23 @@ Try it out:
Hint: `python3 -i fileparse.py`.
```pycon
```python
>>> portfolio = parse_csv('Data/portfolio.csv')
>>> portfolio
[{'price': '32.20', 'name': 'AA', 'shares': '100'}, {'price': '91.10', 'name': 'IBM', 'shares': '50'}, {'price': '83.44', 'name': 'CAT', 'shares': '150'}, {'price': '51.23', 'name': 'MSFT', 'shares': '200'}, {'price': '40.37', 'name': 'GE', 'shares': '95'}, {'price': '65.10', 'name': 'MSFT', 'shares': '50'}, {'price': '70.44', 'name': 'IBM', 'shares': '100'}]
>>>
```
This is great except that you can’t do any kind of useful calculation with the data because everything is represented as a string.
We’ll fix this shortly, but let’s keep building on it.
This is good except that you can’t do any kind of useful calculation
with the data because everything is represented as a string. We’ll
fix this shortly, but let’s keep building on it.
### Exercise 3.4: Building a Column Selector
In many cases, you’re only interested in selected columns from a CSV file, not all of the data.
Modify the `parse_csv()` function so that it optionally allows user-specified columns to be picked out as follows:
In many cases, you’re only interested in selected columns from a CSV
file, not all of the data. Modify the `parse_csv()` function so that
it optionally allows user-specified columns to be picked out as
follows:
```python
>>> # Read all of the data
@@ -311,7 +331,7 @@ Modify the `parse_csv()` function so that it optionally allows user-specified co
>>> portfolio
[{'price': '32.20', 'name': 'AA', 'shares': '100'}, {'price': '91.10', 'name': 'IBM', 'shares': '50'}, {'price': '83.44', 'name': 'CAT', 'shares': '150'}, {'price': '51.23', 'name': 'MSFT', 'shares': '200'}, {'price': '40.37', 'name': 'GE', 'shares': '95'}, {'price': '65.10', 'name': 'MSFT', 'shares': '50'}, {'price': '70.44', 'name': 'IBM', 'shares': '100'}]
>>> # Read some of the data
>>> # Read only some of the data
>>> shares_held = parse_csv('portfolio.csv', select=['name','shares'])
>>> shares_held
[{'name': 'AA', 'shares': '100'}, {'name': 'IBM', 'shares': '50'}, {'name': 'CAT', 'shares': '150'}, {'name': 'MSFT', 'shares': '200'}, {'name': 'GE', 'shares': '95'}, {'name': 'MSFT', 'shares': '50'}, {'name': 'IBM', 'shares': '100'}]
@@ -358,17 +378,18 @@ def parse_csv(filename, select=None):
return records
```
There are a number of tricky bits to this part. Probably the most important one is the mapping of the column selections to row indices.
There are a number of tricky bits to this part. Probably the most
important one is the mapping of the column selections to row indices.
For example, suppose the input file had the following headers:
```pycon
```python
>>> headers = ['name', 'date', 'time', 'shares', 'price']
>>>
```
Now, suppose the selected columns were as follows:
```pycon
```python
>>> select = ['name', 'shares']
>>>
```
@@ -376,7 +397,7 @@ Now, suppose the selected columns were as follows:
To perform the proper selection, you have to map the selected column names to column indices in the file.
That’s what this step is doing:
```pycon
```python
>>> indices = [headers.index(colname) for colname in select ]
>>> indices
[0, 3]
@@ -386,7 +407,7 @@ That’s what this step is doing:
In other words, "name" is column 0 and "shares" is column 3.
When you read a row of data from the file, the indices are used to filter it:
```pycon
```python
>>> row = ['AA', '6/11/2007', '9:50am', '100', '32.20' ]
>>> row = [ row[index] for index in indices ]
>>> row
@@ -396,10 +417,10 @@ When you read a row of data from the file, the indices are used to filter it:
### Exercise 3.5: Performing Type Conversion
Modify the `parse_csv()` function so that it optionally allows type-conversions to be applied to the returned data.
For example:
Modify the `parse_csv()` function so that it optionally allows
type-conversions to be applied to the returned data. For example:
```pycon
```python
>>> portfolio = parse_csv('Data/portfolio.csv', types=[str, int, float])
>>> portfolio
[{'price': 32.2, 'name': 'AA', 'shares': 100}, {'price': 91.1, 'name': 'IBM', 'shares': 50}, {'price': 83.44, 'name': 'CAT', 'shares': 150}, {'price': 51.23, 'name': 'MSFT', 'shares': 200}, {'price': 40.37, 'name': 'GE', 'shares': 95}, {'price': 65.1, 'name': 'MSFT', 'shares': 50}, {'price': 70.44, 'name': 'IBM', 'shares': 100}]
@@ -410,8 +431,8 @@ For example:
>>>
```
You already explored this in [Exercise 2.24](../02_Working_with_data/07_Objects). You'll need to insert the
following fragment of code into your solution:
You already explored this in [Exercise 2.24](../02_Working_with_data/07_Objects).
You'll need to insert the following fragment of code into your solution:
```python
...
@@ -420,7 +441,7 @@ if types:
...
```
### Exercise 3.6: Working with Headers
### Exercise 3.6: Working without Headers
Some CSV files don’t include any header information.
For example, the file `prices.csv` looks like this:
@@ -433,8 +454,8 @@ For example, the file `prices.csv` looks like this:
...
```
Modify the `parse_csv()` function so that it can work with such files by creating a list of tuples instead.
For example:
Modify the `parse_csv()` function so that it can work with such files
by creating a list of tuples instead. For example:
```python
>>> prices = parse_csv('Data/prices.csv', types=[str,float], has_headers=False)
@@ -450,8 +471,10 @@ column names to use for keys.
### Exercise 3.7: Picking a different column delimitier
Although CSV files are pretty common, it’s also possible that you could encounter a file that uses a different column separator such as a tab or space.
For example, the file `Data/portfolio.dat` looks like this:
Although CSV files are pretty common, it’s also possible that you
could encounter a file that uses a different column separator such as
a tab or space. For example, the file `Data/portfolio.dat` looks like
this:
```csv
name shares price
@@ -464,26 +487,30 @@ name shares price
"IBM" 100 70.44
```
The `csv.reader()` function allows a different delimiter to be given as follows:
The `csv.reader()` function allows a different column delimiter to be given as follows:
```python
rows = csv.reader(f, delimiter=' ')
```
Modify your `parse_csv()` function so that it also allows the delimiter to be changed.
Modify your `parse_csv()` function so that it also allows the
delimiter to be changed.
For example:
```pycon
```python
>>> portfolio = parse_csv('Data/portfolio.dat', types=[str, int, float], delimiter=' ')
>>> portfolio
[{'price': '32.20', 'name': 'AA', 'shares': '100'}, {'price': '91.10', 'name': 'IBM', 'shares': '50'}, {'price': '83.44', 'name': 'CAT', 'shares': '150'}, {'price': '51.23', 'name': 'MSFT', 'shares': '200'}, {'price': '40.37', 'name': 'GE', 'shares': '95'}, {'price': '65.10', 'name': 'MSFT', 'shares': '50'}, {'price': '70.44', 'name': 'IBM', 'shares': '100'}]
>>>
```
If you’ve made it this far, you’ve created a nice library function that’s genuinely useful.
You can use it to parse arbitrary CSV files, select out columns of
interest, perform type conversions, without having to worry too much
about the inner workings of files or the `csv` module.
### Commentary
If you’ve made it this far, you’ve created a nice library function
that’s genuinely useful. You can use it to parse arbitrary CSV files,
select out columns of interest, perform type conversions, without
having to worry too much about the inner workings of files or the
`csv` module.
[Contents](../Contents) \| [Previous (3.1 Scripting)](01_Script) \| [Next (3.3 Error Checking)](03_Error_checking)

86
Notes/03_Program_organization/03_Error_checking.md
View File

@@ -1,11 +1,15 @@
[Contents](../Contents) \| [Previous (3.2 More on Functions)](02_More_functions) \| [Next (3.4 Modules)](04_Modules)
# 3.3 Error Checking
This section discusses some aspects of error checking and exception handling.
Although exceptions were introduced earlier, this section fills in some additional
details about error checking and exception handling.
### How programs fail
Python performs no checking or validation of function argument types or values.
A function will work on any data that is compatible with the statements in the function.
Python performs no checking or validation of function argument types
or values. A function will work on any data that is compatible with
the statements in the function.
```python
def add(x, y):
@@ -16,7 +20,7 @@ add('Hello', 'World') # 'HelloWorld'
add('3', '4') # '34'
```
If there are errors in a function, they will show up at run time (as an exception).
If there are errors in a function, they appear at run time (as an exception).
```python
def add(x, y):
@@ -38,8 +42,8 @@ Exceptions are used to signal errors.
To raise an exception yourself, use `raise` statement.
```python
if name not in names:
raise RuntimeError('Name not found')
if name not in authorized:
raise RuntimeError(f'{name} not authorized')
```
To catch an exception use `try-except`.
@@ -78,7 +82,8 @@ def foo():
foo()
```
To handle the exception, use the `except` block. You can add any statements you want to handle the error.
To handle the exception, put statements in the `except` block. You can add any
statements you want to handle the error.
```python
def grok(): ...
@@ -95,7 +100,8 @@ def bar():
bar()
```
After handling, execution resumes with the first statement after the `try-except`.
After handling, execution resumes with the first statement after the
`try-except`.
```python
def grok(): ...
@@ -117,8 +123,10 @@ bar()
### Built-in Exceptions
There are about two-dozen built-in exceptions.
This is not an exhaustive list. Check the documentation for more.
There are about two-dozen built-in exceptions. Usually the name of
the exception is indicative of what's wrong (e.g., a `ValueError` is
raised because you supplied a bad value). This is not an
exhaustive list. Check the documentation for more.
```python
ArithmeticError
@@ -141,22 +149,23 @@ ValueError
### Exception Values
Most exceptions have an associated value. It contains more information about what's wrong.
Exceptions have an associated value. It contains more specific
information about what's wrong.
```python
raise RuntimeError('Invalid user name')
```
This value is passed to the variable supplied in `except`.
This value is part of the exception instance that's placed in the variable supplied to `except`.
```python
try:
...
except RuntimeError as e: # `e` holds the value raised
except RuntimeError as e: # `e` holds the exception raised
...
```
The value is an instance of the exception type. However, it often looks like a string when
`e` is an instance of the exception type. However, it often looks like a string when
printed.
```python
@@ -166,7 +175,7 @@ except RuntimeError as e:
### Catching Multiple Errors
You can catch different kinds of exceptions with multiple `except` blocks.
You can catch different kinds of exceptions using multiple `except` blocks.
```python
try:
@@ -181,7 +190,7 @@ except KeyboardInterrupt as e:
...
```
Alternatively, if the block to handle them is the same, you can group them:
Alternatively, if the statements to handle them is the same, you can group them:
```python
try:
@@ -197,12 +206,12 @@ To catch any exception, use `Exception` like this:
```python
try:
...
except Exception:
except Exception: # DANGER. See below
print('An error occurred')
```
In general, writing code like that is a bad idea because you'll have no idea
why it failed.
In general, writing code like that is a bad idea because you'll have
no idea why it failed.
### Wrong Way to Catch Errors
@@ -215,13 +224,13 @@ except Exception:
print('Computer says no')
```
This swallows all possible errors. It may make it impossible to debug
This catches all possible errors and it may make it impossible to debug
when the code is failing for some reason you didn't expect at all
(e.g. uninstalled Python module, etc.).
### Somewhat Better Approach
This is a more sane approach.
If you're going to catch all errors, this is a more sane approach.
```python
try:
@@ -234,9 +243,9 @@ It reports a specific reason for failure. It is almost always a good
idea to have some mechanism for viewing/reporting errors when you
write code that catches all possible exceptions.
In general though, it's better to catch the error more narrowly. Only
catch the errors you can actually deal with. Let other errors pass to
other code.
In general though, it's better to catch the error as narrowly as is
reasonable. Only catch the errors you can actually handle. Let
other errors pass by--maybe some other code can handle them.
### Reraising an Exception
@@ -250,7 +259,8 @@ except Exception as e:
raise
```
It allows you to take action (e.g. logging) and pass the error on to the caller.
This allows you to take action (e.g. logging) and pass the error on to
the caller.
### Exception Best Practices
@@ -261,7 +271,8 @@ and sanely keep going.
### `finally` statement
It specifies code that must fun regardless of whether or not an exception occurs.
It specifies code that must run regardless of whether or not an
exception occurs.
```python
lock = Lock()
@@ -273,11 +284,11 @@ finally:
lock.release() # this will ALWAYS be executed. With and without exception.
```
Comonly used to properly manage resources (especially locks, files, etc.).
Commonly used to safely manage resources (especially locks, files, etc.).
### `with` statement
In modern code, `try-finally` often replaced with the `with` statement.
In modern code, `try-finally` is often replaced with the `with` statement.
```python
lock = Lock()
@@ -296,8 +307,9 @@ with open(filename) as f:
# File closed
```
It defines a usage *context* for a resource. When execution leaves that context,
resources are released. `with` only works with certain objects.
`with` defines a usage *context* for a resource. When execution
leaves that context, resources are released. `with` only works with
certain objects that have been specifically programmed to support it.
## Exercises
@@ -308,8 +320,7 @@ user-specified columns to be selected, but that only works if the
input data file has column headers.
Modify the code so that an exception gets raised if both the `select`
and `has_headers=False` arguments are passed.
For example:
and `has_headers=False` arguments are passed. For example:
```python
>>> parse_csv('Data/prices.csv', select=['name','price'], has_headers=False)
@@ -335,6 +346,7 @@ in a non-sensical mode (e.g., using a feature that requires column
headers, but simultaneously specifying that there are no headers).
This indicates a programming error on the part of the calling code.
Checking for cases that "aren't supposed to happen" is often a good idea.
### Exercise 3.9: Catching exceptions
@@ -357,9 +369,9 @@ Modify the `parse_csv()` function to catch all `ValueError` exceptions
generated during record creation and print a warning message for rows
that can’t be converted.
The message should include the row number and information about the reason why it failed.
To test your function, try reading the file `Data/missing.csv` above.
For example:
The message should include the row number and information about the
reason why it failed. To test your function, try reading the file
`Data/missing.csv` above. For example:
```python
>>> portfolio = parse_csv('Data/missing.csv', types=[str, int, float])
@@ -375,8 +387,8 @@ Row 7: Reason invalid literal for int() with base 10: ''
### Exercise 3.10: Silencing Errors
Modify the `parse_csv()` function so that parsing error messages can be silenced if explicitly desired by the user.
For example:
Modify the `parse_csv()` function so that parsing error messages can
be silenced if explicitly desired by the user. For example:
```python
>>> portfolio = parse_csv('Data/missing.csv', types=[str,int,float], silence_errors=True)

66
Notes/03_Program_organization/04_Modules.md
View File

@@ -1,6 +1,9 @@
[Contents](../Contents) \| [Previous (3.3 Error Checking)](03_Error_checking) \| [Next (3.5 Main Module)](05_Main_module)
# 3.4 Modules
This section introduces the concept of modules.
This section introduces the concept of modules and working with functions that span
multiple files.
### Modules and import
@@ -27,9 +30,10 @@ b = foo.spam('Hello')
### Namespaces
A module is a collection of named values and is sometimes said to be a *namespace*.
The names are all of the global variables and functions defined in the source file.
After importing, the module name is used as a prefix. Hence the *namespace*.
A module is a collection of named values and is sometimes said to be a
*namespace*. The names are all of the global variables and functions
defined in the source file. After importing, the module name is used
as a prefix. Hence the *namespace*.
```python
import foo
@@ -39,12 +43,12 @@ b = foo.spam('Hello')
...
```
The module name is tied to the file name (foo -> foo.py).
The module name is directly tied to the file name (foo -> foo.py).
### Global Definitions
Everything defined in the *global* scope is what populates the module
namespace. `foo` in our previous example. Consider two modules
namespace. Consider two modules
that define the same variable `x`.
```python
@@ -118,14 +122,16 @@ def rectangular(r, theta):
return x, y
```
It allows parts of a module to be used without having to type the module prefix.
Useful for frequently used names.
This allows parts of a module to be used without having to type the module prefix.
It's useful for frequently used names.
### Comments on importing
Variations on import do *not* change the way that modules work.
```python
import math
# vs
import math as m
# vs
from math import cos, sin
@@ -135,7 +141,10 @@ from math import cos, sin
Specifically, `import` always executes the *entire* file and modules
are still isolated environments.
The `import module as` statement is only manipulating the names.
The `import module as` statement is only changing the name locally.
The `from math import cos, sin` statement still loads the entire
math module behind the scenes. It's merely copying the `cos` and `sin`
names from the module into the local space after it's done.
### Module Loading
@@ -151,6 +160,12 @@ Each module loads and executes only *once*.
>>>
```
**Caution:** A common confusion arises if you repeat an `import` statement after
changing the source code for a module. Because of the module cache `sys.modules`,
repeated imports always return the previously loaded module--even if a change
was made. The safest way to load modified code into Python is to quit and restart
the interpreter.
### Locating Modules
Python consults a path list (sys.path) when looking for modules.
@@ -166,12 +181,11 @@ Python consults a path list (sys.path) when looking for modules.
]
```
Current working directory is usually first.
The current working directory is usually first.
### Module Search Path
`sys.path` contains the search paths.
As noted, `sys.path` contains the search paths.
You can manually adjust if you need to.
```python
@@ -179,7 +193,7 @@ import sys
sys.path.append('/project/foo/pyfiles')
```
Paths are also added via environment variables.
Paths can also be added via environment variables.
```python
% env PYTHONPATH=/project/foo/pyfiles python3
@@ -190,16 +204,26 @@ Python 3.6.0 (default, Feb 3 2017, 05:53:21)
['','/project/foo/pyfiles', ...]
```
As a general rule, it should not be necessary to manually adjust
the module search path. However, it sometimes arises if you're
trying to import Python code that's in an unusual location or
not readily accessible from the current working directory.
## Exercises
For this exercise involving modules, it is critically important to
make sure you are running Python in a proper environment. Modules
are usually when programmers encounter problems with the current working
directory or with Python's path settings.
make sure you are running Python in a proper environment. Modules are
usually when programmers encounter problems with the current working
directory or with Python's path settings. For this course, it is
assumed that you're writing all of your code in the `Work/` directory.
For best results, you should make sure you're also in that directory
when you launch the interpreter. If not, you need to make sure
`practical-python/Work` is added to `sys.path`.
### Exercise 3.11: Module imports
In section 3, we created a general purpose function `parse_csv()` for parsing the contents of CSV datafiles.
In section 3, we created a general purpose function `parse_csv()` for
parsing the contents of CSV datafiles.
Now, we’re going to see how to use that function in other programs.
First, start in a new shell window. Navigate to the folder where you
@@ -217,7 +241,7 @@ Type "help", "copyright", "credits" or "license" for more information.
Once you’ve done that, try importing some of the programs you
previously wrote. You should see their output exactly as before.
Just emphasize, importing a module runs its code.
Just to emphasize, importing a module runs its code.
```python
>>> import bounce
@@ -272,9 +296,9 @@ Try importing a function so that you don’t need to include the module name:
In section 2, you wrote a program `report.py` that produced a stock report like this:
```shell
```
Name Shares Price Change
---------- ---------- ---------- ----------
---------- ---------- ---------- ----------
AA 100 39.91 7.71
IBM 50 106.11 15.01
CAT 150 78.58 -4.86
@@ -312,6 +336,6 @@ programs. `fileparse.py` which contains a general purpose
`parse_csv()` function. `report.py` which produces a nice report, but
also contains `read_portfolio()` and `read_prices()` functions. And
finally, `pcost.py` which computes the portfolio cost, but makes use
of the code written for the `report.py` program.
of the `read_portfolio()` function written for the `report.py` program.
[Contents](../Contents) \| [Previous (3.3 Error Checking)](03_Error_checking) \| [Next (3.5 Main Module)](05_Main_module)

29
Notes/03_Program_organization/05_Main_module.md
View File

@@ -1,3 +1,5 @@
[Contents](../Contents) \| [Previous (3.4 Modules)](04_Modules) \| [Next (3.6 Design Discussion)](06_Design_discussion)
# 3.5 Main Module
This section introduces the concept of a main program or main module.
@@ -22,7 +24,7 @@ class myprog {
}
```
This is the first function that is being executing when an application is launched.
This is the first function that executes when an application is launched.
### Python Main Module
@@ -34,11 +36,11 @@ bash % python3 prog.py
...
```
Whatever module you give to the interpreter at startup becomes *main*. It doesn't matter the name.
Whatever file you give to the interpreter at startup becomes *main*. It doesn't matter the name.
### `__main__` check
It is standard practice for modules that can run as a main script to use this convention:
It is standard practice for modules that run as a main script to use this convention:
```python
# prog.py
@@ -53,22 +55,22 @@ Statements inclosed inside the `if` statement become the *main* program.
### Main programs vs. library imports
Any file can either run as main or as a library import:
Any Python file can either run as main or as a library import:
```bash
bash % python3 prog.py # Running as main
```
```python
import prog
import prog # Running as library import
```
In both cases, `__name__` is the name of the module. However, it will only be set to `__main__` if
running as main.
As a general rule, you don't want statements that are part of the main
program to execute on a library import. So, it's common to have an `if-`check in code
that might be used either way.
Usually, you don't want statements that are part of the main program
to execute on a library import. So, it's common to have an `if-`check
in code that might be used either way.
```python
if __name__ == '__main__':
@@ -221,7 +223,8 @@ bash % prog.py
### Script Template
Here is a common code template for Python programs that run as command-line scripts:
Finally, here is a common code template for Python programs that run
as command-line scripts:
```python
#!/usr/bin/env python3
@@ -251,8 +254,9 @@ if __name__ == '__main__':
### Exercise 3.15: `main()` functions
In the file `report.py` add a `main()` function that accepts a list of command line options and produces the same output as before.
You should be able to run it interatively like this:
In the file `report.py` add a `main()` function that accepts a list of
command line options and produces the same output as before. You
should be able to run it interatively like this:
```python
>>> import report
@@ -280,7 +284,8 @@ Total cost: 44671.15
### Exercise 3.16: Making Scripts
Modify the `report.py` and `pcost.py` programs so that they can execute as a script on the command line:
Modify the `report.py` and `pcost.py` programs so that they can
execute as a script on the command line:
```bash
bash $ python3 report.py Data/portfolio.csv Data/prices.csv

27
Notes/03_Program_organization/06_Design_discussion.md
View File

@@ -1,6 +1,8 @@
[Contents](../Contents) \| [Previous (3.5 Main module)](05_Main_module) \| [Next (4 Classes)](../04_Classes_objects/00_Overview)
# 3.6 Design Discussion
In this section we consider some design decisions made in code so far.
In this section we reconsider a design decision made earlier.
### Filenames versus Iterables
@@ -35,15 +37,17 @@ with open('file.csv') as f:
* Which of these functions do you prefer? Why?
* Which of these functions is more flexible?
### Deep Idea: "Duck Typing"
[Duck Typing](https://en.wikipedia.org/wiki/Duck_typing) is a computer programming concept to determine whether an object can be used for a particular purpose. It is an application of the [duck test](https://en.wikipedia.org/wiki/Duck_test).
[Duck Typing](https://en.wikipedia.org/wiki/Duck_typing) is a computer
programming concept to determine whether an object can be used for a
particular purpose. It is an application of the [duck
test](https://en.wikipedia.org/wiki/Duck_test).
> If it looks like a duck, swims like a duck, and quacks like a duck, then it probably is a duck.
In our previous example that reads the lines, our `read_data` expects
any iterable object. Not just the lines of a file.
In the second version of `read_data()` above, the function expects any
iterable object. Not just the lines of a file.
```python
def read_data(lines):
@@ -76,7 +80,7 @@ data = read_data(lines)
There is considerable flexibility with this design.
*Question: Shall we embrace or fight this flexibility?*
*Question: Should we embrace or fight this flexibility?*
### Library Design Best Practices
@@ -87,10 +91,10 @@ Don't restrict your options. With great flexibility comes great power.
### Exercise 3.17: From filenames to file-like objects
In this section, you worked on a file `fileparse.py` that contained a
You've now created a file `fileparse.py` that contained a
function `parse_csv()`. The function worked like this:
```pycon
```python
>>> import fileparse
>>> portfolio = fileparse.parse_csv('Data/portfolio.csv', types=[str,int,float])
>>>
@@ -103,8 +107,8 @@ with any file-like/iterable object. For example:
```
>>> import fileparse
>>> import gzip
>>> with gzip.open('Data/portfolio.csv.gz', 'rt') as f:
... port = fileparse.parse_csv(f, types=[str,int,float])
>>> with gzip.open('Data/portfolio.csv.gz', 'rt') as file:
... port = fileparse.parse_csv(file, types=[str,int,float])
...
>>> lines = ['name,shares,price', 'AA,34.23,100', 'IBM,50,91.1', 'HPE,75,45.1']
>>> port = fileparse.parse_csv(lines, types=[str,int,float])
@@ -120,8 +124,7 @@ In this new code, what happens if you pass a filename as before?
>>>
```
With flexibility comes power and with power comes responsibility. Sometimes you'll
need to be careful.
Yes, you'll need to be careful. Could you add a safety check to avoid this?
### Exercise 3.18: Fixing existing functions