zev/practical-python
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

More editing

Browse Source
This commit is contained in:
David Beazley
2020-05-28 13:10:10 -05:00
parent cc157243fa
commit 9572f707b2
6 changed files with 161 additions and 113 deletions
Download Patch File Download Diff File Expand all files Collapse all files

114
Notes/05_Object_model/01_Dicts_revisited.md
View File

@@ -1,7 +1,9 @@
[Contents](../Contents) \| [Previous (4.4 Exceptions)](../04_Classes_objects/04_Defining_exceptions) \| [Next (5.2 Encapsulation)](02_Classes_encapsulation)
# 5.1 Dictionaries Revisited
The Python object system is largely based on an implementation based on dictionaries. This
section discusses that.
The Python object system is largely based on an implementation
involving dictionaries. This section discusses that.
### Dictionaries, Revisited
@@ -15,12 +17,14 @@ stock = {
}
```
Dictionaries are commonly used for simple data structures.
However, they are used for critical parts of the interpreter and may be the *most important type of data in Python*.
Dictionaries are commonly used for simple data structures. However,
they are used for critical parts of the interpreter and may be the
*most important type of data in Python*.
### Dicts and Modules
In a module, a dictionary holds all of the global variables and functions.
Within a module, a dictionary holds all of the global variables and
functions.
```python
# foo.py
@@ -33,7 +37,7 @@ def spam():
...
```
If we inspect `foo.__dict__` or `globals()`, you'll see the dictionary.
If you inspect `foo.__dict__` or `globals()`, you'll see the dictionary.
```python
{
@@ -45,8 +49,9 @@ If we inspect `foo.__dict__` or `globals()`, you'll see the dictionary.
### Dicts and Objects
User defined objects also use dictionaries for both instance data and classes.
In fact, the entire object system is mostly an extra layer that's put on top of dictionaries.
User defined objects also use dictionaries for both instance data and
classes. In fact, the entire object system is mostly an extra layer
that's put on top of dictionaries.
A dictionary holds the instance data, `__dict__`.
@@ -59,7 +64,7 @@ A dictionary holds the instance data, `__dict__`.
You populate this dict (and instance) when assigning to `self`.
```python
class Stock(object):
class Stock:
def __init__(self, name, shares, price):
self.name = name
self.shares = shares
@@ -83,14 +88,15 @@ s = Stock('GOOG',100,490.1) # {'name' : 'GOOG','shares' : 100, 'price': 490.
t = Stock('AAPL', 50, 123.45) # {'name' : 'AAPL','shares' : 50, 'price': 123.45 }
```
If you created 200 instances of some class, there are 100 dictionaries sitting around holding data.
If you created 100 instances of some class, there are 100 dictionaries
sitting around holding data.
### Class Members
A separate dictionary also holds the methods.
```python
class Stock(object):
class Stock:
def __init__(self, name, shares, price):
self.name = name
self.shares = shares
@@ -115,8 +121,8 @@ The dictionary is in `Stock.__dict__`.
### Instances and Classes
Instances and classes are linked together.
The `__class__` attribute refers back to the class.
Instances and classes are linked together. The `__class__` attribute
refers back to the class.
```python
>>> s = Stock('GOOG', 100, 490.1)
@@ -127,7 +133,9 @@ The `__class__` attribute refers back to the class.
>>>
```
The instance dictionary holds data unique to each instance, whereas the class dictionary holds data collectively shared by *all* instances.
The instance dictionary holds data unique to each instance, whereas
the class dictionary holds data collectively shared by *all*
instances.
### Attribute Access
@@ -207,26 +215,30 @@ This provides a link to parent classes.
### Reading Attributes with Inheritance
First, check in local `__dict__`. If not found, look in `__dict__` of class through `__class__`.
If not found in class, look in base classes through `__bases__`.
Logically, the process of finding an attribute is as follows. First,
check in local `__dict__`. If not found, look in `__dict__` of the
class. If not found in class, look in the base classes through
`__bases__`. However, there are some subtle aspects of this discussed next.
### Reading Attributes with Single Inheritance
In inheritance hierarchies, attributes are found by walking up the inheritance tree.
In inheritance hierarchies, attributes are found by walking up the
inheritance tree in order.
```python
class A(object): pass
class A: pass
class B(A): pass
class C(A): pass
class D(B): pass
class E(D): pass
```
With Single Inheritance, there ia single path to the top.
With single inheritance, there is single path to the top.
You stop with the first match.
### Method Resolution Order or MRO
Python precomputes an inheritance chain and stores it in the *MRO* attribute on the class.
You can view it.
```python
>>> E.__mro__
@@ -236,38 +248,39 @@ Python precomputes an inheritance chain and stores it in the *MRO* attribute on
>>>
```
This chain is called the **Method Resolutin Order**.
The find the attributes, Python walks the MRO. First match, wins.
This chain is called the **Method Resolutin Order**. The find an
attribute, Python walks the MRO in order. The first match wins.
### MRO in Multiple Inheritance
There is no single path to the top with multiple inheritance.
With multiple inheritance, there is no single path to the top.
Let's take a look at an example.
```python
class A(object): pass
class B(object): pass
class A: pass
class B: pass
class C(A, B): pass
class D(B): pass
class E(C, D): pass
```
What happens when we do?
What happens when you access at attribute?
```python
e = E()
e.attr
```
A similar search process is carried out, but what is the order? That's a problem.
A attribute search process is carried out, but what is the order? That's a problem.
Python uses *cooperative multiple inheritance*.
These are some rules about class ordering:
Python uses *cooperative multiple inheritance* which obeys some rules
about class ordering.
* Children before parents
* Parents go in order
* Children are always checked before parents
* Parents (if multiple) are always checked in the order listed.
The MRO is computed using those rules.
The MRO is computed by sorting all of the classes in a hierarchy
according to those rules.
```python
>>> E.__mro__
@@ -281,12 +294,18 @@ The MRO is computed using those rules.
>>>
```
### An Odd Code Reuse
The underlying algorithm is called the "C3 Linearization Algorithm."
The precise details aren't important as long as you remember that a
class hierarchy obeys the same ordering rules you might follow if your
house was on fire and you had to evacuate--children first, followed by
parents.
### An Odd Code Reuse (Involving Multiple Inheritance)
Consider two completely unrelated objects:
```python
class Dog(object):
class Dog:
def noise(self):
return 'Bark'
@@ -295,14 +314,14 @@ class Dog(object):
class LoudDog(Dog):
def noise(self):
# Code commonality with LoudBike
# Code commonality with LoudBike (below)
return super().noise().upper()
```
And
```python
class Bike(object):
class Bike:
def noise(self):
return 'On Your Left'
@@ -311,19 +330,20 @@ class Bike(object):
class LoudBike(Bike):
def noise(self):
# Code commonality with LoudDog
# Code commonality with LoudDog (above)
return super().noise().upper()
```
There is a code commonality in the implementation of `LoudDog.noise()` and
`LoudBike.noise()`. In fact, the code is exactly the same.
`LoudBike.noise()`. In fact, the code is exactly the same. Naturally,
code like that is bound to attract software engineers.
### The "Mixin" Pattern
The *Mixin* pattern is a class with a fragment of code.
```python
class Loud(object):
class Loud:
def noise(self):
return super().noise().upper()
```
@@ -339,26 +359,31 @@ class LoudBike(Loud, Bike):
pass
```
This is one of the primary uses of multiple inheritance in Python.
Miraculously, loudness was now implemented just once and reused
in two completely unrelated classes. This sort of trick is one
of the primary uses of multiple inheritance in Python.
### Why `super()`
Always use `super()` when overriding methods.
```python
class Loud(object):
class Loud:
def noise(self):
return super().noise().upper()
```
`super()` delegates to the *next class* on the MRO.
The tricky bit is that you don't know what it is when you create the Mixin.
The tricky bit is that you don't know what it is. You especially don't
know what it is if multiple inheritance is being used.
### Some Cautions
Multiple inheritance is a powerful tool. Remember that with power comes responsibility.
Frameworks / libraries sometimes use it for advanced features involving composition of components.
Multiple inheritance is a powerful tool. Remember that with power
comes responsibility. Frameworks / libraries sometimes use it for
advanced features involving composition of components. Now, forget
that you saw that.
## Exercises
@@ -376,7 +401,8 @@ few instances:
### Exercise 5.1: Representation of Instances
At the interactive shell, inspect the underlying dictionaries of the two instances you created:
At the interactive shell, inspect the underlying dictionaries of the
two instances you created:
```python
>>> goog.__dict__

87
Notes/05_Object_model/02_Classes_encapsulation.md
View File

@@ -1,12 +1,14 @@
[Contents](../Contents) \| [Previous (5.1 Dictionaries Revisited)](01_Dicts_revisited) \| [Next (6 Generators)](../06_Generators/00_Overview)
# 5.2 Classes and Encapsulation
When writing classes, it is common to try and encapsulate internal details.
This section introduces a very Python programming idioms for this including
This section introduces a few Python programming idioms for this including
private variables and properties.
### Public vs Private.
One of the primary roles of a class is to encapsulate data an internal
One of the primary roles of a class is to encapsulate data and internal
implementation details of an object. However, a class also defines a
*public* interface that the outside world is supposed to use to
manipulate the object. This distinction between implementation
@@ -39,7 +41,8 @@ class Person(object):
self._name = 0
```
As mentioned earlier, this is only a programming style. You can still access and change it.
As mentioned earlier, this is only a programming style. You can still
access and change it.
```python
>>> p = Person('Guido')
@@ -49,22 +52,34 @@ As mentioned earlier, this is only a programming style. You can still access and
>>>
```
As a general rule, any name with a leading `_` is considered internal implementation
whether it's a variable, a function, or a module name. If you find yourself using such
names directly, you're probably doing something wrong. Look for higher level functionality.
### Simple Attributes
Consider the following class.
```python
class Stock(object):
class Stock:
def __init__(self, name, shares, price):
self.name = name
self.shares = shares
self.price = price
s = Stock('GOOG', 100, 490.1)
s.shares = 50
```
Suppose later you want to add a validation.
A surprising feature is that you can set the attributes
to any value at all:
```python
>>> s = Stock('IBM', 50, 91.1)
>>> s.shares = 100
>>> s.shares = "hundred"
>>> s.shares = [1, 0, 0]
>>>
```
You might look at that and think you want some extra checks.
```python
s.shares = '50' # Raise a TypeError, this is a string
@@ -74,10 +89,10 @@ How would you do it?
### Managed Attributes
You might introduce accessor methods.
One approach: introduce accessor methods.
```python
class Stock(object):
class Stock:
def __init__(self, name, shares, price):
self.name = name self.set_shares(shares) self.price = price
@@ -92,14 +107,15 @@ class Stock(object):
self._shares = value
```
Too bad that this breaks all of our existing code. `s.shares = 50` becomes `s.set_shares(50)`
Too bad that this breaks all of our existing code. `s.shares = 50`
becomes `s.set_shares(50)`
### Properties
There is an alternative approach to the previous pattern.
```python
class Stock(object):
class Stock:
def __init__(self, name, shares, price):
self.name = name
self.shares = shares
@@ -116,33 +132,23 @@ class Stock(object):
self._shares = value
```
Normal attribute access now triggers the getter and setter under `@property` and `@shares.setter`.
Normal attribute access now triggers the getter and setter methods
under `@property` and `@shares.setter`.
```python
class Stock(object):
def __init__(self, name, shares, price):
self.name = name
self.shares = shares
self.price = price
# Triggered with `s.shares`
@property
def shares(self):
return self._shares
# Triggered with `s.shares = ...`
@shares.setter
def shares(self, value):
if not isinstance(value, int):
raise TypeError('Expected int')
self._shares = value
>>> s = Stock('IBM', 50, 91.1)
>>> s.shares # Triggers @property
50
>>> s.shares = 75 # Triggers @shares.setter
>>>
```
With this pattern, there are *no changes* needed to the source code.
The new *setter* is also called when there is an assignment within the class.
The new *setter* is also called when there is an assignment within the class,
including inside the `__init__()` method.
```python
class Stock(object):
class Stock:
def __init__(self, name, shares, price):
...
# This assignment calls the setter below
@@ -164,7 +170,7 @@ of the class (not the property) can continue to use a name like `shares`.
Properties are also useful for computed data attributes.
```python
class Stock(object):
class Stock:
def __init__(self, name, shares, price):
self.name = name
self.shares = shares
@@ -176,7 +182,7 @@ class Stock(object):
...
```
This allows you to drop the extra parantheses, hiding the fact that it's actually method:
This allows you to drop the extra parantheses, hiding the fact that it's actually a method:
```python
>>> s = Stock('GOOG', 100, 490.1)
@@ -206,8 +212,8 @@ can fix this.
### Decorator Syntax
The `@` syntax is known as *decoration".
It specifies a modifier that's applied to the function definition that immediately follows.
The `@` syntax is known as *decoration". It specifies a modifier
that's applied to the function definition that immediately follows.
```python
...
@@ -216,14 +222,14 @@ def cost(self):
return self.shares * self.price
```
It's kind of like a macro. More details in Section 7.
More details are given in [Section 7](../07_Advanced_Topics/00_Overview).
### `__slots__` Attribute
You can restrict the set of attributes names.
```python
class Stock(object):
class Stock:
__slots__ = ('name','_shares','price')
def __init__(self, name, shares, price):
self.name = name
@@ -240,7 +246,7 @@ File "<stdin>", line 1, in ?
AttributeError: 'Stock' object has no attribute 'prices'
```
It prevents errors and restricts usage of objects. It's actually used for performance and
Although this prevents errors and restricts usage of objects, it's actually used for performance and
makes Python use memory more efficiently.
### Final Comments on Encapsulation
@@ -332,7 +338,7 @@ verify that new attributes can't be added:
>>>
```
When you use `__slots__`, Python actually uses a more efficient
When you use `__slots__`, Python uses a more efficient
internal representation of objects. What happens if you try to
inspect the underlying dictionary of `s` above?
@@ -345,5 +351,6 @@ inspect the underlying dictionary of `s` above?
It should be noted that `__slots__` is most commonly used as an
optimization on classes that serve as data structures. Using slots
will make such programs use far-less memory and run a bit faster.
You should probably avoid `__slots__` on most other classes however.
[Contents](../Contents) \| [Previous (5.1 Dictionaries Revisited)](01_Dicts_revisited) \| [Next (6 Generators)](../06_Generators/00_Overview)

18
Notes/06_Generators/01_Iteration_protocol.md
View File

@@ -1,6 +1,8 @@
[Contents](../Contents) \| [Previous (5.2 Encapsulation)](../05_Classes_objects/02_Classes_encapsulation) \| [Next (6.2 Customizing Iteration)](02_Customizing_iteration)
# 6.1 Iteration Protocol
This section looks at the process of iteration.
This section looks at the underlying process of iteration.
### Iteration Everywhere
@@ -26,7 +28,7 @@ for x in f: # Loop over lines in a file
### Iteration: Protocol
Let's take an inside look at the `for` statement.
Consider the `for`-statement.
```python
for x in obj:
@@ -45,7 +47,9 @@ while True:
# statements ...
```
All the objects that work with the `for-loop` implement this low-level iteration protocol.
All the objects that work with the `for-loop` implement this low-level
iteration protocol.
Example: Manual iteration over a list.
```python
@@ -71,7 +75,7 @@ Knowing about iteration is useful if you want to add it to your own objects.
For example, making a custom container.
```python
class Portfolio(object):
class Portfolio:
def __init__(self):
self.holdings = []
@@ -147,7 +151,7 @@ following class:
```python
# portfolio.py
class Portfolio(object):
class Portfolio:
def __init__(self, holdings):
self._holdings = holdings
@@ -203,7 +207,7 @@ that `Portfolio` instances aren't iterable.
Fix this by modifying the `Portfolio` class to support iteration:
```python
class Portfolio(object):
class Portfolio:
def __init__(self, holdings):
self._holdings = holdings
@@ -256,7 +260,7 @@ iteration. Modify the `Portfolio` class so that it has some other
special methods like this:
```python
class Portfolio(object):
class Portfolio:
def __init__(self, holdings):
self._holdings = holdings

25
Notes/06_Generators/02_Customizing_iteration.md
View File

@@ -1,6 +1,8 @@
[Contents](../Contents) \| [Previous (6.1 Iteration Protocol)](01_Iteration_protocol) \| [Next (6.3 Producer/Consumer)](03_Producers_consumers)
# 6.2 Customizing Iteration
This section looks at how you can customize iteration using a generator.
This section looks at how you can customize iteration using a generator function.
### A problem
@@ -42,7 +44,8 @@ For example:
A generator is any function that uses the `yield` statement.
The behavior of generators is different than a normal function.
Calling a generator function creates a generator object. It does not execute the function.
Calling a generator function creates a generator object. It does not
immediately execute the function.
```python
def countdown(n):
@@ -84,7 +87,7 @@ The function resumes on next call to `__next__()`.
8
```
When the generator returns, the iteration raises an error.
When the generator finally returns, the iteration raises an error.
```python
>>> x.__next__()
@@ -95,7 +98,9 @@ File "<stdin>", line 1, in ? StopIteration
>>>
```
*Observation: A generator function implements the same low-level protocol that the for statements uses on lists, tuples, dicts, files, etc.*
*Observation: A generator function implements the same low-level
protocol that the for statements uses on lists, tuples, dicts, files,
etc.*
## Exercises
@@ -147,7 +152,7 @@ explore this idea. To start, follow the next instructions carefully.
The program `Data/stocksim.py` is a program that
simulates stock market data. As output, the program constantly writes
real-time data to a file `stocklog.csv`. In a
real-time data to a file `Data/stocklog.csv`. In a
separate command window go into the `Data/` directory and run this program:
```bash
@@ -199,12 +204,12 @@ return new data or an empty string).
### Exercise 6.6: Using a generator to produce data
If you look at the code in part (b), the first part of the code is producing
If you look at the code in Exercise 6.5, the first part of the code is producing
lines of data whereas the statements at the end of the `while` loop are consuming
the data. A major feature of generator functions is that you can move all
of the data production code into a reusable function.
Modify the code in part (b) so that the file-reading is performed by
Modify the code in Exercise 6.5 so that the file-reading is performed by
a generator function `follow(filename)`. Make it so the following code
works:
@@ -250,9 +255,9 @@ if __name__ == '__main__':
print(f'{name:>10s} {price:>10.2f} {change:>10.2f}')
```
Note: For this to work, your `Portfolio` class must support the
`in` operator. See the last exercise and make sure you implement the
`__contains__()` operator.
Note: For this to work, your `Portfolio` class must support the `in`
operator. See [Exercise 6.3](01_Iteration_protocol) and make sure you
implement the `__contains__()` operator.
### Discussion

9
Notes/06_Generators/03_Producers_consumers.md
View File

@@ -1,11 +1,14 @@
[Contents](../Contents) \| [Previous (6.2 Customizing Iteration)](02_Customizing_iteration) \| [Next (6.4 Generator Expressions)](04_More_generators)
# 6.3 Producers, Consumers and Pipelines
Generators are a useful tool for setting various kinds of producer/consumer
problems and dataflow pipelines. This section discusses that.
Generators are a useful tool for setting various kinds of
producer/consumer problems and dataflow pipelines. This section
discusses that.
### Producer-Consumer Problems
Generators are closely related to various forms of *producer-consumer*.
Generators are closely related to various forms of *producer-consumer* problems.
```python
# Producer

9
Notes/06_Generators/04_More_generators.md
View File

@@ -1,7 +1,9 @@
[Contents](../Contents) \| [Previous (6.3 Producer/Consumer)](03_Producers_consumers) \| [Next (7 Advanced Topics)](../07_Advanced_Topics/00_Overview)
# 6.4 More Generators
This section introduces a few additional generator related topics including
generator expressions and the itertools module.
This section introduces a few additional generator related topics
including generator expressions and the itertools module.
### Generator Expressions
@@ -62,7 +64,8 @@ for line in lines:
f.close()
```
With generators, the code runs faster and uses little memory. It's like a filter applied to a stream.
With generators, the code runs faster and uses little memory. It's
like a filter applied to a stream.
### Why Generators