zev/practical-python
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

link experiment

Browse Source
This commit is contained in:
David Beazley
2020-05-26 09:21:19 -05:00
parent 9aec32e1a9
commit b5244b0e61
24 changed files with 4265 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

11
Notes/05_Object_model/00_Overview.md Normal file
View File

@@ -0,0 +1,11 @@
# Overview
The inner workings of Python Objects.
In this section we will cover:
* A few more details about how objects work.
* How objects are represented.
* Details of attribute access.
* Data encapsulation techniques.

620
Notes/05_Object_model/01_Dicts_revisited.md Normal file
View File

@@ -0,0 +1,620 @@
# 5.1 Dictionaries Revisited
The Python object system is largely based on an implementation based on dictionaries. This
section discusses that.
### Dictionaries, Revisited
Remember that a dictionary is a collection of names values.
```python
stock = {
'name' : 'GOOG',
'shares' : 100,
'price' : 490.1
}
```
Dictionaries are commonly used for simple data structures.
However, they are used for critical parts of the interpreter and may be the *most important type of data in Python*.
### Dicts and Modules
In a module, a dictionary holds all of the global variables and functions.
```python
# foo.py
x = 42
def bar():
...
def spam():
...
```
If we inspect `foo.__dict__` or `globals()`, you'll see the dictionary.
```python
{
'x' : 42,
'bar' : <function bar>,
'spam' : <function spam>
}
```
### Dicts and Objects
User defined objects also use dictionaries for both instance data and classes.
In fact, the entire object system is mostly an extra layer that's put on top of dictionaries.
A dictionary holds the instance data, `__dict__`.
```python
>>> s = Stock('GOOG', 100, 490.1)
>>> s.__dict__
{'name' : 'GOOG','shares' : 100, 'price': 490.1 }
```
You populate this dict (and instance) when assigning to `self`.
```python
class Stock(object):
def __init__(self,name,shares,price):
self.name = name
self.shares = shares
self.price = price
```
The instance data, `self.__dict__`, looks like this:
```python
{
'name': 'GOOG',
'shares': 100,
'price': 490.1
}
```
**Each instance gets its own private dictionary.**
```python
s = Stock('GOOG',100,490.1) # {'name' : 'GOOG','shares' : 100, 'price': 490.1 }
t = Stock('AAPL',50,123.45) # {'name' : 'AAPL','shares' : 50, 'price': 123.45 }
```
If you created 200 instances of some class, there are 100 dictionaries sitting around holding data.
### Class Members
A separate dictionary also holds the methods.
```python
class Stock(object):
def __init__(self,name,shares,price):
self.name = name
self.shares = shares
self.price = price
def cost(self):
return self.shares * self.price
def sell(self,nshares):
self.shares -= nshares
```
The dictionary is in `Stock.__dict__`.
```python
{
'cost': <function>,
'sell': <function>,
'__init__': <function>
}
```
### Instances and Classes
Instances and classes are linked together.
The `__class__` attribute refers back to the class.
```python
>>> s = Stock('GOOG', 100, 490.1)
>>> s.__dict__
{ 'name': 'GOOG', 'shares': 100, 'price': 490.1 }
>>> s.__class__
<class '__main__.Stock'>
>>>
```
The instance dictionary holds data unique to each instance, whereas the class dictionary holds data collectively shared by *all* instances.
### Attribute Access
When you work with objects, you access data and methods using the `.` operator.
```python
x = obj.name # Getting
obj.name = value # Setting
del obj.name # Deleting
```
These operations are directly tied to the dictionaries sitting underneath the covers.
### Modifying Instances
Operations that modify an object update the underlying dictionary.
```python
>>> s = Stock('GOOG', 100, 490.1)
>>> s.__dict__
{ 'name':'GOOG', 'shares': 100, 'price': 490.1 }
>>> s.shares = 50 # Setting
>>> s.date = '6/7/2007' # Setting
>>> s.__dict__
{ 'name': 'GOOG', 'shares': 50, 'price': 490.1, 'date': '6/7/2007' }
>>> del s.shares # Deleting
>>> s.__dict__
{ 'name': 'GOOG', 'price': 490.1, 'date': '6/7/2007' }
>>>
```
### Reading Attributes
Suppose you read an attribute on an instance.
```python
x = obj.name
```
The attribute may exist in two places:
* Local instance dictionary.
* Class dictionary.
Both dictionaries must be checked. First, check in local `__dict__`.
If not found, look in `__dict__` of class through `__class__`.
```python
>>> s = Stock(...)
>>> s.name
'GOOG'
>>> s.cost()
49010.0
>>>
```
This lookup scheme is how the members of a *class* get shared by all instances.
### How inheritance works
Classes may inherit from other classes.
```python
class A(B, C):
...
```
The base classes are stored in a tuple in each class.
```python
>>> A.__bases__
(<class '__main__.B'>, <class '__main__.C'>)
>>>
```
This provides a link to parent classes.
### Reading Attributes with Inheritance
First, check in local `__dict__`. If not found, look in `__dict__` of class through `__class__`.
If not found in class, look in base classes through `__bases__`.
### Reading Attributes with Single Inheritance
In inheritance hierarchies, attributes are found by walking up the inheritance tree.
```python
class A(object): pass
class B(A): pass
class C(A): pass
class D(B): pass
class E(D): pass
```
With Single Inheritance, there ia single path to the top.
You stop with the first match.
### Method Resolution Order or MRO
Python precomputes an inheritance chain and stores it in the *MRO* attribute on the class.
```python
>>> E.__mro__
(<class '__main__.E'>, <class '__main__.D'>,
<class '__main__.B'>, <class '__main__.A'>,
<type 'object'>)
>>>
```
This chain is called the **Method Resolutin Order**.
The find the attributes, Python walks the MRO. First match, wins.
### MRO in Multiple Inheritance
There is no single path to the top with multiple inheritance.
Let's take a look at an example.
```python
class A(object): pass
class B(object): pass
class C(A, B): pass
class D(B): pass
class E(C, D): pass
```
What happens when we do?
```python
e = E()
e.attr
```
A similar search process is carried out, but what is the order? That's a problem.
Python uses *cooperative multiple inheritance*.
These are some rules about class ordering:
* Children before parents
* Parents go in order
The MRO is computed using those rules.
```python
>>> E.__mro__
(
<class 'E'>,
<class 'C'>,
<class 'A'>,
<class 'D'>,
<class 'B'>,
<class 'object'>)
>>>
```
### An Odd Code Reuse
Consider two completely unrelated objects:
```python
class Dog(object):
def noise(self):
return 'Bark'
def chase(self):
return 'Chasing!'
class LoudDog(Dog):
def noise(self):
# Code commonality with LoudBike
return super().noise().upper()
```
And
```python
class Bike(object):
def noise(self):
return 'On Your Left'
def pedal(self):
return 'Pedaling!'
class LoudBike(Bike):
def noise(self):
# Code commonality with LoudDog
return super().noise().upper()
```
There is a code commonality in the implementation of `LoudDog.noise()` and
`LoudBike.noise()`. In fact, the code is exactly the same.
### The "Mixin" Pattern
The *Mixin* pattern is a class with a fragment of code.
```python
class Loud(object):
def noise(self):
return super().noise().upper()
```
This class is not usable in isolation.
It mixes with other classes via inheritance.
```python
class LoudDog(Loud, Dog):
pass
class LoudBike(Loud, Bike):
pass
```
This is one of the primary uses of multiple inheritance in Python.
### Why `super()`
Always use `super()` when overriding methods.
```python
class Loud(object):
def noise(self):
return super().noise().upper()
```
`super()` delegates to the *next class* on the MRO.
The tricky bit is that you don't know what it is when you create the Mixin.
### Some Cautions
Multiple inheritance is a powerful tool. Remember that with power comes responsibility.
Frameworks / libraries sometimes use it for advanced features involving composition of components.
## Exercises
In Exercise 4.1, you defined a class `Stock` that represented a holding of stock.
In this exercise, we will use that class.
### (a) Representation of Instances
At the interactive shell, inspect the underlying dictionaries of the two instances you created:
```python
>>> from stock import Stock
>>> goog = Stock('GOOG',100,490.10)
>>> ibm = Stock('IBM',50, 91.23)
>>> goog.__dict__
... look at the output ...
>>> ibm.__dict__
... look at the output ...
>>>
```
### (b) Modification of Instance Data
Try setting a new attribute on one of the above instances:
```python
>>> goog.date = '6/11/2007'
>>> goog.__dict__
... look at output ...
>>> ibm.__dict__
... look at output ...
>>>
```
In the above output, you’ll notice that the `goog` instance has a attribute `date` whereas the `ibm` instance does not.
It is important to note that Python really doesn’t place any restrictions on attributes.
The attributes of an instance are not limited to those set up in the `__init__()` method.
Instead of setting an attribute, try placing a new value directly into the `__dict__` object:
```python
>>> goog.__dict__['time'] = '9:45am'
>>> goog.time
'9:45am'
>>>
```
Here, you really notice the fact that an instance is just a layer on
top of a dictionary. *Note: it should be emphasized that direct
manipulation of the dictionary is uncommon—you should always write
your code to use the (.) syntax.*
### (c) The role of classes
The definitions that make up a class definition are shared by all instances of that class.
Notice, that all instances have a link back to their associated class:
```python
>>> goog.__class__
... look at output ...
>>> ibm.__class__
... look at output ...
>>>
```
Try calling a method on the instances:
```python
>>> goog.cost()
49010.0
>>> ibm.cost()
4561.5
>>>
```
Notice that the name *cost* is not defined in either `goog.__dict__`
or `ibm.__dict__`. Instead, it is being supplied by the class
dictionary. Try this:
```python
>>> Stock.__dict__['cost']
... look at output ...
>>>
```
Try calling the `cost()` method directly through the dictionary:
```python
>>> Stock.__dict__['cost'](goog)
49010.0
>>> Stock.__dict__['cost'](ibm)
4561.5
>>>
```
Notice how you are calling the function defined in the class definition and how the `self` argument gets the instance.
Try adding a new attribute to the `Stock` class:
```python
>>> Stock.foo = 42
>>>
```
Notice how this new attribute now shows up on all of the instances:
```python
>>> goog.foo
42
>>> ibm.foo
42
>>>
```
However, notice that it is not part of the instance dictionary:
```python
>>> goog.__dict__
... look at output and notice there is no 'foo' attribute ...
>>>
```
The reason you can access the `foo` attribute on instances is that
Python always checks the class dictionary if it can’t find something
on the instance itself.
This part of the exercise illustrates something known as a class
variable. Suppose, for instance, you have a class like this:
```python
class Foo(object):
a = 13 # Class variable
def __init__(self,b):
self.b = b # Instance variable
```
In this class, the variable `a`, assigned in the body of the class itself, is a *class variable*.
It is shared by all of the instances that get created.
```python
>>> f = Foo(10)
>>> g = Foo(20)
>>> f.a # Inspect the class variable (same for both instances)
13
>>> g.a
13
>>> f.b # Inspect the instance variable (differs)
10
>>> g.b
20
>>> Foo.a = 42 # Change the value of the class variable
>>> f.a
42
>>> g.a
42
>>>
```
### (d) Bound Methods
A subtle feature of Python is that invoking a method actually involves
two steps and something known as a bound method.
```python
>>> s = goog.sell
>>> s
<bound method Stock.sell of Stock('GOOG',100,490.1)>
>>> s(25)
>>> goog.shares
75
>>>
```
Bound methods actually contain all of the pieces needed to call a method.
For instance, they keep a record of the function implementing the method:
```python
>>> s.__func__
<function sell at 0x10049af50>
>>>
```
This is the same value as found in the `Stock` dictionary.
```python
>>> Stock.__dict__['sell']
<function sell at 0x10049af50>
>>>
```
Take a close look at both references do `0x10049af50`. They are both the same in `s` and `Stock.__dict__['sell']`.
Bound methods also record the instance, which is the `self` argument.
```python
>>> s.__self__
Stock('GOOG',75,490.1)
>>>
```
When you invoke the function using `()` all of the pieces come together.
For example, calling `s(25)` actually does this:
```python
>>> s.__func__(s.__self__, 25) # Same as s(25)
>>> goog.shares
50
>>>
```
### (e) Inheritance
Make a new class that inherits from `Stock`.
```python
>>> class NewStock(Stock):
def yow(self):
print('Yow!')
>>> n = NewStock('ACME', 50, 123.45)
>>> n.cost()
6172.50
>>> n.yow()
Yow!
>>>
```
Inheritance is implemented by extending the search process for attributes.
The `__bases__` attribute has a tuple of the immediate parents:
```python
>>> NewStock.__bases__
(<class 'stock.Stock'>,)
>>>
```
The `__mro__` attribute has a tuple of all parents, in the order that
they will be searched for attributes.
```python
>>> NewStock.__mro__
(<class '__main__.NewStock'>, <class 'stock.Stock'>, <class 'object'>)
>>>
```
Here’s how the `cost()` method of instance `n` above would be found:
```python
>>> for cls in n.__class__.__mro__:
if 'cost' in cls.__dict__:
break
>>> cls
<class '__main__.Stock'>
>>> cls.__dict__['cost']
<function cost at 0x101aed598>
>>>
```
[Next](02_Classes_encapsulation)

335
Notes/05_Object_model/02_Classes_encapsulation.md Normal file
View File

@@ -0,0 +1,335 @@
# 5.2 Classes and Encapsulation
When writing classes, it is common to try and encapsulate internal details.
This section introduces a very Python programming idioms for this including
private variables and properties.
### Public vs Private.
One of the primary roles of a class is to encapsulate data an internal
implementation details of an object. However, a class also defines a
*public* interface that the outside world is supposed to use to
manipulate the object. This distinction between implementation
details and the public interface is important.
### A Problem
In Python, almost everything about classes and objects is *open*.
* You can easily inspect object internals.
* You can change things at will.
* There is no strong notion of access-control (i.e., private class members)
That is an issue when you are trying to isolate details of the *internal implementation*.
### Python Encapsulation
Python relies on programming conventions to indicate the intended use
of something. These conventions are based on naming. There is a
general attitude that it is up to the programmer to observe the rules
as opposed to having the language enforce them.
### Private Attributes
Any attribute name with leading `_` is considered to be *private*.
```python
class Person(object):
def __init__(self, name):
self._name = 0
```
As mentioned earlier, this is only a programming style. You can still access and change it.
```python
>>> p = Person('Guido')
>>> p._name
'Guido'
>>> p._name = 'Dave'
>>>
```
### Simple Attributes
Consider the following class.
```python
class Stock(object):
def __init__(self, name, shares, price):
self.name = name
self.shares = shares
self.price = price
s = Stock('GOOG', 100, 490.1)
s.shares = 50
```
Suppose later you want to add a validation.
```python
s.shares = '50' # Raise a TypeError, this is a string
```
How would you do it?
### Managed Attributes
You might introduce accessor methods.
```python
class Stock(object):
def __init__(self, name, shares, price):
self.name = name self.set_shares(shares) self.price = price
# Function that layers the "get" operation
def get_shares(self):
return self._shares
# Function that layers the "set" operation
def set_shares(self, value):
if not isinstance(value, int):
raise TypeError('Expected an int')
self._shares = value
```
Too bad that this breaks all of our existing code. `s.shares = 50` becomes `s.set_shares(50)`
### Properties
There is an alternative approach to the previous pattern.
```python
class Stock(object):
def __init__(self, name, shares, price):
self.name = name
self.shares = shares
self.price = price
@property
def shares(self):
return self._shares
@shares.setter
def shares(self, value):
if not isinstance(value, int):
raise TypeError('Expected int')
self._shares = value
```
Normal attribute access now triggers the getter and setter under `@property` and `@shares.setter`.
```python
class Stock(object):
def __init__(self, name, shares, price):
self.name = name
self.shares = shares
self.price = price
# Triggered with `s.shares`
@property
def shares(self):
return self._shares
# Triggered with `s.shares = ...`
@shares.setter
def shares(self, value):
if not isinstance(value, int):
raise TypeError('Expected int')
self._shares = value
```
With this pattern, there are *no changes* needed to the source code.
The new *setter* is also called when there is an assignment within the class.
```python
class Stock(object):
def __init__(self, name, shares, price):
...
# This assignment calls the setter below
self.shares = shares
...
...
@shares.setter
def shares(self, value):
if not isinstance(value, int):
raise TypeError('Expected int')
self._shares = value
```
There is often a confusion between a property and the use of private names.
Although a property internally uses a private name like `_shares`, the rest
of the class (not the property) can continue to use a name like `shares`.
Properties are also useful for computed data attributes.
```python
class Stock(object):
def __init__(self, name, shares, price):
self.name = name
self.shares = shares
self.price = price
@property
def cost(self):
return self.shares * self.price
...
```
This allows you to drop the extra parantheses, hiding the fact that it's actually method:
```python
>>> s = Stock('GOOG', 100, 490.1)
>>> s.shares # Instance variable
100
>>> s.cost # Computed Value
49010.0
>>>
```
### Uniform access
The last example shows how to put a more uniform interface on an object.
If you don't do this, an object might be confusing to use:
```python
>>> s = Stock('GOOG', 100, 490.1)
>>> a = s.cost() # Method
49010.0
>>> b = s.shares # Data attribute
100
>>>
```
Why is the `()` required for the cost, but not for the shares? A property
can fix this.
### Decorator Syntax
The `@` syntax is known as *decoration".
It specifies a modifier that's applied to the function definition that immediately follows.
```python
...
@property
def cost(self):
return self.shares * self.price
```
It's kind of like a macro. More details in Section 7.
### `__slots__` Attribute
You can restrict the set of attributes names.
```python
class Stock(object):
__slots__ = ('name','_shares','price')
def __init__(self, name, shares, price):
self.name = name
...
```
It will raise an error for other attributes.
```python
>>> s.price = 385.15
>>> s.prices = 410.2
Traceback (most recent call last):
File "<stdin>", line 1, in ?
AttributeError: 'Stock' object has no attribute 'prices'
```
It prevents errors and restricts usage of objects. It's actually used for performance and
makes Python use memory more efficiently.
### Final Comments on Encapsulation
Don't go overboard with private attributes, properties, slots,
etc. They serve a specific purpose and you may see them when reading
other Python code. However, they are not necessary for most
day-to-day coding.
## Exercises
### (a) Simple properties
Properties are a useful way to add "computed attributes" to an object.
In Exercise 4.1, you created an object `Stock`. Notice that on your
object there is a slight inconsistency in how different kinds of data
are extracted:
```python
>>> from stock import Stock
>>> s = Stock('GOOG', 100, 490.1)
>>> s.shares
100
>>> s.price
490.1
>>> s.cost()
49010.0
>>>
```
Specifically, notice how you have to add the extra `()` to `cost` because it is a method.
You can get rid of the extra `()` on `cost()` if you turn it into a property.
Take your `Stock` class and modify it so that the cost calculation works like this:
```python
>>> s.cost
49010.0
>>>
```
Try calling `s.cost()` as a function and observe that it doesn’t work now that `cost` has been defined as a property.
```python
>>> s.cost()
... fails ...
>>>
```
### (b) Properties and Setters
Modify the `shares` attribute so that the value is stored in a private
attribute and that a pair of property functions are used to ensure
that it is always set to an integer value.
Here is an example of the expected behavior:
```python
>>> s = Stock('GOOG',100,490.10)
>>> s.shares = 50
>>> s.shares = 'a lot'
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: expected an integer
>>>
```
### (c) Adding slots
Modify the `Stock` class so that it has a `__slots__` attribute.
Then, verify that new attributes can’t be added:
```python
>>> from stock import Stock
>>> s = Stock('GOOG', 100, 490.10)
>>> s.name
'GOOG'
>>> s.blah = 42
... see what happens ...
>>>
```
When you use `__slots__`, Python actually uses a more efficient internal representation of objects.
What happens if you try to inspect the underlying dictionary of `s` above?
```python
>>> s.__dict__
... see what happens ...
>>>
```
It should be noted that `__slots__` is most commonly used as an
optimization on classes that serve as data structures. Using slots
will make such programs use far-less memory and run a bit faster.