diff --git a/Notes/05_Object_model/01_Dicts_revisited.md b/Notes/05_Object_model/01_Dicts_revisited.md index 3823cee..e02ed31 100644 --- a/Notes/05_Object_model/01_Dicts_revisited.md +++ b/Notes/05_Object_model/01_Dicts_revisited.md @@ -1,7 +1,9 @@ +[Contents](../Contents) \| [Previous (4.4 Exceptions)](../04_Classes_objects/04_Defining_exceptions) \| [Next (5.2 Encapsulation)](02_Classes_encapsulation) + # 5.1 Dictionaries Revisited -The Python object system is largely based on an implementation based on dictionaries. This -section discusses that. +The Python object system is largely based on an implementation +involving dictionaries. This section discusses that. ### Dictionaries, Revisited @@ -15,12 +17,14 @@ stock = { } ``` -Dictionaries are commonly used for simple data structures. -However, they are used for critical parts of the interpreter and may be the *most important type of data in Python*. +Dictionaries are commonly used for simple data structures. However, +they are used for critical parts of the interpreter and may be the +*most important type of data in Python*. ### Dicts and Modules -In a module, a dictionary holds all of the global variables and functions. +Within a module, a dictionary holds all of the global variables and +functions. ```python # foo.py @@ -33,7 +37,7 @@ def spam(): ... ``` -If we inspect `foo.__dict__` or `globals()`, you'll see the dictionary. +If you inspect `foo.__dict__` or `globals()`, you'll see the dictionary. ```python { @@ -45,8 +49,9 @@ If we inspect `foo.__dict__` or `globals()`, you'll see the dictionary. ### Dicts and Objects -User defined objects also use dictionaries for both instance data and classes. -In fact, the entire object system is mostly an extra layer that's put on top of dictionaries. +User defined objects also use dictionaries for both instance data and +classes. In fact, the entire object system is mostly an extra layer +that's put on top of dictionaries. A dictionary holds the instance data, `__dict__`. @@ -59,8 +64,8 @@ A dictionary holds the instance data, `__dict__`. You populate this dict (and instance) when assigning to `self`. ```python -class Stock(object): - def __init__(self,name,shares,price): +class Stock: + def __init__(self, name, shares, price): self.name = name self.shares = shares self.price = price @@ -79,19 +84,20 @@ The instance data, `self.__dict__`, looks like this: **Each instance gets its own private dictionary.** ```python -s = Stock('GOOG',100,490.1) # {'name' : 'GOOG','shares' : 100, 'price': 490.1 } -t = Stock('AAPL',50,123.45) # {'name' : 'AAPL','shares' : 50, 'price': 123.45 } +s = Stock('GOOG', 100, 490.1) # {'name' : 'GOOG','shares' : 100, 'price': 490.1 } +t = Stock('AAPL', 50, 123.45) # {'name' : 'AAPL','shares' : 50, 'price': 123.45 } ``` -If you created 200 instances of some class, there are 100 dictionaries sitting around holding data. +If you created 100 instances of some class, there are 100 dictionaries +sitting around holding data. ### Class Members A separate dictionary also holds the methods. ```python -class Stock(object): - def __init__(self,name,shares,price): +class Stock: + def __init__(self, name, shares, price): self.name = name self.shares = shares self.price = price @@ -99,7 +105,7 @@ class Stock(object): def cost(self): return self.shares * self.price - def sell(self,nshares): + def sell(self, nshares): self.shares -= nshares ``` @@ -115,8 +121,8 @@ The dictionary is in `Stock.__dict__`. ### Instances and Classes -Instances and classes are linked together. -The `__class__` attribute refers back to the class. +Instances and classes are linked together. The `__class__` attribute +refers back to the class. ```python >>> s = Stock('GOOG', 100, 490.1) @@ -127,7 +133,9 @@ The `__class__` attribute refers back to the class. >>> ``` -The instance dictionary holds data unique to each instance, whereas the class dictionary holds data collectively shared by *all* instances. +The instance dictionary holds data unique to each instance, whereas +the class dictionary holds data collectively shared by *all* +instances. ### Attribute Access @@ -207,26 +215,30 @@ This provides a link to parent classes. ### Reading Attributes with Inheritance -First, check in local `__dict__`. If not found, look in `__dict__` of class through `__class__`. -If not found in class, look in base classes through `__bases__`. +Logically, the process of finding an attribute is as follows. First, +check in local `__dict__`. If not found, look in `__dict__` of the +class. If not found in class, look in the base classes through +`__bases__`. However, there are some subtle aspects of this discussed next. ### Reading Attributes with Single Inheritance -In inheritance hierarchies, attributes are found by walking up the inheritance tree. +In inheritance hierarchies, attributes are found by walking up the +inheritance tree in order. ```python -class A(object): pass +class A: pass class B(A): pass class C(A): pass class D(B): pass class E(D): pass ``` -With Single Inheritance, there ia single path to the top. +With single inheritance, there is single path to the top. You stop with the first match. ### Method Resolution Order or MRO Python precomputes an inheritance chain and stores it in the *MRO* attribute on the class. +You can view it. ```python >>> E.__mro__ @@ -236,38 +248,39 @@ Python precomputes an inheritance chain and stores it in the *MRO* attribute on >>> ``` -This chain is called the **Method Resolutin Order**. -The find the attributes, Python walks the MRO. First match, wins. +This chain is called the **Method Resolutin Order**. The find an +attribute, Python walks the MRO in order. The first match wins. ### MRO in Multiple Inheritance -There is no single path to the top with multiple inheritance. +With multiple inheritance, there is no single path to the top. Let's take a look at an example. ```python -class A(object): pass -class B(object): pass +class A: pass +class B: pass class C(A, B): pass class D(B): pass class E(C, D): pass ``` -What happens when we do? +What happens when you access at attribute? ```python e = E() e.attr ``` -A similar search process is carried out, but what is the order? That's a problem. +A attribute search process is carried out, but what is the order? That's a problem. -Python uses *cooperative multiple inheritance*. -These are some rules about class ordering: +Python uses *cooperative multiple inheritance* which obeys some rules +about class ordering. -* Children before parents -* Parents go in order +* Children are always checked before parents +* Parents (if multiple) are always checked in the order listed. -The MRO is computed using those rules. +The MRO is computed by sorting all of the classes in a hierarchy +according to those rules. ```python >>> E.__mro__ @@ -281,12 +294,18 @@ The MRO is computed using those rules. >>> ``` -### An Odd Code Reuse +The underlying algorithm is called the "C3 Linearization Algorithm." +The precise details aren't important as long as you remember that a +class hierarchy obeys the same ordering rules you might follow if your +house was on fire and you had to evacuate--children first, followed by +parents. + +### An Odd Code Reuse (Involving Multiple Inheritance) Consider two completely unrelated objects: ```python -class Dog(object): +class Dog: def noise(self): return 'Bark' @@ -295,14 +314,14 @@ class Dog(object): class LoudDog(Dog): def noise(self): - # Code commonality with LoudBike + # Code commonality with LoudBike (below) return super().noise().upper() ``` And ```python -class Bike(object): +class Bike: def noise(self): return 'On Your Left' @@ -311,19 +330,20 @@ class Bike(object): class LoudBike(Bike): def noise(self): - # Code commonality with LoudDog + # Code commonality with LoudDog (above) return super().noise().upper() ``` There is a code commonality in the implementation of `LoudDog.noise()` and -`LoudBike.noise()`. In fact, the code is exactly the same. +`LoudBike.noise()`. In fact, the code is exactly the same. Naturally, +code like that is bound to attract software engineers. ### The "Mixin" Pattern The *Mixin* pattern is a class with a fragment of code. ```python -class Loud(object): +class Loud: def noise(self): return super().noise().upper() ``` @@ -339,26 +359,31 @@ class LoudBike(Loud, Bike): pass ``` -This is one of the primary uses of multiple inheritance in Python. +Miraculously, loudness was now implemented just once and reused +in two completely unrelated classes. This sort of trick is one +of the primary uses of multiple inheritance in Python. ### Why `super()` Always use `super()` when overriding methods. ```python -class Loud(object): +class Loud: def noise(self): return super().noise().upper() ``` `super()` delegates to the *next class* on the MRO. -The tricky bit is that you don't know what it is when you create the Mixin. +The tricky bit is that you don't know what it is. You especially don't +know what it is if multiple inheritance is being used. ### Some Cautions -Multiple inheritance is a powerful tool. Remember that with power comes responsibility. -Frameworks / libraries sometimes use it for advanced features involving composition of components. +Multiple inheritance is a powerful tool. Remember that with power +comes responsibility. Frameworks / libraries sometimes use it for +advanced features involving composition of components. Now, forget +that you saw that. ## Exercises @@ -376,7 +401,8 @@ few instances: ### Exercise 5.1: Representation of Instances -At the interactive shell, inspect the underlying dictionaries of the two instances you created: +At the interactive shell, inspect the underlying dictionaries of the +two instances you created: ```python >>> goog.__dict__ @@ -537,7 +563,7 @@ two steps and something known as a bound method. For example: ```python >>> s = goog.sell >>> s - + >>> s(25) >>> goog.shares 75 diff --git a/Notes/05_Object_model/02_Classes_encapsulation.md b/Notes/05_Object_model/02_Classes_encapsulation.md index 8528a16..2c4d386 100644 --- a/Notes/05_Object_model/02_Classes_encapsulation.md +++ b/Notes/05_Object_model/02_Classes_encapsulation.md @@ -1,12 +1,14 @@ +[Contents](../Contents) \| [Previous (5.1 Dictionaries Revisited)](01_Dicts_revisited) \| [Next (6 Generators)](../06_Generators/00_Overview) + # 5.2 Classes and Encapsulation When writing classes, it is common to try and encapsulate internal details. -This section introduces a very Python programming idioms for this including +This section introduces a few Python programming idioms for this including private variables and properties. ### Public vs Private. -One of the primary roles of a class is to encapsulate data an internal +One of the primary roles of a class is to encapsulate data and internal implementation details of an object. However, a class also defines a *public* interface that the outside world is supposed to use to manipulate the object. This distinction between implementation @@ -39,7 +41,8 @@ class Person(object): self._name = 0 ``` -As mentioned earlier, this is only a programming style. You can still access and change it. +As mentioned earlier, this is only a programming style. You can still +access and change it. ```python >>> p = Person('Guido') @@ -49,22 +52,34 @@ As mentioned earlier, this is only a programming style. You can still access and >>> ``` +As a general rule, any name with a leading `_` is considered internal implementation +whether it's a variable, a function, or a module name. If you find yourself using such +names directly, you're probably doing something wrong. Look for higher level functionality. + ### Simple Attributes Consider the following class. ```python -class Stock(object): +class Stock: def __init__(self, name, shares, price): self.name = name self.shares = shares self.price = price - -s = Stock('GOOG', 100, 490.1) -s.shares = 50 ``` -Suppose later you want to add a validation. +A surprising feature is that you can set the attributes +to any value at all: + +```python +>>> s = Stock('IBM', 50, 91.1) +>>> s.shares = 100 +>>> s.shares = "hundred" +>>> s.shares = [1, 0, 0] +>>> +``` + +You might look at that and think you want some extra checks. ```python s.shares = '50' # Raise a TypeError, this is a string @@ -74,10 +89,10 @@ How would you do it? ### Managed Attributes -You might introduce accessor methods. +One approach: introduce accessor methods. ```python -class Stock(object): +class Stock: def __init__(self, name, shares, price): self.name = name self.set_shares(shares) self.price = price @@ -92,14 +107,15 @@ class Stock(object): self._shares = value ``` -Too bad that this breaks all of our existing code. `s.shares = 50` becomes `s.set_shares(50)` +Too bad that this breaks all of our existing code. `s.shares = 50` +becomes `s.set_shares(50)` ### Properties There is an alternative approach to the previous pattern. ```python -class Stock(object): +class Stock: def __init__(self, name, shares, price): self.name = name self.shares = shares @@ -116,33 +132,23 @@ class Stock(object): self._shares = value ``` -Normal attribute access now triggers the getter and setter under `@property` and `@shares.setter`. +Normal attribute access now triggers the getter and setter methods +under `@property` and `@shares.setter`. ```python -class Stock(object): - def __init__(self, name, shares, price): - self.name = name - self.shares = shares - self.price = price - - # Triggered with `s.shares` - @property - def shares(self): - return self._shares - - # Triggered with `s.shares = ...` - @shares.setter - def shares(self, value): - if not isinstance(value, int): - raise TypeError('Expected int') - self._shares = value +>>> s = Stock('IBM', 50, 91.1) +>>> s.shares # Triggers @property +50 +>>> s.shares = 75 # Triggers @shares.setter +>>> ``` With this pattern, there are *no changes* needed to the source code. -The new *setter* is also called when there is an assignment within the class. +The new *setter* is also called when there is an assignment within the class, +including inside the `__init__()` method. ```python -class Stock(object): +class Stock: def __init__(self, name, shares, price): ... # This assignment calls the setter below @@ -164,7 +170,7 @@ of the class (not the property) can continue to use a name like `shares`. Properties are also useful for computed data attributes. ```python -class Stock(object): +class Stock: def __init__(self, name, shares, price): self.name = name self.shares = shares @@ -176,7 +182,7 @@ class Stock(object): ... ``` -This allows you to drop the extra parantheses, hiding the fact that it's actually method: +This allows you to drop the extra parantheses, hiding the fact that it's actually a method: ```python >>> s = Stock('GOOG', 100, 490.1) @@ -206,8 +212,8 @@ can fix this. ### Decorator Syntax -The `@` syntax is known as *decoration". -It specifies a modifier that's applied to the function definition that immediately follows. +The `@` syntax is known as *decoration". It specifies a modifier +that's applied to the function definition that immediately follows. ```python ... @@ -216,14 +222,14 @@ def cost(self): return self.shares * self.price ``` -It's kind of like a macro. More details in Section 7. +More details are given in [Section 7](../07_Advanced_Topics/00_Overview). ### `__slots__` Attribute You can restrict the set of attributes names. ```python -class Stock(object): +class Stock: __slots__ = ('name','_shares','price') def __init__(self, name, shares, price): self.name = name @@ -240,7 +246,7 @@ File "", line 1, in ? AttributeError: 'Stock' object has no attribute 'prices' ``` -It prevents errors and restricts usage of objects. It's actually used for performance and +Although this prevents errors and restricts usage of objects, it's actually used for performance and makes Python use memory more efficiently. ### Final Comments on Encapsulation @@ -332,7 +338,7 @@ verify that new attributes can't be added: >>> ``` -When you use `__slots__`, Python actually uses a more efficient +When you use `__slots__`, Python uses a more efficient internal representation of objects. What happens if you try to inspect the underlying dictionary of `s` above? @@ -345,5 +351,6 @@ inspect the underlying dictionary of `s` above? It should be noted that `__slots__` is most commonly used as an optimization on classes that serve as data structures. Using slots will make such programs use far-less memory and run a bit faster. +You should probably avoid `__slots__` on most other classes however. [Contents](../Contents) \| [Previous (5.1 Dictionaries Revisited)](01_Dicts_revisited) \| [Next (6 Generators)](../06_Generators/00_Overview) diff --git a/Notes/06_Generators/01_Iteration_protocol.md b/Notes/06_Generators/01_Iteration_protocol.md index c6f834d..eb99f72 100644 --- a/Notes/06_Generators/01_Iteration_protocol.md +++ b/Notes/06_Generators/01_Iteration_protocol.md @@ -1,6 +1,8 @@ +[Contents](../Contents) \| [Previous (5.2 Encapsulation)](../05_Classes_objects/02_Classes_encapsulation) \| [Next (6.2 Customizing Iteration)](02_Customizing_iteration) + # 6.1 Iteration Protocol -This section looks at the process of iteration. +This section looks at the underlying process of iteration. ### Iteration Everywhere @@ -26,7 +28,7 @@ for x in f: # Loop over lines in a file ### Iteration: Protocol -Let's take an inside look at the `for` statement. +Consider the `for`-statement. ```python for x in obj: @@ -45,7 +47,9 @@ while True: # statements ... ``` -All the objects that work with the `for-loop` implement this low-level iteration protocol. +All the objects that work with the `for-loop` implement this low-level +iteration protocol. + Example: Manual iteration over a list. ```python @@ -71,7 +75,7 @@ Knowing about iteration is useful if you want to add it to your own objects. For example, making a custom container. ```python -class Portfolio(object): +class Portfolio: def __init__(self): self.holdings = [] @@ -147,7 +151,7 @@ following class: ```python # portfolio.py -class Portfolio(object): +class Portfolio: def __init__(self, holdings): self._holdings = holdings @@ -203,7 +207,7 @@ that `Portfolio` instances aren't iterable. Fix this by modifying the `Portfolio` class to support iteration: ```python -class Portfolio(object): +class Portfolio: def __init__(self, holdings): self._holdings = holdings @@ -256,7 +260,7 @@ iteration. Modify the `Portfolio` class so that it has some other special methods like this: ```python -class Portfolio(object): +class Portfolio: def __init__(self, holdings): self._holdings = holdings diff --git a/Notes/06_Generators/02_Customizing_iteration.md b/Notes/06_Generators/02_Customizing_iteration.md index 50e7ac4..6ff089a 100644 --- a/Notes/06_Generators/02_Customizing_iteration.md +++ b/Notes/06_Generators/02_Customizing_iteration.md @@ -1,6 +1,8 @@ +[Contents](../Contents) \| [Previous (6.1 Iteration Protocol)](01_Iteration_protocol) \| [Next (6.3 Producer/Consumer)](03_Producers_consumers) + # 6.2 Customizing Iteration -This section looks at how you can customize iteration using a generator. +This section looks at how you can customize iteration using a generator function. ### A problem @@ -42,7 +44,8 @@ For example: A generator is any function that uses the `yield` statement. The behavior of generators is different than a normal function. -Calling a generator function creates a generator object. It does not execute the function. +Calling a generator function creates a generator object. It does not +immediately execute the function. ```python def countdown(n): @@ -84,7 +87,7 @@ The function resumes on next call to `__next__()`. 8 ``` -When the generator returns, the iteration raises an error. +When the generator finally returns, the iteration raises an error. ```python >>> x.__next__() @@ -95,7 +98,9 @@ File "", line 1, in ? StopIteration >>> ``` -*Observation: A generator function implements the same low-level protocol that the for statements uses on lists, tuples, dicts, files, etc.* +*Observation: A generator function implements the same low-level + protocol that the for statements uses on lists, tuples, dicts, files, + etc.* ## Exercises @@ -147,7 +152,7 @@ explore this idea. To start, follow the next instructions carefully. The program `Data/stocksim.py` is a program that simulates stock market data. As output, the program constantly writes -real-time data to a file `stocklog.csv`. In a +real-time data to a file `Data/stocklog.csv`. In a separate command window go into the `Data/` directory and run this program: ```bash @@ -199,12 +204,12 @@ return new data or an empty string). ### Exercise 6.6: Using a generator to produce data -If you look at the code in part (b), the first part of the code is producing +If you look at the code in Exercise 6.5, the first part of the code is producing lines of data whereas the statements at the end of the `while` loop are consuming the data. A major feature of generator functions is that you can move all of the data production code into a reusable function. -Modify the code in part (b) so that the file-reading is performed by +Modify the code in Exercise 6.5 so that the file-reading is performed by a generator function `follow(filename)`. Make it so the following code works: @@ -250,9 +255,9 @@ if __name__ == '__main__': print(f'{name:>10s} {price:>10.2f} {change:>10.2f}') ``` -Note: For this to work, your `Portfolio` class must support the -`in` operator. See the last exercise and make sure you implement the -`__contains__()` operator. +Note: For this to work, your `Portfolio` class must support the `in` +operator. See [Exercise 6.3](01_Iteration_protocol) and make sure you +implement the `__contains__()` operator. ### Discussion diff --git a/Notes/06_Generators/03_Producers_consumers.md b/Notes/06_Generators/03_Producers_consumers.md index c3f19f3..ccfd8f1 100644 --- a/Notes/06_Generators/03_Producers_consumers.md +++ b/Notes/06_Generators/03_Producers_consumers.md @@ -1,11 +1,14 @@ +[Contents](../Contents) \| [Previous (6.2 Customizing Iteration)](02_Customizing_iteration) \| [Next (6.4 Generator Expressions)](04_More_generators) + # 6.3 Producers, Consumers and Pipelines -Generators are a useful tool for setting various kinds of producer/consumer -problems and dataflow pipelines. This section discusses that. +Generators are a useful tool for setting various kinds of +producer/consumer problems and dataflow pipelines. This section +discusses that. ### Producer-Consumer Problems -Generators are closely related to various forms of *producer-consumer*. +Generators are closely related to various forms of *producer-consumer* problems. ```python # Producer diff --git a/Notes/06_Generators/04_More_generators.md b/Notes/06_Generators/04_More_generators.md index 33a5126..aefdfc6 100644 --- a/Notes/06_Generators/04_More_generators.md +++ b/Notes/06_Generators/04_More_generators.md @@ -1,7 +1,9 @@ +[Contents](../Contents) \| [Previous (6.3 Producer/Consumer)](03_Producers_consumers) \| [Next (7 Advanced Topics)](../07_Advanced_Topics/00_Overview) + # 6.4 More Generators -This section introduces a few additional generator related topics including -generator expressions and the itertools module. +This section introduces a few additional generator related topics +including generator expressions and the itertools module. ### Generator Expressions @@ -62,7 +64,8 @@ for line in lines: f.close() ``` -With generators, the code runs faster and uses little memory. It's like a filter applied to a stream. +With generators, the code runs faster and uses little memory. It's +like a filter applied to a stream. ### Why Generators