diff --git a/Notes/02_Working_with_data/01_Datatypes.md b/Notes/02_Working_with_data/01_Datatypes.md index 000c6c5..93f5fca 100644 --- a/Notes/02_Working_with_data/01_Datatypes.md +++ b/Notes/02_Working_with_data/01_Datatypes.md @@ -1,7 +1,5 @@ # 2.1 Datatypes and Data structures -This section introduces data structures in the form of tuples and dicts. - ### Primitive Datatypes Python has a few primitive types of data: @@ -10,7 +8,7 @@ Python has a few primitive types of data: * Floating point numbers * Strings (text) -We have learned about these in the previous section. +We learned about these in the introduction. ### None type @@ -212,7 +210,7 @@ TypeError: can't multiply sequence by non-int of type 'str' To do more, you typically want to interpret the raw data in some way and turn it into a more useful kind of object so that you can work with it later. Two simple options are tuples or dictionaries. -### (a) Tuples +### Exercise 2.1: Tuples At the interactive prompt, create the following tuple that represents the above row, but with the numeric columns converted to proper @@ -298,7 +296,7 @@ Take the above variables and pack them back into a tuple >>> ``` -### (b) Dictionaries as a data structure +### Exercise 2.2: Dictionaries as a data structure An alternative to a tuple is to create a dictionary instead. @@ -341,7 +339,7 @@ Unlike tuples, dictionaries can be freely modified. Add some attributes: >>> ``` -### (c) Some additional dictionary operations +### Exercise 2.3: Some additional dictionary operations If you turn a dictionary into a list, you’ll get all of its keys: @@ -428,4 +426,4 @@ dict_items([('name', 'AA'), ('shares', 75), ('price', 32.2), ('date', (6, 11, 20 >>> ``` -[Next](02_Containers) +[Contents](../Contents) \| [Previous (1.6 Files)](../01_Introduction/06_Files) \| [Next (2.2 Containers)](02_Containers) diff --git a/Notes/02_Working_with_data/02_Containers.md b/Notes/02_Working_with_data/02_Containers.md index 855f613..5d04d36 100644 --- a/Notes/02_Working_with_data/02_Containers.md +++ b/Notes/02_Working_with_data/02_Containers.md @@ -194,12 +194,12 @@ s1 - s2 # Set difference ## Exercises -### Objectives +### Exercise 2.4: A list of tuples -### Exercise A: A list of tuples - -The file `Data/portfolio.csv` contains a list of stocks in a portfolio. -In [Section 1.7](), you wrote a function `portfolio_cost(filename)` that read this file and performed a simple calculation. +The file `Data/portfolio.csv` contains a list of stocks in a +portfolio. In [Exercise 1.30](../01_Introduction/07_Functions), you +wrote a function `portfolio_cost(filename)` that read this file and +performed a simple calculation. Your code should have looked something like this: @@ -245,7 +245,9 @@ for row in rows: Finally, you’ll return the resulting `portfolio` list. -Experiment with your function interactively (just a reminder that in order to do this, you first have to run the `report.py` program in the interpreter): +Experiment with your function interactively (just a reminder that in +order to do this, you first have to run the `report.py` program in the +interpreter): *Hint: Use `-i` when executing the file in the terminal* @@ -270,8 +272,10 @@ Experiment with your function interactively (just a reminder that in order to do >>> ``` -This list of tuples that you have created is very similar to a 2-D array. -For example, you can access a specific column and row using a lookup such as `portfolio[row][column]` where `row` and `column` are integers. +This list of tuples that you have created is very similar to a 2-D +array. For example, you can access a specific column and row using a +lookup such as `portfolio[row][column]` where `row` and `column` are +integers. That said, you can also rewrite the last for-loop using a statement like this: @@ -285,12 +289,14 @@ That said, you can also rewrite the last for-loop using a statement like this: >>> ``` -### (b) List of Dictionaries +### Exercise 2.5: List of Dictionaries -Take the function you wrote in part (a) and modify to represent each stock in the portfolio with a dictionary instead of a tuple. -In this dictionary use the field names of "name", "shares", and "price" to represent the different columns in the input file. +Take the function you wrote in part (a) and modify to represent each +stock in the portfolio with a dictionary instead of a tuple. In this +dictionary use the field names of "name", "shares", and "price" to +represent the different columns in the input file. -Experiment with this new function in the same manner as you did in part (a). +Experiment with this new function in the same manner as you did in Exercise 2.4. ```pycon >>> portfolio = read_portfolio('portfolio.csv') @@ -314,10 +320,12 @@ Experiment with this new function in the same manner as you did in part (a). >>> ``` -Here, you will notice that the different fields for each entry are accessed by key names instead of numeric column numbers. -This is often preferred because the resulting code is easier to read later. +Here, you will notice that the different fields for each entry are +accessed by key names instead of numeric column numbers. This is +often preferred because the resulting code is easier to read later. -Viewing large dictionaries and lists can be messy. To clean up the output for debugging, considering using the `pprint` function. +Viewing large dictionaries and lists can be messy. To clean up the +output for debugging, considering using the `pprint` function. ```pycon >>> from pprint import pprint @@ -332,10 +340,11 @@ Viewing large dictionaries and lists can be messy. To clean up the output for de >>> ``` -### (c) Dictionaries as a container +### Exercise 2.6: Dictionaries as a container -A dictionary is a useful way to keep track of items where you want to look up items using an index other than an integer. -In the Python shell, try playing with a dictionary: +A dictionary is a useful way to keep track of items where you want to +look up items using an index other than an integer. In the Python +shell, try playing with a dictionary: ```pycon >>> prices = { } @@ -364,14 +373,20 @@ The file looks something like this: ... ``` -Write a function `read_prices(filename)` that reads a set of prices such as this into a dictionary where the keys of the dictionary are the stock names and the values in the dictionary are the stock prices. +Write a function `read_prices(filename)` that reads a set of prices +such as this into a dictionary where the keys of the dictionary are +the stock names and the values in the dictionary are the stock prices. -To do this, start with an empty dictionary and start inserting values into it just -as you did above. However, you are reading the values from a file now. +To do this, start with an empty dictionary and start inserting values +into it just as you did above. However, you are reading the values +from a file now. -We’ll use this data structure to quickly lookup the price of a given stock name. +We’ll use this data structure to quickly lookup the price of a given +stock name. -A few little tips that you’ll need for this part. First, make sure you use the `csv` module just as you did before—there’s no need to reinvent the wheel here. +A few little tips that you’ll need for this part. First, make sure you +use the `csv` module just as you did before—there’s no need to +reinvent the wheel here. ```pycon >>> import csv @@ -388,12 +403,16 @@ A few little tips that you’ll need for this part. First, make sure you use the >>> ``` -The other little complication is that the `Data/prices.csv` file may have some blank lines in it. Notice how the last row of data above is an empty list—meaning no data was present on that line. +The other little complication is that the `Data/prices.csv` file may +have some blank lines in it. Notice how the last row of data above is +an empty list—meaning no data was present on that line. -There’s a possibility that this could cause your program to die with an exception. -Use the `try` and `except` statements to catch this as appropriate. +There’s a possibility that this could cause your program to die with +an exception. Use the `try` and `except` statements to catch this as +appropriate. -Once you have written your `read_prices()` function, test it interactively to make sure it works: +Once you have written your `read_prices()` function, test it +interactively to make sure it works: ```python >>> prices = read_prices('Data/prices.csv') @@ -404,10 +423,12 @@ Once you have written your `read_prices()` function, test it interactively to ma >>> ``` -### (e) Finding out if you can retire +### Exercise 2.7: Finding out if you can retire -Tie all of this work together by adding the statements to your `report.py` program. -It takes the list of stocks in part (b) and the dictionary of prices in part (c) and -computes the current value of the portfolio along with the gain/loss. +Tie all of this work together by adding the statements to your +`report.py` program. It takes the list of stocks in Exercise 2.5 and +the dictionary of prices in Exercise 2.6 and computes the current +value of the portfolio along with the gain/loss. + +[Contents](../Contents) \| [Previous (2.1 Datatypes)](01_Datatypes) \| [Next (2.3 Formatting)](03_Formatting) -[Next](03_Formatting) diff --git a/Notes/02_Working_with_data/03_Formatting.md b/Notes/02_Working_with_data/03_Formatting.md index d8e3158..97ab2cd 100644 --- a/Notes/02_Working_with_data/03_Formatting.md +++ b/Notes/02_Working_with_data/03_Formatting.md @@ -101,7 +101,7 @@ b'Dave has 37 messages' ## Exercises -In the previous exercise, you wrote a program called `report.py` that computed the gain/loss of a +In Exercise 2.7, you wrote a program called `report.py` that computed the gain/loss of a stock portfolio. In this exercise, you're going to modify it to produce a table like this: ```code @@ -116,12 +116,15 @@ stock portfolio. In this exercise, you're going to modify it to produce a table IBM 100 106.28 35.84 ``` -In this report, "Price" is the current share price of the stock and "Change" is the change in the share price from the initial purchase price. +In this report, "Price" is the current share price of the stock and +"Change" is the change in the share price from the initial purchase +price. -### (a) How to format numbers +### Exercise 2.8: How to format numbers -A common problem with printing numbers is specifying the number of decimal places. One way to fix this is to use f-strings. Try -these examples: +A common problem with printing numbers is specifying the number of +decimal places. One way to fix this is to use f-strings. Try these +examples: ```python >>> value = 42863.1 @@ -150,7 +153,8 @@ is also sometimes performed using the `%` operator of strings. >>> ``` -Documentation on various codes used with `%` can be found [here](https://docs.python.org/3/library/stdtypes.html#printf-style-string-formatting). +Documentation on various codes used with `%` can be found +[here](https://docs.python.org/3/library/stdtypes.html#printf-style-string-formatting). Although it’s commonly used with `print`, string formatting is not tied to printing. If you want to save a formatted string. Just assign it to a variable. @@ -162,14 +166,15 @@ If you want to save a formatted string. Just assign it to a variable. >>> ``` -### (b) Collecting Data +### Exercise 2.9: Collecting Data In order to generate the above report, you’ll first want to collect all of the data shown in the table. Write a function `make_report()` that takes a list of stocks and dictionary of prices as input and returns a list of tuples containing the rows of the above table. -Add this function to your `report.py` file. Here’s how it should work if you try it interactively: +Add this function to your `report.py` file. Here’s how it should work +if you try it interactively: ```pycon >>> portfolio = read_portfolio('Data/portfolio.csv') @@ -187,9 +192,10 @@ Add this function to your `report.py` file. Here’s how it should work if you t >>> ``` -### (c) Printing a formatted table +### Exercise 2.10: Printing a formatted table -Redo the above for-loop, but change the print statement to format the tuples. +Redo the for-loop in Exercise 2.9, but change the print statement to +format the tuples. ```pycon >>> for r in report: @@ -220,7 +226,7 @@ You can also expand the values and use f-strings. For example: Take the above statements and add them to your `report.py` program. Have your program take the output of the `make_report()` function and print a nicely formatted table as shown. -### (d) Adding some headers +### Exercise 2.11: Adding some headers Suppose you had a tuple of header names like this: @@ -257,7 +263,7 @@ When you’re done, your program should produce the table shown at the top of th IBM 100 106.28 35.84 ``` -### (e) Formatting Challenge +### Exercise 2.12: Formatting Challenge How would you modify your code so that the price includes the currency symbol ($) and the output looks like this: @@ -273,4 +279,4 @@ How would you modify your code so that the price includes the currency symbol ($ IBM 100 $106.28 35.84 ``` -[Next](04_Sequences) \ No newline at end of file +[Contents](../Contents) \| [Previous (2.2 Containers)](02_Containers) \| [Next (2.4 Sequences)](04_Sequences) diff --git a/Notes/02_Working_with_data/04_Sequences.md b/Notes/02_Working_with_data/04_Sequences.md index e92ae0d..cb53e8b 100644 --- a/Notes/02_Working_with_data/04_Sequences.md +++ b/Notes/02_Working_with_data/04_Sequences.md @@ -1,10 +1,8 @@ # 2.4 Sequences -In this part, we look at some common idioms for working with sequence data. +### Sequence Datatypes -### Introduction - -Python has three *sequences* datatypes. +Python has three *sequence* datatypes. * String: `'Hello'`. A string is considered a sequence of characters. * List: `[1, 4, 5]`. @@ -266,7 +264,7 @@ d = dict(zip(columns, values)) ## Exercises -### (a) Counting +### Exercise 2.13: Counting Try some basic counting examples: @@ -286,7 +284,7 @@ Try some basic counting examples: >>> ``` -### (b) More sequence operations +### Exercise 2.14: More sequence operations Interactively experiment with some of the sequence reduction operations. @@ -335,13 +333,17 @@ emerged from the depths of a rusty C program. >>> ``` -Don’t do that! Not only does reading it make everyone’s eyes bleed, it’s inefficient with memory and it runs a lot slower. -Just use a normal `for` loop if you want to iterate over data. Use `enumerate()` if you happen to need the index for some reason. +Don’t do that! Not only does reading it make everyone’s eyes bleed, +it’s inefficient with memory and it runs a lot slower. Just use a +normal `for` loop if you want to iterate over data. Use `enumerate()` +if you happen to need the index for some reason. -### (c) A practical `enumerate()` example +### Exercise 2.15: A practical `enumerate()` example -Recall that the file `Data/missing.csv` contains data for a stock portfolio, but has some rows with missing data. -Using `enumerate()` modify your `pcost.py` program so that it prints a line number with the warning message when it encounters bad input. +Recall that the file `Data/missing.csv` contains data for a stock +portfolio, but has some rows with missing data. Using `enumerate()` +modify your `pcost.py` program so that it prints a line number with +the warning message when it encounters bad input. ```python >>> cost = portfolio_cost('Data/missing.csv') @@ -361,9 +363,10 @@ for rowno, row in enumerate(rows, start=1): print(f'Row {rowno}: Bad row: {row}') ``` -### (d) Using the `zip()` function +### Exercise 2.16: Using the `zip()` function -In the file `portfolio.csv`, the first line contains column headers. In all previous code, we’ve been discarding them. +In the file `portfolio.csv`, the first line contains column +headers. In all previous code, we’ve been discarding them. ```pycon >>> f = open('Data/portfolio.csv') @@ -374,8 +377,9 @@ In the file `portfolio.csv`, the first line contains column headers. In all prev >>> ``` -However, what if you could use the headers for something useful? This is where the `zip()` function enters the picture. -First try this to pair the file headers with a row of data: +However, what if you could use the headers for something useful? This +is where the `zip()` function enters the picture. First try this to +pair the file headers with a row of data: ```pycon >>> row = next(rows) @@ -391,7 +395,8 @@ We’ve used `list()` here to turn the result into a list so that you can see it. Normally, `zip()` creates an iterator that must be consumed by a for-loop. -This pairing is just an intermediate step to building a dictionary. Now try this: +This pairing is just an intermediate step to building a +dictionary. Now try this: ```pycon >>> record = dict(zip(headers, row)) @@ -425,7 +430,8 @@ def portfolio_cost(filename): ... ``` -Now, try your function on a completely different data file `Data/portfoliodate.csv` which looks like this: +Now, try your function on a completely different data file +`Data/portfoliodate.csv` which looks like this: ```csv name,date,time,shares,price @@ -459,7 +465,7 @@ the same technique to pick out column headers. Try running the `report.py` program on the `Data/portfoliodate.csv` file and see that it produces the same answer as before. -### (e) Inverting a dictionary +### Exercise 2.17: Inverting a dictionary A dictionary maps keys to values. For example, a dictionary of stock prices. @@ -491,7 +497,8 @@ However, what if you wanted to get a list of `(value, key)` pairs instead? >>> ``` -Why would you do this? For one, it allows you to perform certain kinds of data processing on the dictionary data. +Why would you do this? For one, it allows you to perform certain kinds +of data processing on the dictionary data. ```pycon >>> min(pricelist) @@ -535,4 +542,4 @@ Also, be aware that `zip()` stops once the shortest input sequence is exhausted. >>> ``` -[Next](05_Collections) \ No newline at end of file +[Contents](../Contents) \| [Previous (2.3 Formatting)](03_Formatting) \| [Next (2.5 Collections)](05_Collections) diff --git a/Notes/02_Working_with_data/05_Collections.md b/Notes/02_Working_with_data/05_Collections.md index 0921134..8117258 100644 --- a/Notes/02_Working_with_data/05_Collections.md +++ b/Notes/02_Working_with_data/05_Collections.md @@ -89,7 +89,7 @@ stocks loaded in the interactive mode. bash % python3 -i report.py ``` -### (a) Tabulating with Counters +### Exercise 2.18: Tabulating with Counters Suppose you wanted to tabulate the total number of shares of each stock. This is easy using `Counter` objects. Try it: @@ -157,4 +157,4 @@ This is only a small taste of what counters provide. However, if you ever find yourself needing to tabulate values, you should consider using one. -[Next](06_List_comprehension) \ No newline at end of file +[Contents](../Contents) \| [Previous (2.4 Sequences)](04_Sequences) \| [Next (2.6 List Comprehensions)](06_List_comprehensions) diff --git a/Notes/02_Working_with_data/06_List_comprehension.md b/Notes/02_Working_with_data/06_List_comprehension.md index 8143726..e93689c 100644 --- a/Notes/02_Working_with_data/06_List_comprehension.md +++ b/Notes/02_Working_with_data/06_List_comprehension.md @@ -100,7 +100,7 @@ bash % python3 -i report.py Now, at the Python interactive prompt, type statements to perform the operations described below. These operations perform various kinds of data reductions, transforms, and queries on the portfolio data. -### (a) List comprehensions +### Exercise 2.19: List comprehensions Try a few simple list comprehensions just to become familiar with the syntax. @@ -117,7 +117,7 @@ Try a few simple list comprehensions just to become familiar with the syntax. Notice how the list comprehensions are creating a new list with the data suitably transformed or filtered. -### (b) Sequence Reductions +### Exercise 2.20: Sequence Reductions Compute the total cost of the portfolio using a single Python statement. @@ -155,7 +155,7 @@ The `sum()` function is then performing a reduction across the result: With this knowledge, you are now ready to go launch a big-data startup company. -### (c) Data Queries +### Exercise 2.21: Data Queries Try the following examples of various data queries. @@ -187,7 +187,7 @@ A list of all portfolio holdings that cost more than $10000. >>> ``` -### (d) Data Extraction +### Exercise 2.22: Data Extraction Show how you could build a list of tuples `(name, shares)` where `name` and `shares` are taken from `portfolio`. @@ -240,7 +240,7 @@ Try this example that filters the `prices` dictionary down to only those names t >>> ``` -### (e) Advanced Bonus: Extracting Data From CSV Files +### Exercise 2.23: Extracting Data From CSV Files Knowing how to use various combinations of list, set, and dictionary comprehensions can be useful in various forms of data processing. Here’s an example that shows how to extract selected columns from a CSV file. @@ -313,4 +313,4 @@ extraction, and so forth. Becoming a guru master of list comprehensions can substantially reduce the time spent devising a solution. Also, don't forget about the `collections` module. -[Next](07_Objects) +[Contents](../Contents) \| [Previous (2.5 Collections)](05_Collections) \| [Next (2.7 Object Model)](07_Objects) diff --git a/Notes/02_Working_with_data/07_Objects.md b/Notes/02_Working_with_data/07_Objects.md index 28df397..9c923dd 100644 --- a/Notes/02_Working_with_data/07_Objects.md +++ b/Notes/02_Working_with_data/07_Objects.md @@ -215,7 +215,7 @@ except items[2]: # ValueError In this set of exercises, we look at some of the power that comes from first-class objects. -### (a) First-class Data +### Exercise 2.24: First-class Data In the file `Data/portfolio.csv`, we read data organized as columns that look like this: @@ -226,7 +226,8 @@ name,shares,price ... ``` -In previous code, we used the `csv` module to read the file, but still had to perform manual type conversions. For example: +In previous code, we used the `csv` module to read the file, but still +had to perform manual type conversions. For example: ```python for row in rows: @@ -235,9 +236,11 @@ for row in rows: price = float(row[2]) ``` -This kind of conversion can also be performed in a more clever manner using some list basic operations. +This kind of conversion can also be performed in a more clever manner +using some list basic operations. -Make a Python list that contains the names of the conversion functions you would use to convert each column into the appropriate type: +Make a Python list that contains the names of the conversion functions +you would use to convert each column into the appropriate type: ```pycon >>> types = [str, int, float] @@ -347,7 +350,7 @@ The above code can be compressed into a single list comprehension. >>> ``` -### (b) Making dictionaries +### Exercise 2.25: Making dictionaries Remember how the `dict()` function can easily make a dictionary if you have a sequence of key names and values? Let’s make a dictionary from the column headers: @@ -370,7 +373,7 @@ Of course, if you’re up on your list-comprehension fu, you can do the whole co >>> ``` -### (c) The Big Picture +### Exercise 2.26: The Big Picture Using the techniques in this exercise, you could write statements that easily convert fields from just about any column-oriented datafile into a Python dictionary. @@ -406,3 +409,5 @@ Let’s convert the fields using a similar trick: ``` Spend some time to ponder what you’ve done in this exercise. We’ll revisit these ideas a little later. + +[Contents](../Contents) \| [Previous (2.6 List Comprehensions)](06_List_comprehensions) \| [Next (3 Program Organization)](../03_Program_organization/00_Cotents)