246 lines
6.3 KiB
Markdown
246 lines
6.3 KiB
Markdown
[Contents](../Contents.md) \| [Previous (1.5 Lists)](05_Lists.md) \| [Next (1.7 Functions)](07_Functions.md)
|
||
|
||
# 1.6 File Management
|
||
|
||
Most programs need to read input from somewhere. This section discusses file access.
|
||
|
||
### File Input and Output
|
||
|
||
Open a file.
|
||
|
||
```python
|
||
f = open('foo.txt', 'rt') # Open for reading (text)
|
||
g = open('bar.txt', 'wt') # Open for writing (text)
|
||
```
|
||
|
||
Read all of the data.
|
||
|
||
```python
|
||
data = f.read()
|
||
|
||
# Read only up to 'maxbytes' bytes
|
||
data = f.read([maxbytes])
|
||
```
|
||
|
||
Write some text.
|
||
|
||
```python
|
||
g.write('some text')
|
||
```
|
||
|
||
Close when you are done.
|
||
|
||
```python
|
||
f.close()
|
||
g.close()
|
||
```
|
||
|
||
Files should be properly closed and it's an easy step to forget.
|
||
Thus, the preferred approach is to use the `with` statement like this.
|
||
|
||
```python
|
||
with open(filename, 'rt') as file:
|
||
# Use the file `file`
|
||
...
|
||
# No need to close explicitly
|
||
...statements
|
||
```
|
||
|
||
This automatically closes the file when control leaves the indented code block.
|
||
|
||
### Common Idioms for Reading File Data
|
||
|
||
Read an entire file all at once as a string.
|
||
|
||
```python
|
||
with open('foo.txt', 'rt') as file:
|
||
data = file.read()
|
||
# `data` is a string with all the text in `foo.txt`
|
||
```
|
||
|
||
Read a file line-by-line by iterating.
|
||
|
||
```python
|
||
with open(filename, 'rt') as file:
|
||
for line in file:
|
||
# Process the line
|
||
```
|
||
|
||
### Common Idioms for Writing to a File
|
||
|
||
Write string data.
|
||
|
||
```python
|
||
with open('outfile', 'wt') as out:
|
||
out.write('Hello World\n')
|
||
...
|
||
```
|
||
|
||
Redirect the print function.
|
||
|
||
```python
|
||
with open('outfile', 'wt') as out:
|
||
print('Hello World', file=out)
|
||
...
|
||
```
|
||
|
||
## Exercises
|
||
|
||
These exercises depend on a file `Data/portfolio.csv`. The file
|
||
contains a list of lines with information on a portfolio of stocks.
|
||
It is assumed that you are working in the `practical-python/Work/`
|
||
directory. If you're not sure, you can find out where Python thinks
|
||
it's running by doing this:
|
||
|
||
```python
|
||
>>> import os
|
||
>>> os.getcwd()
|
||
'/Users/beazley/Desktop/practical-python/Work' # Output vary
|
||
>>>
|
||
```
|
||
|
||
### Exercise 1.26: File Preliminaries
|
||
|
||
First, try reading the entire file all at once as a big string:
|
||
|
||
```python
|
||
>>> with open('Data/portfolio.csv', 'rt') as f:
|
||
data = f.read()
|
||
|
||
>>> data
|
||
'name,shares,price\n"AA",100,32.20\n"IBM",50,91.10\n"CAT",150,83.44\n"MSFT",200,51.23\n"GE",95,40.37\n"MSFT",50,65.10\n"IBM",100,70.44\n'
|
||
>>> print(data)
|
||
name,shares,price
|
||
"AA",100,32.20
|
||
"IBM",50,91.10
|
||
"CAT",150,83.44
|
||
"MSFT",200,51.23
|
||
"GE",95,40.37
|
||
"MSFT",50,65.10
|
||
"IBM",100,70.44
|
||
>>>
|
||
```
|
||
|
||
In the above example, it should be noted that Python has two modes of
|
||
output. In the first mode where you type `data` at the prompt, Python
|
||
shows you the raw string representation including quotes and escape
|
||
codes. When you type `print(data)`, you get the actual formatted
|
||
output of the string.
|
||
|
||
Although reading a file all at once is simple, it is often not the
|
||
most appropriate way to do it—especially if the file happens to be
|
||
huge or if contains lines of text that you want to handle one at a
|
||
time.
|
||
|
||
To read a file line-by-line, use a for-loop like this:
|
||
|
||
```python
|
||
>>> with open('Data/portfolio.csv', 'rt') as f:
|
||
for line in f:
|
||
print(line, end='')
|
||
|
||
name,shares,price
|
||
"AA",100,32.20
|
||
"IBM",50,91.10
|
||
...
|
||
>>>
|
||
```
|
||
|
||
When you use this code as shown, lines are read until the end of the
|
||
file is reached at which point the loop stops.
|
||
|
||
On certain occasions, you might want to manually read or skip a
|
||
*single* line of text (e.g., perhaps you want to skip the first line
|
||
of column headers).
|
||
|
||
```python
|
||
>>> f = open('Data/portfolio.csv', 'rt')
|
||
>>> headers = next(f)
|
||
>>> headers
|
||
'name,shares,price\n'
|
||
>>> for line in f:
|
||
print(line, end='')
|
||
|
||
"AA",100,32.20
|
||
"IBM",50,91.10
|
||
...
|
||
>>> f.close()
|
||
>>>
|
||
```
|
||
|
||
`next()` returns the next line of text in the file. If you were to call it repeatedly, you would get successive lines.
|
||
However, just so you know, the `for` loop already uses `next()` to obtain its data.
|
||
Thus, you normally wouldn’t call it directly unless you’re trying to explicitly skip or read a single line as shown.
|
||
|
||
Once you’re reading lines of a file, you can start to perform more processing such as splitting.
|
||
For example, try this:
|
||
|
||
```python
|
||
>>> f = open('Data/portfolio.csv', 'rt')
|
||
>>> headers = next(f).split(',')
|
||
>>> headers
|
||
['name', 'shares', 'price\n']
|
||
>>> for line in f:
|
||
row = line.split(',')
|
||
print(row)
|
||
|
||
['"AA"', '100', '32.20\n']
|
||
['"IBM"', '50', '91.10\n']
|
||
...
|
||
>>> f.close()
|
||
```
|
||
|
||
*Note: In these examples, `f.close()` is being called explicitly because the `with` statement isn’t being used.*
|
||
|
||
### Exercise 1.27: Reading a data file
|
||
|
||
Now that you know how to read a file, let’s write a program to perform a simple calculation.
|
||
|
||
The columns in `portfolio.csv` correspond to the stock name, number of
|
||
shares, and purchase price of a single stock holding. Write a program called
|
||
`pcost.py` that opens this file, reads all lines, and calculates how
|
||
much it cost to purchase all of the shares in the portfolio.
|
||
|
||
*Hint: to convert a string to an integer, use `int(s)`. To convert a string to a floating point, use `float(s)`.*
|
||
|
||
Your program should print output such as the following:
|
||
|
||
```bash
|
||
Total cost 44671.15
|
||
```
|
||
|
||
### Exercise 1.28: Other kinds of "files"
|
||
|
||
What if you wanted to read a non-text file such as a gzip-compressed
|
||
datafile? The builtin `open()` function won’t help you here, but
|
||
Python has a library module `gzip` that can read gzip compressed
|
||
files.
|
||
|
||
Try it:
|
||
|
||
```python
|
||
>>> import gzip
|
||
>>> with gzip.open('Data/portfolio.csv.gz') as f:
|
||
for line in f:
|
||
print(line, end='')
|
||
|
||
... look at the output ...
|
||
>>>
|
||
```
|
||
|
||
### Commentary: Shouldn't we being using Pandas for this?
|
||
|
||
Data scientists are quick to point out that libraries like
|
||
[Pandas](https://pandas.pydata.org) already have a function for
|
||
reading CSV files. This is true--and it works pretty well.
|
||
However, this is not a course on learning Pandas. Reading files
|
||
is a more general problem than the specifics of CSV files.
|
||
The main reason we're working with a CSV file is that it's a
|
||
familiar format to most coders and it's relatively easy to work with
|
||
directly--illustrating many Python features in the process.
|
||
So, by all means use Pandas when you go back to work. For the
|
||
rest of this course however, we're going to stick with standard
|
||
Python functionality.
|
||
|
||
[Contents](../Contents.md) \| [Previous (1.5 Lists)](05_Lists.md) \| [Next (1.7 Functions)](07_Functions.md)
|