Python for non programmers, part 3

Today, we will look at data format errors. Replace your old input file with the following file and re-run your program:

Row A,Row B,Row C
Not a number,a,2
Missing a value,1

After newline,3,4

The first time you run the program, you will see an error message:

Traceback (most recent call last):
  File "csvcopy.py", line 17, in 
    row.append(float(row[1]) + float(row[2]))
ValueError: invalid literal for float(): a

If you deleted the line that causes the issue and reran the program, you would see

Traceback (most recent call last):
  File "csvcopy.py", line 17, in 
    row.append(float(row[1]) + float(row[2]))
IndexError: list index out of range

Would you also delete that line, you would get the same error also for the blank line in your file. How do we fix this?

Replace your current program without the following code:

# Import libraries                                                                                           
import csv

# Open files and csv readers/writers                                                                         
infile = open('data.txt', 'rb')
in_csv = csv.reader(infile, delimiter=',')
outfile = open('data2.txt', 'wb')
out_csv = csv.writer(outfile, delimiter=',')

# Go through the rows                                                                                        
is_header = True
row_number = 0
for row in in_csv:
  row_number = row_number + 1
  is_valid = True
  if is_header:
    row = row + ['Sum', 'Max', 'Description']
    is_header = False
  else:
    try:
      row.append(float(row[1]) + float(row[2]))
      row.append(max(int(row[1]), int(row[2])))
      row.append('Value pair of %s and %s' % (row[1], row[2]))
    except(ValueError, IndexError):
      print 'Row %s is invalid, skipping...' % row_number
      is_valid = False
  if is_valid:
    out_csv.writerow(row)

This code will use a try/except statement to intercept any errors in parsing numbers (ValueError) or rows with insufficient amount of columns (IndexError) and not write them into the file. Your mission, should you choose toy accept: extend the program to collect all these incorrect lines in a second output-file called "invalid.txt". Try doing so by adding 6 lines or fewer.