Writing a .CSV file in Python that works for both Python 2.7+ and Python 3.3+ in Windows

Tamerz picture Tamerz · Apr 24, 2015 · Viewed 21.3k times · Source

EDIT: I put it in the title, but just realized I didn't mention it in the body. This seems to be specific to Windows.

I'm having a hard time writing output using the csv Python module in a script that works with both Python 2.7 and 3.3.

First try which works as expected in Python 2.7:

with open('test.csv', 'wb') as csv_file:
    writer = csv.DictWriter(csv_file, ['header1', 'header2'])
    writer.writeheader()
    for item in items:
        writer.writerow(item)

However, when that same thing is run in Python 3.3 you wind up with:

TypeError: 'str' does not support the buffer interface

So I change 'wb' to 'wt' and it runs, but now I have an extra blank row every other line in the file.

To fix that, I change:

with open('test.csv', 'wt') as csv_file:

to:

with open('test.csv', 'wt', newline='') as csv_file:

But now, it breaks Python 2.7:

TypeError: 'newline' is an invalid keyword argument for this function

I know I could just do something like:

try:
    with open('test.csv', 'wt', newline='') as csv_file:
        writer = csv.DictWriter(csv_file, ['header1', 'header2'])
        writer.writeheader()
        for item in items:
            writer.writerow(item)
except TypeError:
    with open('test.csv', 'wb') as csv_file:
        writer = csv.DictWriter(csv_file, ['header1', 'header2'])
        writer.writeheader()
        for item in items:
            writer.writerow(item)

However, that has some seriously bad duplication.

Does anyone have a cleaner way of doing this?

EDIT: The test data is simple and has no newlines or anything:

items = [{'header1': 'value', 'header2': 'value2'},
         {'header1': 'blah1', 'header2': 'blah2'}]

Answer

skyline75489 picture skyline75489 · Apr 24, 2015

I've tried a few ways. As far as I can see, simple using 'w' could be a solution:

with open('test.csv', 'w') as csv_file:
    writer = csv.DictWriter(csv_file, fieldnames=['header1', 'header2'], lineterminator='\n')
    # write something