Customizing the separator in pandas read_csv

python pandas separator
Peaceful picture Peaceful · Dec 20, 2016 · Viewed 39.5k times · Source

I am reading many different data files into various pandas dataframes. The columns in these datafiles are separated by spaces. However, for each file, the number of spaces is different (for some of them, there is only one space, for others, there are two spaces and so on). Thus, every time I import the file, I have to manually go to that file and see the number of spaces that have been used and give those many number of spaces in sep:

import pandas as pd
df = pd.read_csv('myfile.dat', sep = '    ')

Is there any way I can tell pandas to assume "any number of spaces" as the separator? Also, is there any way I can tell pandas to use either tab (\t) or spaces as the separator?

Answer

Ted Petrou picture Ted Petrou · Dec 20, 2016

Yes, you can use a simple regular expression like sep='\s+' to denote one or more spaces.

Related questions

Convert commas decimal separators to dots within a Dataframe

I am importing a CSV file like the one below, using pandas.read_csv: df = pd.read_csv(Input, delimiter=";") Example of CSV file: 10;01.02.2015 16:58;01.02.2015 16:58;-0.59;0.1;-4.39;NotApplicable;0.79;0.2 11;01.02.2015 16:58;01.02.2015 16:58;-0.57;0.2;-2.87;NotApplicable;0.79;0.21 The problem is that when I later on in my …

Read More
Python import CSV short code (pandas?) delimited with ';' and ',' in entires

I need to import a CSV file in Python on Windows. My file is delimited by ';' and has strings with non-English symbols and commas (','). I've read posts: Importing a CSV file into a sqlite3 database table …

Read More
How to select rows from a DataFrame based on column values

How can I select rows from a DataFrame based on values in some column in Pandas? In SQL, I would use: SELECT * FROM table WHERE colume_name = some_value I tried to look at Pandas' documentation, but I did not …

Read More