Pandas Convert 'NA' to NaN

python pandas bioinformatics
ericmjl picture ericmjl · May 16, 2013 · Viewed 8.4k times · Source

I just picked up Pandas to do with some data analysis work in my biology research. Turns out one of the proteins I'm analyzing is called 'NA'.

I have a matrix with pairwise 'HA, M1, M2, NA, NP...' on the column headers, and the same as "row headers" (for the biologists who might read this, I'm working with influenza).

When I import the data into Pandas directly from a CSV file, it reads the "row headers" as 'HA, M1, M2...' and then NA gets read as NaN. Is there any way to stop this? The column headers are fine - 'HA, M1, M2, NA, NP etc...'

Answer

Dan Allan picture Dan Allan · May 16, 2013

Turn off NaN detection this way: pd.read_csv(filename, keep_default_na=False)

I originally suggested na_filter=False, which gets the job done. But, if I understand Jeff's comments below, this is a cleaner solution.

Example:

In [1]: pd.read_csv('test')
Out[1]:[4]: pd.read_csv('test', keep_default_na=False)
Out[4]:1   2
2   3

Related questions

Pandas: .groupby().size() and percentages

I have a DataFrame that originates from a df.groupby().size() operation, and looks like this: Localization RNA level cytoplasm 1 Non-expressed 7 2 Very low 13 3 Low 8 4 Medium 6 5 Moderate 8 6 High 2 7 Very high 6 cytoplasm & nucleus 1 Non-expressed 5 2 Very low 8 3 Low 2 4 Medium 10 5 Moderate 16 6 High 6 7 Very …

Read More
How to select rows from a DataFrame based on column values

How can I select rows from a DataFrame based on values in some column in Pandas? In SQL, I would use: SELECT * FROM table WHERE colume_name = some_value I tried to look at Pandas' documentation, but I did not …

Read More
Renaming columns in pandas

I have a DataFrame using pandas and column labels that I need to edit to replace the original column labels. I'd like to change the column names in a DataFrame A where the original column names are: ['$a', '$…

Read More