unicode datas of a dataframe to strings

emilio.molina picture emilio.molina · Feb 23, 2017 · Viewed 7.2k times · Source

I have some troubles with a dataframe obtained from reading a xls file. Every data on such dataframe has the type 'unicode' and I can't do anything with this. I wanna change it to str values. Also, iff possible, I'd like to know the reason of this fact. I heard something about 'external data', and I know that both columns and index also present the 'u' of unicode before the names of these ones. I don't know neither almost anything about encoding and I would be really grateful if someone explains something about this in addition.

I'm using Python 2 and I tryed to solve it column by column with functions as

.astype(str) 
.astype(basestring)
.apply(str) 

and

.str.decode('iso-8859-1').str.encode('utf-8') 

(I read this last one here and I just wrote it in my code to try another thing). I also tried

unicodedata.normalize('NFKD', df_bolsa[l]).encode('ascii','ignore')

but this last one cannot be used with a series. I hope someone to be able to help me to clarify this matter. Thank you very much in advance!!

Answer

emilio.molina picture emilio.molina · Apr 4, 2017

You can use the following code.

for column in df:
    df[column] = df_peru[column].str.encode('utf-8')