I have a dataframe t_unit
, which is the result of a pd.read_csv()
function.
datetime B18_LR_T B18_B1_T
24/03/2016 09:00 21.274 21.179
24/03/2016 10:00 19.987 19.868
24/03/2016 11:00 21.632 21.417
24/03/2016 12:00 26.285 24.779
24/03/2016 13:00 26.897 24.779
I am resampling the dataframe to calculate the 5th and 05th percentiles with the code:
keys_actual = list(t_unit.columns.values)
for key in keys_actual:
ts_wk = t_unit[key].resample('W-MON')
ts_wk_05p = ts_wk.apply(lambda x: x.quantile(0.05)).round(decimals=1).rename(key+'_05p', inplace=True)
ts_wk_95p = ts_wk.apply(lambda x: x.quantile(0.95)).round(decimals=1).rename(key+'_95p', inplace=True)
All works fine, but when I add a column to my dataframe, by means of pd.concat
, into:
datetime B18_LR_T B18_B1_T ext_T
24/03/2016 09:00 21.274 21.179 6.9
24/03/2016 10:00 19.987 19.868 7.5
24/03/2016 11:00 21.632 21.417 9.1
24/03/2016 12:00 26.285 24.779 9.9
24/03/2016 13:00 26.897 24.779 9.2
ts_wk_05p = ts_wk.apply(lambda x: x.quantile(0.05)).round(decimals=1).rename(key+'_05p', inplace=True)
TypeError: can't multiply sequence by non-int of type 'float'
Do you have any idea why?
There is problem some column is not numeric.
You can check dtypes
:
print (t_unit.dtypes)
B18_LR_T float64
B18_B1_T float64
ext_T object
dtype: object
Then try convert to numeric first by astype
:
t_unit.ext_T = t_unit.ext_T.astype(float)
If:
ValueError: could not convert string to float
then use to_numeric
with parameter errors='coerce'
for convert bad data to NaN
:
t_unit.ext_T = pd.to_numeric(t_unit.ext_T, errors='coerce')
All code:
#simulate string column
t_unit.ext_T = t_unit.ext_T.astype(str)
print (t_unit.dtypes)
B18_LR_T float64
B18_B1_T float64
ext_T object
dtype: object
#convert to float
t_unit.ext_T = t_unit.ext_T.astype(float)
print (t_unit)
L = []
for key in t_unit.columns:
ts_wk = t_unit[key].resample('W-MON')
#remove inplace=True
ts_wk_05p = ts_wk.apply(lambda x: x.quantile(0.05)).round(decimals=1).rename(key+'_05p')
ts_wk_95p = ts_wk.apply(lambda x: x.quantile(0.95)).round(decimals=1).rename(key+'_95p')
L.append(ts_wk_05p)
L.append(ts_wk_95p)
print (pd.concat(L, axis=1))
B18_LR_T_05p B18_LR_T_95p B18_B1_T_05p B18_B1_T_95p ext_T_05p \
datetime
2016-03-28 20.2 26.8 20.1 24.8 7.0
ext_T_95p
datetime
2016-03-28 9.8