pd.read_hdf throws 'cannot set WRITABLE flag to True of this array'

python pandas pytables hdf
Landmaster picture Landmaster · Jan 16, 2019 · Viewed 12.1k times · Source

When running

pd.read_hdf('myfile.h5')

I get the following traceback error:

[[...some longer traceback]]

~/.local/lib/python3.6/site-packages/pandas/io/pytables.py in read_array(self, key, start, stop) 2487 2488 if isinstance(node, tables.VLArray): -> 2489 ret = node[0][start:stop] 2490 else: 2491 dtype = getattr(attrs, 'value_type', None)

~/.local/lib/python3.6/site-packages/tables/vlarray.py in getitem(self, key)

~/.local/lib/python3.6/site-packages/tables/vlarray.py in read(self, start, stop, step)

tables/hdf5extension.pyx in tables.hdf5extension.VLArray._read_array()

ValueError: cannot set WRITEABLE flag to True of this array

No clue what's going on. I've tried reinstalling tables, pandas everything basically, but doesn't want to read it.

Answer

Eddie Bell picture Eddie Bell · Jan 21, 2019

Are you using numpy 1.16? It is incompatible with the latest release of pytables (see https://github.com/PyTables/PyTables/blob/v3.4.4/tables/hdf5extension.pyx#L2155) but the pytables team have not yet released a fixed version: https://github.com/PyTables/PyTables/issues/719

The only way I found to fix this is to downgrade numpy.

Related questions

Convert large csv to hdf5

I have a 100M line csv file (actually many separate csv files) totaling 84GB. I need to convert it to a HDF5 file with a single float dataset. I used h5py in testing without any problems, but now I …

Read More
Merging two tables with millions of rows in Python

I am using Python for some data analysis. I have two tables, the first (let's call it 'A') has 10 million rows and 10 columns and the second ('B') has 73 million rows and 2 columns. They have 1 column with common ids and I …

Read More
Iteratively writing to HDF5 Stores in Pandas

Pandas has the following examples for how to store Series, DataFrames and Panelsin HDF5 files: Prepare some data: In [1142]: store = HDFStore('store.h5') In [1143]: index = date_range('1/1/2000', periods=8) In [1144]: s = Series(randn(5), index=['a', 'b', 'c', 'd', …

Read More