H5py store list of list of strings

python hdf5 h5py
PKuhn picture PKuhn · Jun 17, 2016 · Viewed 7.2k times · Source

Is there a possibility in h5py to create a dataset which consists of lists of strings. I tried to create a nested datatype of variable length, but this results in segmentation fault in my python interpreter. 

def create_dataset(h5py_file):
    data = [['I', 'am', 'a', 'sentecne'], ['another', 'sentence']]
    string_dt = h5py.special_dtype(vlen=str)
    nested_dt = h5py.special_dtype(vlen=string_dt)
    h5py_file.create_dataset("sentences", data=data, dtype = nested_dt)

Answer

jan-glx picture jan-glx · Feb 1, 2018

If you don't intend to edit the hdf5 file (and potentially use longer strings), you can also simply use:

h5py_file.create_dataset("sentences", data=np.array(data, dtype='S'))

Related questions

Read HDF5 file into numpy array

I have the following code to read a hdf5 file as a numpy array: hf = h5py.File('path/to/file', 'r') n1 = hf.get('dataset_name') n2 = np.array(n1) and when I print n2 I get this: Out[15]: …

Read More
HDF5 file created with h5py can't be opened by h5py

I created an HDF5 file apparently without any problems, under Ubuntu 12.04 (32bit version), using Anaconda as Python distribution and writing in ipython notebooks. The underlying data are all numpy arrays. For example, import numpy as np import h5py f = …

Read More
How to append data to one specific dataset in a hdf5 file with h5py

I am looking for a possibility to append data to an existing dataset inside a .h5 file using Python (h5py). A short intro to my project: I try to train a CNN using medical image data. Because of the …

Read More