In our lab we store our data in hdf5
files trough the python package h5py
.
At the beginning of an experiment we create an hdf5
file and store array after array of array of data in the file (among other things). When an experiment fails or is interrupted the file is not correctly closed.
Because our experiments run from iPython
the reference to the data object remains (somewhere) in memory.
Is there a way to scan for all open h5py data objects and close them?
This is how it could be done (I could not figure out how to check for closed-ness of the file without exceptions, maybe you will find):
import gc
for obj in gc.get_objects(): # Browse through ALL objects
if isinstance(obj, h5py.File): # Just HDF5 files
try:
obj.close()
except:
pass # Was already closed
Another idea:
Dpending how you use the files, what about using the context manager and the with
keyword like this?
with h5py.File("some_path.h5") as f:
f["data1"] = some_data
When the program flow exits the with-block, the file is closed regardless of what happens, including exceptions etc.