Adding records to a numpy record array

Question

Let's say I define a record array

>>> y=np.zeros(4,dtype=('a4,int32,float64'))

and then I proceed to fill up the 4 records available. Now I get more data, something like

>>> c=('a',7,'24.5')

and I want to add this record to y. I can't figure out a clean way to do it. The best I have seen in np.concatenate(), but that would require turning c into an record array in and of itself. Is there any simple way to tack my tuple c onto y? This seems like it should be really straightforward and widely documented. Apologies if it is. I haven't been able to find it.

Answer 1

You can use numpy.append(), but as you need to convert the new data into a record array also:

import numpy as np
y = np.zeros(4,dtype=('a4,int32,float64'))
y = np.append(y, np.array([("0",7,24.5)], dtype=y.dtype))

Since ndarray can't dynamic change it's size, you need to copy all the data when you want to append some new data. You can create a class that reduce the resize frequency:

import numpy as np

class DynamicRecArray(object):
    def __init__(self, dtype):
        self.dtype = np.dtype(dtype)
        self.length = 0
        self.size = 10
        self._data = np.empty(self.size, dtype=self.dtype)

    def __len__(self):
        return self.length

    def append(self, rec):
        if self.length == self.size:
            self.size = int(1.5*self.size)
            self._data = np.resize(self._data, self.size)
        self._data[self.length] = rec
        self.length += 1

    def extend(self, recs):
        for rec in recs:
            self.append(rec)

    @property
    def data(self):
        return self._data[:self.length]

y = DynamicRecArray(('a4,int32,float64'))
y.extend([("xyz", 12, 3.2), ("abc", 100, 0.2)])
y.append(("123", 1000, 0))
print y.data
for i in xrange(100):
    y.append((str(i), i, i+0.1))