Concatenate Numpy arrays without copying

Fred Foo picture Fred Foo · Oct 23, 2011 · Viewed 38.8k times · Source

In Numpy, I can concatenate two arrays end-to-end with np.append or np.concatenate:

>>> X = np.array([[1,2,3]])
>>> Y = np.array([[-1,-2,-3],[4,5,6]])
>>> Z = np.append(X, Y, axis=0)
>>> Z
array([[ 1,  2,  3],
       [-1, -2, -3],
       [ 4,  5,  6]])

But these make copies of their input arrays:

>>> Z[0,:] = 0
>>> Z
array([[ 0,  0,  0],
       [-1, -2, -3],
       [ 4,  5,  6]])
>>> X
array([[1, 2, 3]])

Is there a way to concatenate two arrays into a view, i.e. without copying? Would that require an np.ndarray subclass?

Answer

pv. picture pv. · Oct 23, 2011

The memory belonging to a Numpy array must be contiguous. If you allocated the arrays separately, they are randomly scattered in memory, and there is no way to represent them as a view Numpy array.

If you know beforehand how many arrays you need, you can instead start with one big array that you allocate beforehand, and have each of the small arrays be a view to the big array (e.g. obtained by slicing).