How can I change the value of a masked array in numpy?

Question

In my code, at some point I try to modify a value of a masked array, yet python seems to ignore this. I'm thinking this has to do with the way memory is stored in arrays, as if I were modifying a copy of the value and not the value itself, but I'm not well versed enough in this to have any clue how to resolve it.

Here is a simplified version of what I'm trying to do :

    x = np.zeros((2,5)) # create 2D array of zeroes
    x[0][1:3] = 5       # replace some values along 1st dimension with 5

    mask = (x[0] > 0)   # create a mask to only deal with the non negative values

    x[0][mask][1] = 10  # change one of the values that is non negative 

    print x[0][mask][1] # value isn't changed in the original array

the output of this is :

    5.0

when it should be 10.

Any help would be greatly appreciated, ideally this need to be scalable (meaning I don't necessarily know the shape of x, or where the values are non-negative, or which one I will need to modify).

I'm working with numpy 1.11.0, on python 2.7.12 on Ubuntu 16.04.2

Thanks !

Answer 1

Let's generalize your problem a bit:

In [164]: x=np.zeros((2,5))
In [165]: x[0, [1, 3]] = 5      # index with a list, not a slice
In [166]: x
Out[166]: 
array([[ 0.,  5.,  0.,  5.,  0.],
       [ 0.,  0.,  0.,  0.,  0.]])

When the indexing occurs right before the =, it's part of a __setitem__ and acts on the original array. This is true whether the indexing uses slices, a list or a boolean mask.

But a selection with the list or mask produces a copy. Further indexed assignment affects only that copy, not the original.

In [167]: x[0, [1, 3]]
Out[167]: array([ 5.,  5.])
In [168]: x[0, [1, 3]][1] = 6
In [169]: x
Out[169]: 
array([[ 0.,  5.,  0.,  5.,  0.],
       [ 0.,  0.,  0.,  0.,  0.]])

The best way around this is to modify the mask itself:

In [170]: x[0, np.array([1,3])[1]] = 6
In [171]: x
Out[171]: 
array([[ 0.,  5.,  0.,  6.,  0.],
       [ 0.,  0.,  0.,  0.,  0.]])

If the mask is boolean, you may need to convert it to indexing array

In [174]: mask = x[0]>0
In [175]: mask
Out[175]: array([False,  True, False,  True, False], dtype=bool)
In [176]: idx = np.where(mask)[0]
In [177]: idx
Out[177]: array([1, 3], dtype=int32)
In [178]: x[0, idx[1]]
Out[178]: 6.0

Or you can tweak the boolean values directly

In [179]: mask[1]=False
In [180]: x[0,mask]
Out[180]: array([ 6.])

So in your big problem you need to be aware of when indexing produces a view and it is a copy. And you need to be comfortable with index with lists, arrays and booleans, and understand how to switch between them.