3

I have the masked array (toy example) - [1 2 3 4 -- 6]

points = [1, 2, 3, 4, 0, 6]
mask = [0, 0, 0, 0, 1, 0]
points = ma.array(points, mask=mask)

And I would like to interpolate it from 6 dimensions to any number, for example, 6. My interpolation criteria is that it will ignore masked values, and skip that index.

Undesired behaviour, for example with lin = np.linspace(0, 1, 6):

f = interp1d(lin, points, axis=0, kind='cubic')
f(lin) # [1  2 3 4 -8.8817842e-16 6]

Instead, I'm expecting it to behave like:

compressed_lin = [0, 0.2, 0.4, 0.6, 1]
compressed_points = np.array([1,2,3,4,6])
f = interp1d(compressed_lin, compressed_points, axis=0, kind='cubic')
f(lin) # [1 2 3 4 5 6]

Actual Data:

My data is in the shape of [100, 100, 2], so its not as simple as to hide the masked values from a linspace of size 100.

Note on the masked array:

I already know there is a way to do a 2d interpolation on a masked array (https://modelhelptokyo.wordpress.com/2017/10/25/how-to-interpolate-missing-values-2d-python/, Scipy interpolation with masked data?, Scipy interp2d interpolate masked fill values)

If there was a way to interpolate all of the values in the masked array to be filled, rather than a constant fill, that will solve the problem, as then I'll be dealing with a filled numpy array.

The reason I can't use compressed is it can change the order of indexes and the size of the array, e.g:

points = np.array([1, 2, 3, 4, 5, 6])

points = ma.stack([
    ma.array(points, mask=[0, 0, 0, 0, 1, 0]),
    ma.array(points, mask=[0, 0, 1, 0, 0, 0]),
])

print(points)
# [[1 2 3 4 -- 6]
# [1 2 -- 4 5 6]]

print(np.reshape(points.compressed(), (2, 5)))
# [[1 2 3 4 6]
# [1 2 4 5 6]]
Amit
  • 5,118
  • 6
  • 39
  • 82
  • 2
    You already saw my explanation on how `np.ma` works. Only `np.ma` functions and methods know about the `mask` and know when/how to apply `filled` and `compressed`. Anything else uses the `data` attribute, without modification. https://stackoverflow.com/questions/60685061/numpy-faster-operations-on-masked-array/60685339#60685339 – hpaulj Mar 20 '20 at 00:48
  • Thanks @hpaulj your answer was indeed very helpful and I implemented many functions relying on it. However, I couldn't find a way to use it for this case. I added a note on the question for the reasons I can't use `filled` or `compressed`. I am aware that `interp1d` only uses the `data` itself, but I'm trying to figure out how to (efficiently) make this work anyway. – Amit Mar 20 '20 at 08:31
  • You say that your data is greater than 1 dimension, so which dimension should be interpolated? The dummy problem you provide is qualitatively different from your actual problem. Interpolation on different dimensions could lead to different interpolated values. – jpf Mar 29 '20 at 19:23

0 Answers0