13

I'm simply trying to use a masked array to filter out some nanentries.

import numpy as np
# x = [nan, -0.35, nan]
x = np.ma.masked_equal(x, np.nan)
print x

This outputs the following:

masked_array(data = [        nan -0.33557216         nan],
         mask = False,
   fill_value = nan)

Calling np.isnan() on x returns the correct boolean array, but the mask just doesn't seem to work. Why would my mask not be working as I expect?

chris
  • 4,840
  • 5
  • 35
  • 66
  • 3
    You should probably use [`np.ma.masked_invalid()`](http://docs.scipy.org/doc/numpy/reference/generated/numpy.ma.masked_invalid.html#numpy.ma.masked_invalid). – Sven Marnach Jan 26 '15 at 23:26
  • that works, thanks. if you post an answer I can close this question – chris Jan 26 '15 at 23:27

2 Answers2

18

You can use np.ma.masked_invalid:

import numpy as np

x = [np.nan, 3.14, np.nan]
mx = np.ma.masked_invalid(x)

print(repr(mx))
# masked_array(data = [-- 3.14 --],
#              mask = [ True False  True],
#        fill_value = 1e+20)

Alternatively, use np.isnan(x) as the mask= parameter to np.ma.masked_array:

print(repr(np.ma.masked_array(x, np.isnan(x))))
# masked_array(data = [-- 3.14 --],
#              mask = [ True False  True],
#        fill_value = 1e+20)

Why doesn't your original approach work? Because, rather counterintuitively, NaN is not equal to NaN!

print(np.nan == np.nan)
# False

This is actually part of the IEEE-754 definition of NaN

Community
  • 1
  • 1
ali_m
  • 71,714
  • 23
  • 223
  • 298
7

Here is another alternative without using mask:

import numpy as np
#x = [nan, -0.35, nan]
xmask=x[np.logical_not(np.isnan(x))]
print(xmask)

Result:

array([-0.35])

chuseuiti
  • 783
  • 1
  • 9
  • 32
  • 2
    And for anyone wondering, `np.logical_not` is needed here instead of the built-in `not` because the latter does not broadcast. – Guimoute Jun 19 '19 at 12:53
  • `np.isnan()` returns a boolean array, and as such using `x[~np.isnan(x)]` would suffice. `np.logical_not()` is useful for non-boolean array use-cases. – ryanjdillon May 18 '22 at 10:32