Find boolean mask by pattern

Question

I have array:

arr = np.array([1,2,3,2,3,4,3,2,1,2,3,1,2,3,2,2,3,4,2,1])
print (arr)
[1 2 3 2 3 4 3 2 1 2 3 1 2 3 2 2 3 4 2 1]

I would like find this pattern and return booelan mask:

pat = [1,2,3]
N = len(pat)

I use strides:

#https://stackoverflow.com/q/7100242/2901002
def rolling_window(a, window):
    shape = a.shape[:-1] + (a.shape[-1] - window + 1, window)
    strides = a.strides + (a.strides[-1],)
    c = np.lib.stride_tricks.as_strided(a, shape=shape, strides=strides)
    return c
print (rolling_window(arr, N))
[[1 2 3]
 [2 3 2]
 [3 2 3]
 [2 3 4]
 [3 4 3]
 [4 3 2]
 [3 2 1]
 [2 1 2]
 [1 2 3]
 [2 3 1]
 [3 1 2]
 [1 2 3]
 [2 3 2]
 [3 2 2]
 [2 2 3]
 [2 3 4]
 [3 4 2]
 [4 2 1]]

I find positions of first values only:

b = np.all(rolling_window(arr, N) == pat, axis=1)
c = np.mgrid[0:len(b)][b]
print (c)
[ 0  8 11]

And positions another vals:

d = [i  for x in c for i in range(x, x+N)]
print (d)
[0, 1, 2, 8, 9, 10, 11, 12, 13]

Last return mask by in1d:

e = np.in1d(np.arange(len(arr)), d)
print (e)
[ True  True  True False False False False False  True  True  
  True  True  True  True False False False False False False]

Verify mask:

print (np.vstack((arr, e))) 
[[1 2 3 2 3 4 3 2 1 2 3 1 2 3 2 2 3 4 2 1]
 [1 1 1 0 0 0 0 0 1 1 1 1 1 1 0 0 0 0 0 0]]
  1 2 3           1 2 3 1 2 3

I think my solution is a bit over-complicated. Is there some better, more pythonic solution?

have you seen [this](https://stackoverflow.com/questions/7100242/python-numpy-first-occurrence-of-subarray)? They seem to be doing the same thing though. — Ma0, Feb 26 '18 at 12:09
Can you [Boyer--Moore](https://en.wikipedia.org/wiki/Boyer%E2%80%93Moore_string_search_algorithm)-search it? — Cong Ma, Feb 26 '18 at 12:12
@Divakar - I think `efficiency` should be nice, but `shorted code` is not problem also. — jezrael, Feb 26 '18 at 12:24
Shouldn't the output be of length 18, because rolling_window would have 18 windows? — Divakar, Feb 26 '18 at 12:29
@Divakar - unfortuantely not, I need mask with same length as `arr` — jezrael, Feb 26 '18 at 12:30

Divakar · Accepted Answer · 2018-02-26T13:16:10.283

3

We can simplify things at the end with Scipy supported binary-dilation -

from scipy.ndimage.morphology import binary_dilation

m = (rolling_window(arr, len(pat)) == pat).all(1)
m_ext = np.r_[m,np.zeros(len(arr) - len(m), dtype=bool)]
out = binary_dilation(m_ext, structure=[1]*N, origin=-(N//2))

For performance, we can bring in OpenCV with its template matching capability, as we are basically doing the same here, like so -

import cv2

tol = 1e-5
pat_arr = np.asarray(pat, dtype='uint8')
m = (cv2.matchTemplate(arr.astype('uint8'),pat_arr,cv2.TM_SQDIFF) < tol).ravel()

edited Feb 26 '18 at 13:16

answered Feb 26 '18 at 12:47

Divakar

218,885
19
262
358

Thank you for help, but second solution failed for me, can you check what should be problem in sample from this [answer](https://stackoverflow.com/a/49005205/2901002) ? – jezrael Feb 27 '18 at 09:33

score 1 · Answer 2 · answered Feb 27 '18 at 07:03

Not sure how safe this is, but another method would be to read back to an as_strided view of the boolean output. As long as you only have one pat at a time it shouldn't be a problem I think, and it may work with more but I can't gurantee it because reading back to as_strided can be a bit unpredictable:

def vview(a):  #based on @jaime's answer: https://stackoverflow.com/a/16973510/4427777
    return np.ascontiguousarray(a).view(np.dtype((np.void, a.dtype.itemsize * a.shape[1])))

def roll_mask(arr, pat):
    pat = np.atleast_2d(pat)
    out = np.zeros_like(arr).astype(bool)
    vout = rolling_window(out, pat.shape[-1])
    vout[np.in1d(vview(rolling_window(arr, pat.shape[-1])), vview(pat))] = True
    return out

np.where(roll_mask(arr, pat))
(array([ 0,  1,  2,  8,  9, 10, 11, 12, 13], dtype=int32),)

pat = np.array([[1, 2, 3], [3, 2, 3]])
print([i for i in arr[roll_mask(arr, pat)]])
[1, 2, 3, 2, 3, 1, 2, 3, 1, 2, 3]

It seems to work, but I wouldn't give this answer to a beginner!

Find boolean mask by pattern

2 Answers2

Linked