I don't know about OpenCV, so I can't tell you what cv.GetMat() does.  Apparently, it returns something that can be used as or converted to a two-dimensional array.  The C or C++ interface to OpenCV that you are using will probably have a similarly names function.
The following lines create an array of index pairs of the entries in grey_image_as_array that are bigger than 3.  Each entry in non_black_coords_array are zero based x-y-coordinates into grey_image_as_array.  Given such a coordinates pair x, y, you can access the corresponsing entry in the two-dimensional C++ array grey_image_as_array with grey_image_as_array[y][x].
The Python code has to avoid explicit loops over the image to achieve good performance, so it needs to make to with the vectorised functions NumPy offers.  The expression grey_image_as_array > 3 is a vectorised comparison and results in a Boolean array of the same shape as grey_image_as_array.  Next, numpy.where() extracts the indices of the True entries in this Boolean array, but the result is not in the format described above, so we need zip() to restructure it.
In C++, there's no need to avoid explicit loops, and an equivalent of numpy.where() would be rather pointless -- you just write the loops  and store the result in the format of your choice.