How to iterate through a pygame 3d surfarray and change the individual color of the pixels if they're less than a specific value?

Question

I'm trying to iterate a pygame 3d surfarray, more specifically pygame.surfarray.array3d("your image"). I'm receiving video captured from my webcam then converting them into a 3d array, then displaying it onto my window with this code.

def cameraSee(self):
    while True:
        self.cam.query_image()
        self.image = self.cam.get_image()
        self.imageArr = pygame.surfarray.array3d(self.image)

        pygame.surfarray.blit_array(self.screen,self.imageArr)

        os.system("clear")

        pygame.display.update()

        for event in pygame.event.get():
            if event.type == pygame.QUIT:
                sys.exit()

My problem is that I'm trying to have my camera only display any pixel which has an amount of blue > 200 (ranging from 0 - 255) and change the color value of all other pixels to 0. I've tried using an if statement for an array but I get an error stating that I should be using the any() or all().

all my code:

try:
    import pygame
    import pygame.camera
    import pygame.surfarray
    import numpy
    import os
    import sys
    import time
except:
    print("there was an error importing modules...")

os.system("espeak 'there, was, an, error, importing, modules'")
time.sleep(2)

class aaiVision(object):
    def __init__(self,screen,cam,image,imageArr):
        self.screen = screen
        self.cam = cam
        self.image = image
        self.imageArr = imageArr

    def startUp(self):
        os.system("espeak 'eh, eh, i, vision, initializing'")
        pygame.init()
        pygame.camera.init()
        time.sleep(1)
        os.system("espeak 'Vision, initialized'")

        camList = pygame.camera.list_cameras()
        print(camList)
        time.sleep(1)
        os.system("espeak 'cameras, found, %s'" % str(len(camList)))
        time.sleep(1)

        self.screen = pygame.display.set_mode((640,480))
        time.sleep(0.5)
        self.cam = pygame.camera.Camera("/dev/video0",(640,480),"YUV")
        time.sleep(0.5)
        self.cam.start()
        os.system("espeak 'eh, eh, i, vision, online'")

    def cameraSee(self):
        while True:
            self.cam.query_image()
            self.image = self.cam.get_image()
            self.imageArr = pygame.surfarray.array3d(self.image)



            pygame.surfarray.blit_array(self.screen,self.imageArr)

            os.system("clear")

            pygame.display.update()

            for event in pygame.event.get():
                if event.type == pygame.QUIT:
                    sys.exit()

  eyesAwake = aaiVision('', '', '', '')

  if __name__ == "__main__":
      eyesAwake.startUp()
      eyesAwake.cameraSee()

Sorry about some of the indentation errors, I'm not sure how to use the in text code blocks XD

Where do you get an error stating that I should be using the `any()` or `all()`? Please [edit] your question and include the full traceback leading up to where this occurs. You may be having trouble properly indenting your code because you're using tabs instead of space to do it. — martineau, Aug 14 '17 at 22:07

score 2 · Accepted Answer · answered Aug 16 '17 at 04:44

Rather than iterating over the pixels in a list comprehension (which is bound to be slow in python) and then converting that list result into the desired numpy array, we can "vectorize" the problem using the magic of numpy. This is convenient here, since pygame.surfarray.array3d already returns a numpy array!

Here is a possible solution that takes an image (loaded from disk; I could not get your code to work since it relies on some linux directories like /dev/video0 for the webcam input and the espeak command, which is unavailable in Windows):

import numpy as np
import pygame
import sys

if __name__ == "__main__":
    pygame.init()
    screen = pygame.display.set_mode((800, 600))
    img = pygame.image.load("test.jpg").convert_alpha()
    clock = pygame.time.Clock()

    while True:
        for event in pygame.event.get():
            if event.type == pygame.QUIT:
                pygame.quit()
                sys.exit()

        data = pygame.surfarray.array3d(img)
        mask = data[..., 2] < 200 #mark all super-NOT-blue pixels as True (select the OPPOSITE!)
        masked = data.copy() #make a copy of the original image data
        masked[mask] = [0, 0, 0] #set any of the True pixels to black (rgb: 0, 0, 0)
        out = pygame.surfarray.make_surface(masked) #convert the numpy array back to a surface
        screen.blit(out, (0, 0))
        pygame.display.update()
        print clock.get_fps()
        clock.tick(60)

On my computer, this runs at ~30 fps, which, while not great, should be a significant improvement! The slowness here appears to be due to masked[mask] = [0, 0, 0] in particular. If any numpy experts could chime in, that would be terrific! Otherwise, I can chime in with an additional cython answer instead (where adding type data should significantly improve loop performance).

score 0 · Answer 2 · answered Aug 14 '17 at 23:19

I'm not sure if it works, but please try it

outArr = np.array( [[w if w[2] > 200 else np.zeros(3) for w in h] for h in self.imageArr], dtype='uint8')

The idea is that the three dimensional array has three indices, the first describing the position's height, the second its width and the last the color of a pixel. So a list comprehension of the color lists is made where the third entry of each, corresponding to blue, is compared if its greater than 200. If this is not the case, all color values are reset.

How to iterate through a pygame 3d surfarray and change the individual color of the pixels if they're less than a specific value?

2 Answers2

Linked