How should I structure this code to break out of multiple nested loops at once?

Question

I'm learning image detection and python writing a bot for a 2d game. There are certain targets we want to attack and certain targets we want to avoid. One target is displayed on the screen at a time out of a pool of 100 and you click next or attack depending on what you want to do. In doing this you're able to cycle through targets until you find one you want to attack. My code currently checks to see if it finds one of 24 different targets we want to avoid, if it does it avoids that target. Otherwise, it attacks that target.

At the moment my code:

1. Loads 24 different images of things we want to avoid into a NumPy array and checks with `py.locateCenterOnScreen` 
   if we are on the correct screen to start the video loop
2. Uses cv2.imshow to display a real-time view of what is happening for use later with cv2.rectangles
3. Continuously loops through the NumPy array to determine if any of the loaded images have been found
4. If anything has been found it will either print "avoid" or "attack" to the terminal.

There's also additional logging that I've not added to the below code as it isn't relevant. It simply logs the attack/avoid to a .txt file with time stamps.

My issue:

At the moment my object detection loop for i in image_list: never stops when it finds one of the 24 images, it loops over them again and again swapping away from the found image.

My question:

When one of the 24 images is detected, how do I stop my detection loop without stopping my cv2.imshow loop. For example, if image 5 is found. I want to keep image 5 highlighted with cv2.rectangle rather than only having it highlight for a fraction of a second when it comes up in the detection loop.

# Load Images 
def loadImages(directory):
    # Intialise empty array
    image_list = []
    # Add images to array
    for i in directory:
        img = cv.imread(i, cv.IMREAD_REDUCED_GRAYSCALE_2)
        image_list.append((img, i))
    return image_list

def objectDetection(image_list, threshold_value):
    # Video Loop
    while True:
        # Listener to check if attack! button displaying
        if py.locateCenterOnScreen('!attack.jpg', region = (1282, 931, 1585-1282, 1032-931), grayscale=True) != None:
            # Set attack_loop to false as haven't found anything yet, only set to true when something has been attacked
            attack_loop = False
            while attack_loop is False:
                # Grab PSS Window and find size 
                window = pygetwindow.getWindowsWithTitle('App')[0]
                x1 = window.left
                y1 = window.top
                height = window.height
                width = window.width
                x2 = x1 + width
                y2 = y1 + height
                # Actual Video Loop, cropped down to the specific window,
                # resized to 1/2 size, and converted to BGR for OpenCV
                haystack_img = ImageGrab.grab(bbox=(x1, y1, x2, y2))
                (width, height) = (haystack_img.width // 2, haystack_img.height // 2)
                haystack_img_resized = haystack_img.resize((width, height))
                crop = haystack_img_resized.crop((360,22,600,65))
                haystack_img_np = np.array(crop)
                haystack = cv.cvtColor(haystack_img_np, cv.COLOR_BGR2GRAY)
                cv.imshow("Screen", haystack)

                # Object Detection
                for i in image_list:
                    needle_img = i[0]
                    needle_name = i[1]
                    result = cv.matchTemplate(haystack, needle_img, cv.TM_CCORR_NORMED)
            
                    # Get the best match position
                    min_val, max_val, min_loc, max_loc = cv.minMaxLoc(result)

                    # Define top left and bottom right and threshold
                    (H, W) = i[0].shape[:2] 
                    top_left = max_loc
                    bottom_right = (top_left[0] + W, top_left[1] + H)
                    threshold = threshold_value
                    
                    # If something has been detected click keep looking code
                    if max_val >= threshold:
                        cv.rectangle(haystack, top_left, bottom_right, 255, 2)
                        print("avoid")
                    else:
                        print("attack")
                        if max_val >= threshold:print(max_val)
    
       
### Main Script ###

# load images to detect 
ships_to_avoid = loadImages(defeat)
# Run actual image detection
objectDetection(ships_to_avoid, 0.92)

please review [mre]. your question seems to be on basic (python) programming but you tagged various other things... are they relevant to the problem? — Christoph Rackwitz, Feb 08 '22 at 13:02
You could take the inner loop and turn it into a function. Call it from the outer loop. The inner loop can `continue` and `break` (or `return`) without issue. You might want the function to `yield` data to the outer loop rather than, say, return a huge list. — Jack Deeth, Feb 08 '22 at 13:04
@ChristophRackwitz you told me previously not to do that: https://stackoverflow.com/questions/69932804/how-do-i-use-pyautogui-locateonscreen-for-multiple-images-in-multiple-regions it felt relevant because simply breaking out of the loop isn't the issue but maintaining the video loop concurrently. — Stevphen, Feb 08 '22 at 13:47
I was talking about the tags specifically, not the code. basic programming questions don't need to summon the people that monitor those technology-specific tags. -- I realize there can be ambiguity between simply omitting code and preparing a MRE. the MRE is complete/executable, but omitted code would still be required to understand the code that _is_ there, so its absence is a hurdle. -- your question here would benefit from structuring and condensing the code. at least half the lines, if not more, could be moved into a function `do_something`. out of sight, out of mind. — Christoph Rackwitz, Feb 08 '22 at 13:52
I believe you need some programming 101, and I don't mean "how to use this language" (syntax), but *concepts* like state, and state machines (which react to changes of state). -- some literal pen-on-paper sketches of the "things" here and their "actions" might help you structure your code. — Christoph Rackwitz, Feb 08 '22 at 13:55
I understand now, thank you and yes probably. Although a lot of those types of courses start by saying, find a project. This is one of those projects and maybe I don't know enough to make use of stack properly. — Stevphen, Feb 08 '22 at 13:57
ok, this is turning into a code review. please try out https://codereview.stackexchange.com/ -- practically: *refactoring*. move the detection into its own function, and make that _only_ detect (and return/print information). move the imshow stuff out of there. invert `if ...` to instead break/continue the loop: `if locateCenterOnScreen... is None: continue`. that removes a level of indentation, requiring you to keep fewer "conditions" in your head while reading the code. -- @JackDeeth already said it, I skipped those comments :S — Christoph Rackwitz, Feb 08 '22 at 14:02
Thank you on both accounts, I didn't know that stack existed! — Stevphen, Feb 08 '22 at 14:03
Does this answer your question? [How to break out of multiple loops?](https://stackoverflow.com/questions/189645/how-to-break-out-of-multiple-loops) — FLAK-ZOSO, Feb 08 '22 at 15:16

score 1 · Answer 1 · answered Feb 08 '22 at 13:17

You can simply stop the inner loop with the python word break inside your condition, so it will stop whenever it is True. This break will not affect the outer loop. For more info about stopping nested loops I would recommend you this article: 5 Ways To Break Out of Nested Loops in Python

score 1 · Accepted Answer · answered Feb 08 '22 at 15:16

1

If you want to break an inner loop breaking also the outer loop I would suggest an approach like this:

for a in xrange(10):
    for b in xrange(20):
        if <condition>:
            # Break the inner loop...
            break
    else: # The loop ended without any break
        # Continue if the inner loop wasn't broken.
        continue
    # Inner loop was broken, break the outer.
    break

answered Feb 08 '22 at 15:16

FLAK-ZOSO

3,873
4
8
28

1

This in conjunction with @Christoph Rackwitz and JackDeeth in the comments solved the issue. – Stevphen Feb 08 '22 at 21:09

score 0 · Answer 3 · answered Feb 09 '22 at 10:03

The full working code.

import pyautogui as py
import glob
import cv2 as cv
import os
import numpy as np
import pygetwindow
import time
from PIL import ImageGrab

os.chdir(r'C:\test')
avoid = glob.glob(r"C:\test\*.png")

# Load Images 
def loadImages(directory):
    # Intialise empty array
    image_list = []
    # Add images to array
    for i in directory:
        img = cv.imread(i, cv.IMREAD_REDUCED_GRAYSCALE_2)
        image_list.append((img, i))
    return image_list

def videoLoop():
    # Grab Window and find size 
    window = pygetwindow.getWindowsWithTitle('App')[0]
    x1 = window.left
    y1 = window.top
    height = window.height
    width = window.width
    x2 = x1 + width
    y2 = y1 + height
    # Actual Video Loop, cropped down to the specific window,
    # resized to 1/2 size, and converted to BGR for OpenCV
    haystack_img = ImageGrab.grab(bbox=(x1, y1, x2, y2))
    (width, height) = (haystack_img.width // 2, haystack_img.height // 2)
    haystack_img_resized = haystack_img.resize((width, height))
    crop = haystack_img_resized.crop((360,22,600,65))
    haystack_img_np = np.array(crop)
    haystack = cv.cvtColor(haystack_img_np, cv.COLOR_BGR2GRAY)
    cv.imshow("Screen", haystack)
    return haystack


def objectDetection(image_list, threshold, haystack, a):
    # Object Detection
    for x in range (0, a):
        for i in image_list:
            needle_img = i[0]
            needle_name = i[1]
            sliced_name = needle_name.split("\\")[-1]
            result = cv.matchTemplate(haystack, needle_img, cv.TM_CCORR_NORMED)

            # Get the best match position
            min_val, max_val, min_loc, max_loc = cv.minMaxLoc(result)

            # Define top left and bottom right and threshold
            (H, W) = i[0].shape[:2] 
            top_left = max_loc
            bottom_right = (top_left[0] + W, top_left[1] + H)
            
            # If something has been detected click keep looking code
            if max_val >= threshold:
                cv.rectangle(haystack, top_left, bottom_right, 255, 2)
                cv.imshow("Screen", haystack)
                keep_looking()

            if cv.waitKey(1) == ord('q'):
                cv.destroyAllWindows()
                break             


def keep_looking():
    py.moveTo(1715,1000,0.1)
    py.sleep(0.01)
    py.click()    


# load images to detect 
ships_to_avoid = loadImages(avoid)

while True:
    window = videoLoop()
    objectDetection(ships_to_avoid, 0.92, window, 10)

Thank you @ChristophRackwitz although that other stack sent me back here, said it wsan't a review. Probably isn't perfect but it does what I intended. — Stevphen, Feb 09 '22 at 10:04

How should I structure this code to break out of multiple nested loops at once?

3 Answers3