0

Working on project in Python, and unfortunately i am newbie in Python. The code must execute multiple probes simultaneously and gather the results.

I created the following code to run the probes in different threads and gather the results (used one of the answers here : How to get the return value from a thread in python?)

        summary = ProbeRunnerResult()
        jobs = []
        queues = []

        for probe in self.probes.values():
                que = Queue()
                t = threading.Thread(target=lambda q, arg1: q.put(probe.run_check(arg1)), args=(que, context))
                jobs.append(t)
                queues.append(que)
               
        for j in jobs:
            j.start()

        for j in jobs:
            j.join()

        for next_queue in queues:
            summary.probe_results.append(next_queue.get())

and the code of ProbeRunnerResult is :

class ProbeRunnerResult:
    """Probe runner result. Includes results from multiple probes"""
    def __init__(self) -> None:
        self.start_time = time.time()
        self.end_time = 0
        self.probe_results: List[ProbeResult] = []

The probe.run_check() returns an object of type/class ProbeResult. I face two problems: 1.I put break points and dumps in the run_check() method of the Probe class, but they show, that the run_check() method of only the first listed probe is called multiple times. 2.Some of the probes execution (the execution of the run_check() method) can take minutes and i am not sure whether the current code is actually handling this correctly.

So i need advices how to handle these two problems correctly.

Ivajlo Iliev
  • 303
  • 1
  • 3
  • 19
  • you need to have 1 (and only 1) instance of `Queue()`, pass it to each thread that will put the result into the queue. The main thread will read the result from the queue. – balderman Sep 05 '21 at 09:50
  • Thank you @balderman but this did not fix the fact, that the run_check method of only the first probe was called. – Ivajlo Iliev Sep 06 '21 at 06:01

0 Answers0