Running multiple instances of a shell function parallelly

Question

I'm having a shell script as follows.

#!/bin/bash

myfunc() {
   #do something (call a rest service)
   sleep 300
   status=$(get status of the operation performed above)
   while [ "$status" != "succeeded" ]; do
       #do something (call a rest service)
       sleep 300
       status=$(get status of the operation performed above)
   done
}

a=0

while [ $a -lt 1000 ]
do
   a=`expr $a + 1`
   myfunc
done

In best case, above script takes at least 5*1000=5000 seconds to complete running.

Is there a way to make the myfunc call parallel so that we can let the while loop to spawn multiple running instances of the myfunc??

Note: Somehow, this script should wait until all instances of myfunc executions are completed.

I am not sure if bash does multi threading. You might wanna use some more sophisticated language than shell script to write this ... — mahatmanich, Jun 09 '21 at 14:20
To avoid output from jobs mixing, you can iuse GNU Parallel: `export -f myfunc`, run `parallel myfunc ::: {1..1000}` — Ole Tange, Jun 13 '21 at 10:10

Antonio Petricca · Accepted Answer · 2021-06-09T14:23:26.183

2

I think you could update your script as follow:

#!/bin/bash

myfunc() {
   job_id=$1
   echo "Started job ${job_id}..."

   #do something (call a rest service)
   sleep 300
   status=$(get status of the operation performed above)
   while [ "$status" != "succeeded" ]; do
       #do something (call a rest service)
       sleep 300
       status=$(get status of the operation performed above)
   done

   echo "Terminated job ${job_id}."
}

a=0

while [ $a -lt 1000 ]
do
   echo $a
   a=(($a + 1))
   myfunc $a &
done

wait
echo "Parallel execution terminated."

edited Jun 09 '21 at 14:23

answered Jun 09 '21 at 14:21

Antonio Petricca

8,891
5
36
74

`a=($a + 1)` that's a typo? – KamilCuk Jun 09 '21 at 14:23
Sorry, it is a typo, I fixed it. – Antonio Petricca Jun 09 '21 at 14:24
@AntonioPetricca sorry, I kept it as it is and asked a new question. https://stackoverflow.com/questions/67923291/running-multiple-instances-of-a-shell-function-parallelly can you check this as well? – Vajira Prabuddhaka Jun 10 '21 at 14:38

score 2 · Answer 2 · answered Jun 09 '21 at 14:25

2

Put them in background.

while [ $a -lt 1000 ]
do echo $a
   a=`expr $a + 1`
   myfunc &
done
wait

Or, in more idiomatic bash,

while (( a < 1000 ))
do echo $((a++))
   myfunc &
done
wait

answered Jun 09 '21 at 14:25

Paul Hodges

13,382
1
17
36

Running multiple instances of a shell function parallelly

2 Answers2

Linked

Related