How can I use foreach and fork together to do something in parallel?

Question

This question is not UVM specific but the example that I am working on is UVM related. I have an array of agents in my UVM environment and I would like to launch a sequence on all of them in parallel.

If I do the below:

foreach (env.agt[i])
  begin
    seq.start(env.agt[i].sqr);
  end

, the sequence seq first executes on env.agt[0].sqr. Once that gets over, it then executes on env.agt[1].sqr and so on.

I want to implement a foreach-fork statement so that seq is executed in parallel on all agt[i] sequencers.

No matter how I order the fork-join and foreach, I am not able to achieve that. Can you please help me get that parallel sequence launching behavior?

Thanks.

Update to clarify the problem I am trying to solve: The outcome of the below code constructs is the exact same as above without the fork-join.


foreach (env.agt[i])
  fork
    seq.start(env.agt[i].sqr);
  join


fork
  foreach (env.agt[i])
    seq.start(env.agt[i].sqr);
  join


// As per example in § 9.3.2 of IEEE SystemVerilog 2012 standard
for (int i=0; i<`CONST; ++i)
  begin
    fork
      automatic int var_i = i;
      seq.start(env.agt[var_i].sqr);
    join
  end

Greg · Answer 1 · 2017-12-18T19:52:20.397

The issue is each thread of the fork is pointing to the same static variable i. Each thread needs its own unique copy and this can be achieved with the automatic keyword.

foreach (env.agt[i])
  begin
    automatic int var_i = i;
    fork
      seq.start(env.agt[var_i].sqr);
    join_none // non_blocking, allow next operation to start
  end
wait fork;// wait for all forked threads in current scope to end

IEEE std 1800-2012 § 6.21 "Scope and lifetime" gives examples of the uses static and automatic. Also check out § 9.3.2 "Parallel blocks", the last example shows demonstrates parallel threads in a for-loop.

Use join_none to create new threads; § 9.3.2 "Parallel blocks", Table 9-1—"fork-join control options".

Use fork wait statement to wait for all threads in the current scope to complete; § 9.6.1 "Wait fork statement"

Example:

byte a[4];
initial begin
    foreach(a[i]) begin
        automatic int j =i;
        fork
            begin
                a[j] = j;
                #($urandom_range(3,1));
                $display("%t :: a[i:%0d]:%h a[j:%0d]:%h",
                        $time, i,a[i], j,a[j]);
            end
        join_none // non-blocking thread
    end
    wait fork; // wait for all forked threads in current scope to end
    $finish;
end

Outputs:

2 :: a[i:4]:00 a[j:3]:03
2 :: a[i:4]:00 a[j:0]:00
3 :: a[i:4]:00 a[j:2]:02
3 :: a[i:4]:00 a[j:1]:01

Hi Greg, that didn't make the sequences start in parallel. I referred to § 9.3.2 and tried using for loop instead of foreach, with the fork-join keywords (I can't use fork-join_none in my application as the example shows). But still I see the sequences starting in succession. I have updated my original post with the example that didn't work either. — Kaushal Modi, Nov 05 '13 at 18:19
@kaushalmodi didn't realize that `seq.start(...)` was a blocking task method. I have updated my answer with an example. — Greg, Nov 05 '13 at 19:28
Thank you for the detailed explanation. The example you posted worked perfectly well. I just needed to do some tweaking to fit my UVM environment. — Kaushal Modi, Nov 05 '13 at 22:08
BTW, no need for the `automatic` keyword if this code is inside a class method - all variable declarations inside class methods are automatic by default. The reverse is default static outside of a class, to be backward compatible with Verilog. — dave_59, Nov 06 '13 at 06:24

score 2 · Answer 2 · answered Nov 07 '13 at 12:32

I think that the more "UVM" way to approach this is with a virtual sequence. Assuming you already have a virtual sequencer which instantiates an array of agent sequencers then the body your virtual sequence would look something like this:

fork
  begin: isolation_thread
    foreach(p_sequencer.agent_sqr[i])
      automatic int j = i;
      fork
        begin
          `uvm_do_on(seq, p_sequencer.agent_sqr[j]);
        end
      join_none
    end
    wait fork;
  end: isolation_thread
join

This has worked for me in the past.

Kaushal Modi · Accepted Answer · 2022-04-20T17:15:51.167

Greg's solution below helped me derive the solution to my UVM based problem. Here is my solution:

The below fork-join block resides in the main_phase task in a test case class. The wait fork; statement waits for all the fork statements in its scope (= the foreach_fork begin-end block) to finish before proceeding further. An important thing to note is that the wrapping fork-join around the begin-end was required for setting the scope of wait fork; to the foreach_fork block.


fork 
  begin : foreach_fork
    seq_class seq **[`CONST]**;
    foreach(env.agt[i])
      begin
        int j = i;
        **seq[j] = seq_class::type_id::create
                  (.name($sformatf("seq_%0d", j)), .contxt(get_full_name()));**
        fork
          begin
            seq[j].start(env.agt[j].sqr);
          end
        join_none // non-blocking thread
      end
    **wait fork;**
  end : foreach_fork  
join

Alternative solution that makes use of the in-sequence objection to delay the sim end.


begin
  seq_class seq **[`CONST]**;
  foreach(env.agt[i])
    begin
      int j = i;
      **seq[j] = seq_class::type_id::create
                (.name($sformatf("seq_%0d", j)), .contxt(get_full_name()));**
      fork
        begin
          **seq[j].starting_phase = phase;**
          seq[j].start(env.agt[j].sqr);
        end
      join_none // non-blocking thread
    end
end

I realized that I also needed to create a new sequence object for each seq I wanted to run in parallel.

Thanks to Dave for making the point that the properties in System Verilog classes are automatic by default.

Note for the alternative solution: As I didn't use wait fork; I use the UVM objections raised in the sequence itself to do the job of holding off the simulation $finish call. To enable raising of the objections in the sequences, I use the seq[j].starting_phase = phase; construct.

Create a sub-scope to wrap around the `foreach-begin-fork-join_none-end` with a `begin-end`, make sure the "watchdog timer" is not within this sub-scope (Adding additional sub-scopes is okay too; recommend using scope labels). Add the `wait fork` to the end this new sub-scope. The "watchdog timer" will not be part of the sub-scopes `fork wait` monitoring. A `semaphore` is another option but too complected to to explain in a comment. — Greg, Nov 05 '13 at 22:58
Hi Greg, thanks. I had tried that yesterday and due to some mistake that didn't work out. Today I simply put the `wait fork;` before that last `end` and it worked! Now I am sticking with the `wait fork;` solution. :) — Kaushal Modi, Nov 06 '13 at 18:29
Alright, I figured why creating the begin-end scope didn't work yesterday but worked today. Yesterday I did not have a fork-join wrapping that begin-end. So my understand is that the `wait fork;` treated the first fork-join block around it as the scope. If the `wait fork;` is not inside any fork-join, it waits for ALL fork statements to end. — Kaushal Modi, Nov 06 '13 at 18:47

score -2 · Answer 4 · answered Nov 05 '13 at 16:18

-2

try with

int i = 0
foreach (env.agt)
  begin
    seq.start(env.agt[i].sqr);
    i++;
end

answered Nov 05 '13 at 16:18

Mauro Midolo

1,841
3
14
34

You did not get the question. The code snippet I posted works; you don't need to manually increment i. The problem is that I can't get the seq.start(env.agt[i].sqr) to happen in parallel whether I put fork-join inside or outside the foreach. – Kaushal Modi Nov 05 '13 at 16:26

How can I use foreach and fork together to do something in parallel?

4 Answers4