-2

There are various task executors, with different properties, and some of them only support non-blocking calls. So, I was thinking, whether there's a need to use mutex/channel to safely deliver task results to calling go-routine, or whether is it enough simple WaitGroup?

For sake of simplicity, and specificity of the question, an example using very naive task executor launching function directly as go routine:

func TestRace(t *testing.T) {
    var wg sync.WaitGroup

    a, b := 1, 2

    wg.Add(1)

    // this func would be passed to real executor
    go func() {
        a, b = a+1, b+1
        wg.Done()
    }()

    wg.Wait()

    assert.Equal(t, a, 2)
    assert.Equal(t, b, 3)
}

Execution of the test above with -race option didn't fail, on my machine. However, is that enough guarantee? What if go-routine is executed on different CPU core, or on CPU core block (AMD CCX), or on different CPU in multi-socket setups?

So, the question is, can I use WaitGroup to provide synchronization (block and return values) for non-blocking executors?

kravemir
  • 10,636
  • 17
  • 64
  • 111
  • 3
    Yes WaitGroup provides synchronization. Variables shared between goroutines can’t be stack allocated, they must escape to the heap. – JimB Nov 20 '19 at 12:32
  • @JimB is than an answer or a comment? And, if it is an answer, could you please provide some source / reference backing that statement? – kravemir Nov 20 '19 at 12:46
  • 1
    Also note that there are no "on stack" or "on heap" variables in Go. While technically there is a stack and a heap Go as the language makes absolutely no claim about this: All variables are equal. Also There is zero need to think about CPUs, cores or threads. Your code is either correct or not (your's is). All that stack/heap, CPU/core/thread stuff is handled by the compiler. – Volker Nov 20 '19 at 13:12
  • @Volker what is triviality? How `WaitGroup` works? If yes, then I agree, it's trivial. However, the question is about something else,... and not about how `WaitGroup` works,... The question is about safety of shared variables (there are caches involved, and other factors,... ), which doesn't always need to be immediately in synchronized states among different threads/cores/cpus. – kravemir Nov 20 '19 at 13:12
  • 2
    "there are caches involved, and other factors,... ", No there are not. CPU caches and that like is abstracted away by Go the language. See the Go Memory Model. Several things ensure a happens-before relation and package sync is one of them (channel send/receives an other). From a Go perspective there simply is nether a stack variable nor a CPU cache. If there is a HB relation between write and read you will read what you wrote. Dead simple. The whole raison-d'etre of Go is to get rid of the nasty details. – Volker Nov 20 '19 at 13:19
  • 2
    To put it another way, a `WaitGroup` wouldn't be of much use if it didn't actually work. The `WaitGroup` here is to ensure that `a, b = a+1, b+1` has executed, so there's no reason to assume it hasn't. – JimB Nov 20 '19 at 13:24
  • 3
    The title asks one question, the body asks four more different questions. Please limit posts to focus on one question per post. – Adrian Nov 20 '19 at 14:19
  • 1
    I'm sorry you feel that way. I'm trying to help you craft a question that can be effectively answered. You can take or leave that advice, but if you want to get a good answer from the community, you might want to consider it. – Adrian Nov 20 '19 at 15:15
  • @Adrian can you please phrase these four more different questions? From my PoV, these questions are just different phrasing of the title, or they mention possible edge-cases,... However, the primary question is still the same: if I use WaitGroup and write to stack variables, then is safety guaranteed? – kravemir Nov 20 '19 at 15:18
  • 1
    As @JimB noted, if a value is shared between goroutines, it cannot be stack-allocated, so the question is moot (see https://stackoverflow.com/questions/51489323/how-are-go-closures-layed-out-in-memory). `WaitGroup` works correctly. – Adrian Nov 20 '19 at 15:31
  • @Adrian that's actually part of the correct answer, a clarification, because I didn't know those variables would turn to be a heap allocated memory. And, where are these *four different* questions in the body, as you've stated before? – kravemir Nov 20 '19 at 15:35
  • They're easily spotted by the question marks. – Adrian Nov 20 '19 at 15:38
  • 1
    To answer some of the other questions, no the `-race` detector is not a guarantee that there are no possible data races, just that none happened during execution. You have no control over if the goroutine is on another core, or another socket; the guarantees you have are laid out by the go memory model, which is well documented. – JimB Nov 20 '19 at 15:38
  • @JimB can you please answer the question with references to documentation you're stating? I would like to read it. – kravemir Nov 20 '19 at 15:43
  • 1
    https://golang.org/ref/mem – JimB Nov 20 '19 at 15:46

1 Answers1

2

JimB should perhaps provide this as the answer, but I'll copy it from his comments, starting with this one:

The WaitGroup here is to ensure that a, b = a+1, b+1 has executed, so there's no reason to assume it hasn't.

[and]

[T]he guarantees you have are laid out by the go memory model, which is well documented [here]. [Specifically, the combination of wg.Done() and wg.Wait() in the example suffices to guarantee non-racy access to the two variables a and b.]

As long as this question exists, it's probably a good idea to copy Adrian's comment too:

As @JimB noted, if a value is shared between goroutines, it cannot be stack-allocated, so the question is moot (see How are Go closures layed out in memory?). WaitGroup works correctly.

The fact that closure variables are heap-allocated is an implementation detail: it might not be true in the future. But the sync.WaitGroup guarantee will still be true in the future, even if some clever future Go compiler is able to keep those variables on some stack.

("Which stack?" is another question entirely, but one for the hypothetical future clever Go compiler to answer. The WaitGroup and memory model provide the rules.)

torek
  • 448,244
  • 59
  • 642
  • 775