Representing a collection of different structs in an easy to access way in Go

Question

First of all, just to preface this, I'm stuck with Go version 1.16, so the generics stuff in 1.18 is not an option unfortunately. (However feel free to add it as an answer in case someone stumbles across this in the future)

So I've got a bunch of functions that look like this:

func GetPerson(id int) chan Person { // stuff }
func GetPet(id int) chan Pet { // stuff }
// similar methods that get different structs

And the simple goal is to call all these methods and get the data out of the channels and provide them for access. The reason why the objects are being loaded into channels is so that we can call these functions concurrently, as often they'll go ahead and make http requests and so on.

Now we want to pull all the data off these channels and make it accessible to use all at once, the easiest way to do this would be something like

person := <- GetPerson(id)
pet := <- GetPet(id)
...

This wouldn't then take advantage of the concurrent nature of the functions above, so my next solution would be to have a type that has a function on it that instead basically loops over the channels that are returned and loads them into the fields, like so.

type OverallData struct {
    field1 struct{}
    field2 struct{}
    // etc.
}

func (data OverallData) RetrieveData(personId int, petId int) {
    for x:= 0, x < chanLen, x++ {
        select {
        case data.field1 = <- GetPerson(personId):
        case data.field2 = <- GetPet(petId):
        // etc
        }
    }  

}

func GetOverallData(personId int, petId int) OverallData {
    data := OverallData{}
    data.RetrieveData(personId, petId)
    return data
}

But I'm not a massive fan of this, it requires a type to aggregate the responses for any particular group, and these types would be very similar in nature. Also the GetOverallData function will take more and more arguments depending on the amount of data you need back, and it becomes unruly (you can just add another type in for this, but I think that's just a way to move the problem somewhere else)

So I started thinking of a more generic solution, and came up with something a bit worse

func ProcessResults(results []chan struct{}) map[reflect.Type]struct{} {
    respMap := map[reflect.Type]struct{}
    for _, ch := range results {
        go func(c chan struct{}) {
            val <- ch
            typ := reflect.TypeOf(val)
            respMap[typ] = val
        }(ch)
    }
    return respMap
}

The goal with the above was that you would then just have one call that looks something like. This would be just as much code as you can need, no need to add any types or pass tons of stuff as paramaeters. And this is easy to write and read (mostly, last line is a bit messy). You also would know for a fact your type would match when you cast it.

responses := ProcessResults([]chan struct{} {
    GetPerson(personId),
    GetPet(petId),
})


person := responses[reflect.TypeOf(Person)].(Person)

The problem with it, is that it doesn't work, you can't just map the types returned by my methods into struct{} within an array, and while I could just cast it all to structs I suspect the reflect.TypeOf() I was using before will not work after converting it to a struct (as the type with then by struct{} and my map key will not work), there are also other issues like say I had two channels that returned the same type, but could get around this.

I'm also thinking about adding a embedded struct that all the types returned from the channel will have, forcing them to confine to an interface like in this question: Go Interface Fields

I can use that struct to specify a key for the map so I don't have to mess around with the type checking, and also I can pass all that data as that interface type rather than type casting it back to a struct and therefore wouldn't lose the typing and it can cast it back to the original type returned by the channel.

When I got to this point I realise I'm probably trying to do something that I shouldn't be doing with Go, and there is a better way to accomplish it. Does anyone have any better ideas? Else I'll probably have a crack at the above, and if it feels wrong, will leave it and go with my second solution.

Answer 1

It is typical to manage concurrency in the caller instead of the called function. Use an error group to manage a group of concurrent calls.

Declare the functions like this:

func GetPerson(id int) (*Person, error)
func GetPet(id int) (*Pet, error)

Call the functions concurrently like this:

var g errgroup.Group

var person *Person
g.Go(func() error {
    var err error
    person, err = GetPerson(personId)
    return err
})

var pet *Pet
g.Go(func() error {
    var err error
    pet, err = GetPet(petId)
    return err
})

if err := g.Wait(); err != nil {
   // handle error
}

// access person and pet variables here.

Use the reflect package to eliminate the boilerplate:

// ego calls fn with args in group g. fn's second return value
// is assumed to be an error and is returned to the error group.
// If the error is nil, then fn's first return value is assigned
// to the value that presult points to.
func ego(g *errgroup.Group, presult any, fn any, args ...any) {
    g.Go(func() error {
        var rargs []reflect.Value
        for _, arg := range args {
            rargs = append(rargs, reflect.ValueOf(arg))
        }
        result := reflect.ValueOf(fn).Call(rargs)
        if err, ok := result[1].Interface().(error); ok && err != nil {
            return err
        }
        reflect.ValueOf(presult).Elem().Set(result[0])
        return nil
    })
}

Use it like this:

var g &errgroup.Group{}
var person *Person
var pet *Pet
ego(g, &person, GetPerson, personId)
ego(g, &pet, GetPet, petId)
if err := g.Wait(); err != nil {
   // handle error
}
// access person and pet variables here.