Using dynamic struct of slices (which contains foreign keys) and go-gorm to Insert data into db

Question

I've a dynamic struct created from Ompluscator which contains a nested struct (slices of struct) with foreign key references.

The Parent (User Table) and Child (Credit Card) Tables gets created successfully. Even inserting into User Table is successful, but later it ends up with field value not valid error when processing credit_card table entries.

Here is the complete code:

creditCardStruct := dynamicstruct.NewStruct().
    AddField("Bank", "", `protobuf:"bytes,1,opt,name=number,proto3" json:"bank,omitempty" gorm:"column:bank;not null"`).
    AddField("Number", "", `protobuf:"bytes,1,opt,name=number,proto3" json:"number,omitempty" gorm:"column:number;not null;size:16;primary_key"`).
    AddField("Cvv", "", `protobuf:"bytes,2,opt,name=cvv,proto3" json:"cvv,omitempty" gorm:"column:cvv;not null;size:5" skyflow:"only4digits"`).
    //AddField("Expiry", timeNow, `protobuf:"bytes,3,opt,name=expiry,proto3" json:"expiry,omitempty" gorm:"column:expiry;type:timestamp;not null;"`).
    AddField("Name", "", `protobuf:"bytes,1,opt,name=number,proto3" json:"name,omitempty" gorm:"column:name;not null"`).
    AddField("ProfileID", "", `protobuf:"bytes,5,opt,name=userName,proto3" json:"profile_id,omitempty" sql:"type:string REFERENCES user_normalized1(profile_id)" gorm:"column:profile_id;not null;primary_key"`).
    Build().NewSliceOfStructs()

userInstance := dynamicstruct.NewStruct().
    AddField("ProfileID", "", `protobuf:"bytes,1,opt,name=number,proto3" json:"profile_id,omitempty" gorm:"column:profile_id;unique;not null;unique_index;primary_key"`).
    AddField("CreditCards", creditCardStruct, `protobuf:"bytes,1,opt,name=number,proto3" json:"credit_cards,omitempty" gorm:"foreignkey:ProfileID;association_foreignkey:ProfileID"`).
    Build().
    New()

db.Table("user_normalized1").AutoMigrate(userInstance)
db.Table("credit_card1").AutoMigrate(creditCardStruct)

data := []byte(`
            {
            "profile_id":"123",
            "first_name": "fname",
            "last_name": "lname",
            "some_text": "dummy"

            ,"credit_cards": [{"bank":"bank1", "cvv":"123", "name":"fname", "number":"nlabla1"},{"bank":"bank2", "cvv":"1234", "name":"lname", "number":"nlabla2"}]
            }
            `)

err := json.Unmarshal(data, &userInstance)
if err != nil {
    log.Fatal(err)
}
data, err = json.Marshal(userInstance)
if err != nil {
    log.Fatal(err)
}

fmt.Println(string(data))

db.Table("user_normalized1").Create(userInstance)

This is the log:

[2020-06-04 16:03:25]  [8.93ms]  CREATE TABLE "user_normalized1" ("profile_id" text NOT NULL UNIQUE , PRIMARY KEY ("profile_id"))  
[0 rows affected or returned ] 

[2020-06-04 16:03:25]  [106.26ms]  CREATE UNIQUE INDEX uix_user_normalized1_profile_id ON "user_normalized1"(profile_id)   
[0 rows affected or returned ] 

[2020-06-04 16:03:25]  [44.12ms]  CREATE TABLE "credit_card1" ("profile_id" string REFERENCES user_normalized1(profile_id) NOT NULL,"bank" text NOT NULL,"number" varchar(16) NOT NULL,"cvv" varchar(5) NOT NULL,"name" text NOT NULL , PRIMARY KEY ("profile_id","number"))  
[0 rows affected or returned ] 
{"profile_id":"123","credit_cards":[{"bank":"bank1","number":"nlabla1","cvv":"123","name":"fname"},{"bank":"bank2","number":"nlabla2","cvv":"1234","name":"lname"}]}
[2020-06-04 16:03:25]  [1.50ms]  INSERT INTO "user_normalized1" ("profile_id") VALUES ('123') RETURNING "user_normalized1"."profile_id"  
[1 rows affected or returned ] 
[2020-06-04 16:03:25]  field value not valid

score 0 · Accepted Answer · answered Jun 04 '20 at 11:12

I couldn't find any info on how to achieve it. After a lot of debugging I was able to get it working, so posting my solution.

Short Answer:

The Nested Slices must be Array of Pointers

Define your Nested/Child structs this way

 creditCardStruct := dynamicstruct.NewStruct().
        AddField("Bank", "", `protobuf:"bytes,1,opt,name=number,proto3" json:"bank,omitempty" gorm:"column:bank;not null"`).
        AddField("Number", "", `protobuf:"bytes,1,opt,name=number,proto3" json:"number,omitempty" gorm:"column:number;not null;size:16;primary_key"`).
        AddField("Cvv", "", `protobuf:"bytes,2,opt,name=cvv,proto3" json:"cvv,omitempty" gorm:"column:cvv;not null;size:5" skyflow:"only4digits"`).
        //AddField("Expiry", timeNow, `protobuf:"bytes,3,opt,name=expiry,proto3" json:"expiry,omitempty" gorm:"column:expiry;type:timestamp;not null;"`).
        AddField("Name", "", `protobuf:"bytes,1,opt,name=number,proto3" json:"name,omitempty" gorm:"column:name;not null"`).
        AddField("ProfileID", "", `protobuf:"bytes,5,opt,name=userName,proto3" json:"profile_id,omitempty" sql:"type:string REFERENCES user_normalized1(profile_id)" gorm:"column:profile_id;not null;primary_key"`).
        Build().New()

 creditCardArrayInstance := reflect.New(reflect.SliceOf(reflect.TypeOf(creditCardStruct))).Elem().Interface()

 userInstance := dynamicstruct.NewStruct().
    AddField("ProfileID", "", `protobuf:"bytes,1,opt,name=number,proto3" json:"profile_id,omitempty" gorm:"column:profile_id;unique;not null;unique_index;primary_key"`).
    AddField("CreditCards", creditCardArrayInstance, `protobuf:"bytes,1,opt,name=number,proto3" json:"credit_cards,omitempty" gorm:"foreignkey:ProfileID;association_foreignkey:ProfileID"`).
    Build().
    New()

db.Table("user_normalized1").AutoMigrate(userInstance)
db.Table("credit_card1").AutoMigrate(creditCardStruct)

Long Answer:

When DynamicStruct was creating a Slice of Struct, it was creating a pointer of the slice reflect.New(reflect.SliceOf(ds.definition)).Interface(). This was actually a Pointer to the Slice. So instead of []*CreditCards This became *[]*CreditCards
When gorm performs a callback to save Associations, which has a separate block for handling slices, the runtime type identified was not slices (It was pointer). So it assumed it to be struct and went ahead processing it as a simple struct.
However when gorm built out the fields from scope, for non-struct fields (like slices) it doesn't set the Field Definition. So when gorm was processing the previous slice as a normal struct, its field was of Invalid type as it was never set.

So the solution was to just create the dynamic structure manually as an Array of Pointers rather than the provided NewSliceOfStructs API.

Using dynamic struct of slices (which contains foreign keys) and go-gorm to Insert data into db

1 Answers1