Cosmos DB Save Documents With Interface properties .NET SDK V3. How do I Save A Document?

Question

I have run into problems saving documents with Properties that are Interfaces' I am receiving this error.

Could not create an instance of type GTA__Tests.ItemTests.IMyCommand. Type is an interface or abstract class and cannot be instantiated.

My Interface:

public interface IMyCommand
{
   [JsonProperty(PropertyName = "$type")]
   string CommandType { get; } 
}

My implementation:

public class MyCommand1 : IMyCommand
{
    [JsonProperty("$type")]
    public string CommandType => nameof(MyCommand1);
}

public class MyCommand2 : IMyCommand
{
    [JsonProperty("$type")]
    public string CommandType => nameof(MyCommand2);
}

//partition key is CompanyId
public class MyClass
{
     [JsonProperty("id")]
     public Guid Id { get; set; }

     public string CompanyId { get; set; }

     public List<IMyCommand> Commands { get; set; }

     public MyClass(string partitionKey)
     {
         CompanyId = partitionKey;
     }

}

My test code: Using NUnit

//Using cosmosdb .net sdk v3
public class Item_Tests
{
    private string url;
    private string key;
    private string dbName;
    private string containerName;
    private string partitionKey = "Test";
    CosmosClient client;
    Container db;

   [SetUp]
    public void SetUp()
    {
        var config = Config.GetConfig();
        url = config["CosmosLive:Url"];
        key = config["CosmosLive:Key"];
       dbName = config["CosmosLive:DbName"];
       containerName = config["CosmosLive:ContainerName"];
       client = new CosmosClient(url, key);
       db = client.GetContainer(dbName, containerName);
   }

    [Test]
    public async Task SaveAMyClassDocumentToCosmosDb_ReturnsOK()
   {
       var myclass = new MyClass(partitionKey)
       {
            Id = Guid.NewGuid(),
           Commands = new List<IMyCommand>
           {
                new MyCommand1(),
               new MyCommand2()
           }
       };

       var result = await db.UpsertItemAsync<MyClass>(myclass, new 
       PartitionKey(partitionKey));
       Assert.IsTrue(result.StatusCode is System.Net.HttpStatusCode.Created);

    }

This produces a Newtonsoft.Json.JsonSerializationException Where is my code wrong? This is the full excetion:

Newtonsoft.Json.JsonSerializationException: Could not create an instance of type GTA__Tests.ItemTests.IMyCommand. Type is an interface or abstract class and cannot be instantiated. Path 'Commands[0]', line 1, position 103.
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateNewObject(JsonReader reader, JsonObjectContract objectContract, JsonProperty containerMember, JsonProperty containerProperty, String id, Boolean& createdFromNonDefaultCreator)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateList(IList list, JsonReader reader, JsonArrayContract contract, JsonProperty containerProperty, String id)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateList(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, Object existingValue, String id)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.ResolvePropertyAndCreatorValues(JsonObjectContract contract, JsonProperty containerProperty, JsonReader reader, Type objectType)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObjectUsingCreatorWithParameters(JsonReader reader, JsonObjectContract contract, JsonProperty containerProperty, ObjectConstructor`1 creator, String id)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateNewObject(JsonReader reader, JsonObjectContract objectContract, JsonProperty containerMember, JsonProperty containerProperty, String id, Boolean& createdFromNonDefaultCreator)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)
   at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType)
   at Newtonsoft.Json.JsonSerializer.Deserialize(JsonReader reader, Type objectType)
   at Newtonsoft.Json.JsonSerializer.Deserialize[T](JsonReader reader)
   at Microsoft.Azure.Cosmos.CosmosJsonDotNetSerializer.FromStream[T](Stream stream)
   at Microsoft.Azure.Cosmos.CosmosJsonSerializerWrapper.FromStream[T](Stream stream)
   at Microsoft.Azure.Cosmos.CosmosSerializerCore.FromStream[T](Stream stream)
   at Microsoft.Azure.Cosmos.CosmosResponseFactoryCore.ToObjectpublic[T](ResponseMessage responseMessage)
   at Microsoft.Azure.Cosmos.CosmosResponseFactoryCore.<CreateItemResponse>b__8_0[T](ResponseMessage cosmosResponseMessage)
   at Microsoft.Azure.Cosmos.CosmosResponseFactoryCore.ProcessMessage[T](ResponseMessage responseMessage, Func`2 createResponse)
   at Microsoft.Azure.Cosmos.CosmosResponseFactoryCore.CreateItemResponse[T](ResponseMessage responseMessage)
   at Microsoft.Azure.Cosmos.ContainerCore.UpsertItemAsync[T](T item, ITrace trace, Nullable`1 partitionKey, ItemRequestOptions requestOptions, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.ClientContextCore.RunWithDiagnosticsHelperAsync[TResult](ITrace trace, Func`2 task, Func`2 openTelemetry, String operationName, RequestOptions requestOptions)
   at Microsoft.Azure.Cosmos.ClientContextCore.OperationHelperWithRootTraceAsync[TResult](String operationName, RequestOptions requestOptions, Func`2 task, Func`2 openTelemetry, TraceComponent traceComponent, TraceLevel traceLevel)
   at GTA__Tests.ItemTests.Item_Tests.SaveAMyClassDocumentToCosmosDb_ReturnsOK() in C:\Users\pjsta\source\repos\GosportTaxiApp\GTA__Tests\ItemTests\Item_Tests.cs:line 142

Answer 1

Your problem is that you are not implementing the "$type" property correctly. It isn't a property you add yourself, it's a synthetic property that Json.NET inserts when you set TypeNameHandling. This matters because UpsertItemAsync<T> is a round-trip operation: it serializes your MyClass, upserts it to azure, then deserializes the upserted result. Your implementation fails during deserialization.

To enable TypeNameHandling correctly for your MyClass.Commands list, remove all the CommandType properties and apply the attribute JsonPropertyAttribute.ItemTypeNameHandling like so:

public interface IMyCommand
{
}

public class MyCommand1 : IMyCommand
{
}

public class MyCommand2 : IMyCommand
{
}

//partition key is CompanyId
public class MyClass
{
    [JsonProperty("id")]
    public Guid Id { get; set; }

    public string CompanyId { get; set; }

    [JsonProperty(ItemTypeNameHandling = TypeNameHandling.Auto)] // Add this
    public List<IMyCommand> Commands { get; set; }

    public MyClass(string partitionKey)
    {
        CompanyId = partitionKey;
    }
}

Demo fiddle #1 here.

Now, it also seems as though you want to control the value of the "$type" properties, possibly to avoid the security risks related to TypeNameHandling. Under normal circumstances, one would create a custom serialization binder such as the following:

public class KnownTypesSerializationBinder : ISerializationBinder
{
    Dictionary<Type, string> typesToNames;
    Dictionary<string, Type> namesToTypes;
    
    public KnownTypesSerializationBinder(IEnumerable<KeyValuePair<string, Type>> namesToTypes) =>
        (this.namesToTypes, this.typesToNames) = (namesToTypes.ToDictionary(t => t.Key, t => t.Value), namesToTypes.ToDictionary(t => t.Value, t => t.Key));

    public Type BindToType(string assemblyName, string typeName) => namesToTypes[typeName];

    public void BindToName(Type serializedType, out string assemblyName, out string typeName) => (assemblyName, typeName) = (null, typesToNames[serializedType]);
}

Then one would create JsonSerializerSettings as follows:

var namesToTypes = new KeyValuePair<string, Type>[] 
{
    new(nameof(MyCommand1), typeof(MyCommand1)),
    new(nameof(MyCommand2), typeof(MyCommand2)),
};
var settings = new JsonSerializerSettings
{
    SerializationBinder = new KnownTypesSerializationBinder(namesToTypes),
};

However, azure-cosmos-dotnet-v3 seems to be designed to allow for Json.NET to be replaced with System.Text.Json (or any other serializer) in the future. As such, the type CosmosJsonDotNetSerializer has been made internal and CosmosSerializationOptions does not have any way to directly set JsonSerializerSettings. If you want to configure the serialization binder, I can imagine the following possibilities:

1. Implement your own version of CosmosJsonDotNetSerializer with the necessary settings and inject it via CosmosClientOptions.Serializer.

   For an example of this approach, see e.g. this answer by Ankit Vijay to Serialize and Deserialize Cosmos DB document property as string.

2. Configure the binder indirectly by adding a converter to a parent type that configures JsonSerializer.SerializationBinder then continues with a default (de)serialization.

3. Abandon TypeNameHandling and create a custom converter for IMyCommand that implements its own custom type discriminator. For examples see Json.Net Serialization of Type with Polymorphic Child Object or Deserializing polymorphic json classes without type information using json.net.

Adopting the second approach, you will need to add the following converters, binders and associated extension methods:

internal class MyCommandSerializationBinderSettingConverter : KnownTypesJsonConverter<object>
{
    public MyCommandSerializationBinderSettingConverter() : base(new KeyValuePair<string, Type>[] 
        {
            new(nameof(MyCommand1), typeof(MyCommand1)),
            new(nameof(MyCommand2), typeof(MyCommand2)),
        }) { }
}

public class KnownTypesJsonConverter<TValue> : RecursiveConverterBase<TValue>
{
    readonly ISerializationBinder binder;
    
    public KnownTypesJsonConverter(IEnumerable<KeyValuePair<string, Type>> namesToTypes) => this.binder = new KnownTypesSerializationBinder(namesToTypes);
    public KnownTypesJsonConverter(ISerializationBinder binder) => this.binder = binder;

    protected override void WriteJsonWithDefault(JsonWriter writer, TValue value, JsonSerializer serializer)
    {
        var old = serializer.SerializationBinder;
        try
        {
            serializer.SerializationBinder = binder;
            base.WriteJsonWithDefault(writer, value, serializer);
        }
        finally
        {
            serializer.SerializationBinder = old;
        }
    }
    
    protected override TValue ReadJsonWithDefault(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer) 
    {
        var old = serializer.SerializationBinder;
        try
        {
            serializer.SerializationBinder = binder;
            return base.ReadJsonWithDefault(reader, objectType, existingValue, serializer);
        }
        finally
        {
            serializer.SerializationBinder = old;
        }
    }
}

public abstract class RecursiveConverterBase<TValue> : JsonConverter
{
    static readonly ThreadLocal<Stack<Type>> typeStack = new (() => new Stack<Type>());

    bool IsSuspended => typeStack.IsValueCreated && typeStack.Value.TryPeek(out var type) && type == GetType();

    public override bool CanRead => !IsSuspended;
    public override bool CanWrite => !IsSuspended;

    public override bool CanConvert(Type objectType) => !IsSuspended && typeof(TValue).IsAssignableFrom(objectType); // Or typeof(TValue) == objectType if you prefer.

    protected virtual void WriteJsonWithDefault(JsonWriter writer, TValue value, JsonSerializer serializer)
    {
        Console.WriteLine(value);
        serializer.Serialize(writer, value, typeof(TValue));
    }

    public sealed override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
    {
        using (var pushValue = typeStack.Value.PushUsing(GetType()))
        {
            WriteJsonWithDefault(writer, (TValue)value, serializer);
        }
    }

    protected virtual TValue ReadJsonWithDefault(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer) => (TValue)serializer.Deserialize(reader, objectType);

    public sealed override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
    {
        using (var pushValue = typeStack.Value.PushUsing(GetType()))
        {
            return ReadJsonWithDefault(reader, objectType, existingValue, serializer);
        }
    }
}

public static class StackExtensions
{
    public class PushValue<T> : IDisposable
    {
        readonly Stack<T> stack;
        readonly int count;

        public PushValue(T value, Stack<T> stack)
        {
            this.stack = stack;
            this.count = stack.Count;
            stack.Push(value);
        }

        // By using a disposable struct we avoid the overhead of allocating and freeing an instance of a finalizable class.
        public void Dispose()
        {
            if (stack != null)
            {
                while (stack.Count > count)
                    stack.Pop();
            }
        }
    }

    public static PushValue<T> PushUsing<T>(this Stack<T> stack, T value)
    {
        if (stack == null)
            throw new ArgumentNullException();
        return new PushValue<T>(value, stack);
    }
}

public class KnownTypesSerializationBinder : ISerializationBinder
{
    Dictionary<Type, string> typesToNames;
    Dictionary<string, Type> namesToTypes;
    
    public KnownTypesSerializationBinder(IEnumerable<KeyValuePair<string, Type>> namesToTypes) =>
        (this.namesToTypes, this.typesToNames) = (namesToTypes.ToDictionary(t => t.Key, t => t.Value), namesToTypes.ToDictionary(t => t.Value, t => t.Key));

    public Type BindToType(string assemblyName, string typeName) => namesToTypes[typeName];

    public void BindToName(Type serializedType, out string assemblyName, out string typeName) => (assemblyName, typeName) = (null, typesToNames[serializedType]);
}

Then add MyCommandSerializationBinderSettingConverter to the container class MyClass like so:

[JsonConverter(typeof(MyCommandSerializationBinderSettingConverter))] // Add this
public class MyClass
{
    [JsonProperty("id")]
    public Guid Id { get; set; }

    public string CompanyId { get; set; }
    
    [JsonProperty(ItemTypeNameHandling = TypeNameHandling.Auto)] // Add this
    public List<IMyCommand> Commands { get; set; }

    public MyClass(string partitionKey)
    {
        CompanyId = partitionKey;
    }
}

And you will be able to serialize your IMyCommand polymorphic type hierarchy when contained inside a MyClass object using Json.NET with custom type names without needing to configure JsonSerializerSettings directly.

Second Json.NET demo fiddle here.

---

As an aside, if azure-cosmos-dotnet-v3 or its successor ever does transition to System.Text.Json, you will need to re-implement your classes as follows. First, add JsonDerivedTypeAttribute attributes to IMyCommand for each concrete implementation:

[System.Text.Json.Serialization.JsonDerivedType(typeof(MyCommand1), nameof(MyCommand1))]
[System.Text.Json.Serialization.JsonDerivedType(typeof(MyCommand2), nameof(MyCommand2))]
public interface IMyCommand
{
}

Second, fix the MyClass constructor so that the constructor arguments have the same case-invariant names as the corresponding properties:

public class MyClass
{
    [System.Text.Json.Serialization.JsonPropertyName("id")]
    public Guid Id { get; set; }

    public string CompanyId { get; set; }

    public List<IMyCommand> Commands { get; set; }

    public MyClass(string companyId) // Change the argument name to match the property name
    {
        CompanyId = companyId;
    }
}

This looks simpler than the Json.NET implementation because System.Text.Json's support for polymorphism only allows for whitelisted types with specified type discriminator values -- which is exactly the situation you are in.

Demo using System.Text.Json here.