Newtonsoft Json Serialization and Deserialization Issue (Self referencing loop detected for property)

Question

I have a many to many Relation between Course and Student Entities some Weird Behavior Happens When i try to Get Courses for the options

• in Edit Screen, I Get the Self referencing loop detected for property Course
• In Add Screen , it works Fine With No Issue

I tried Looking For the cause but did not come up with anything useful.

the Relation is :- 1- Course can have Many Students 2- Student can have many course

The tables are 1- Course 2- StudentCourse (With CourseId as Foreign Key, StudentId as Foreign Key) 3- Student

please note that the person table has the same exact scenario but , I do not face the Same issue There

    public class Repository<TEntity>:IRepository<TEntity> where TEntity : class
    {
        protected readonly DbContext Context;

        public Repository(DbContext context)
        {
            Context = context;
        }
        public async Task<List<T>> GetAllAsync<T>(Expression<Func<TEntity, bool>> predicate)
        {
            var q = Context.Set<TEntity>().Where(predicate);
            var ret = await q.ToListAsync();
            List<T> result = JsonConvert.DeserializeObject<List<T>>(JsonConvert.SerializeObject(ret));
            return result;
        }
   }

the Line Causing The Exception

List<T> result = JsonConvert.DeserializeObject<List<T>>(JsonConvert.SerializeObject(ret));

Here Is the Stack Trace.

at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.CheckForCircularReference(JsonWriter writer, Object value, JsonProperty property, JsonContract contract, JsonContainerContract containerContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.CalculatePropertyValues(JsonWriter writer, Object value, JsonContainerContract contract, JsonProperty member, JsonProperty property, JsonContract& memberContract, Object& memberValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(JsonWriter writer, Object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeValue(JsonWriter writer, Object value, JsonContract valueContract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeList(JsonWriter writer, IEnumerable values, JsonArrayContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeValue(JsonWriter writer, Object value, JsonContract valueContract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(JsonWriter writer, Object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeValue(JsonWriter writer, Object value, JsonContract valueContract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeList(JsonWriter writer, IEnumerable values, JsonArrayContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeValue(JsonWriter writer, Object value, JsonContract valueContract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.Serialize(JsonWriter jsonWriter, Object value, Type objectType)
   at Newtonsoft.Json.JsonSerializer.SerializeInternal(JsonWriter jsonWriter, Object value, Type objectType)
   at Newtonsoft.Json.JsonSerializer.Serialize(JsonWriter jsonWriter, Object value, Type objectType)
   at Newtonsoft.Json.JsonConvert.SerializeObjectInternal(Object value, Type type, JsonSerializer jsonSerializer)
   at Newtonsoft.Json.JsonConvert.SerializeObject(Object value, Type type, JsonSerializerSettings settings)
   at Newtonsoft.Json.JsonConvert.SerializeObject(Object value)
   at ApheliateProgram.Data.DataAccessLayer.Repositories.Repository`1.<GetAllAsync>d__6`1.MoveNext()

Answer 1

Adding ReferenceLoopHandling.Ignore option Seems to fix the problem.

JsonConvert.SerializeObject(ert, new JsonSerializerSettings{ ReferenceLoopHandling = ReferenceLoopHandling.Ignore})

thanks @Guru Stron

Answer 2

My advice to avoid pain like this in the present and the future, do away with the Generic Repository.

This method looks to be an attempt to use lazy loading to fetch an entire object graph in a generic way using a serializer to do a deep copy, or you are stumbling on a side-effect of either a lazy load serializer behaviour or happen to have some related entities already pre-fetched by the DbContext and that is tripping up your serialization.

You can avoid all manner of issues like this by leveraging projection to populate a view model for consumption. However this is not a Generic compatible operation, but it keeps the code simple and fast.

Where I use repositories I have them pass back IQueryable<TEntity> rather than IEnumerable<TEntity> or Task<IEnumerable<TEntity>> as this gives the callers full control over how the data is consumed including projection, pagination, sorting, additional filtering, as well as whether the call should be asynchronous or synchronous. The same goes for calls where I do want to work with entities (such as doing Updates) the caller control over whether and what related data is eager loaded.

To help explain the issue you are encountering with the Generic implementation:

By loading Student data using what amounts to _context.Set<Student>() that can fetch a Student record. However a Student record contains references to Courses, and Courses contain references back to Student. (Including the record for this student which amounts to a circular reference)

With lazy loading enabled, the serializer is going to "touch" Courses and proceed to fetch all related courses. Then as it goes through each course, it is going to "touch" the Students for that course, then for each Student, touch courses... You can limit the depth, but also have to account for any circular references. Even without running into errors this gets extremely expensive as each "touch" results in another query getting run against the database.

Even with lazy loading disabled, when you fetch a Student, the DbContext will look through any courses it might happen to be tracking and it will automatically add references to those courses in the Student when it is returned. Where you have a reference to a Course that refers back to this same student, the serializer can trigger an exception on finding the circular reference. This also leads to incomplete and unpredictable related data being sent to your view / consumer as the DbContext will fill in anything it knows about which might be all related data, some related data, or no related data.

Instead, if I have a StudentRepository that returns IQueryable<Student> you get something like this:

public class StudentRepository
{
    public IQueryable<Student> GetStudents()
    {
        var query = Context.Students.AsQueryable();
        // or could use:
        //var query = Context.Set<Student>().AsQueryable();
        return query;
    }
}

You don't need the predicate, as the caller is composing that anyways. What filtering the repository can do is low level rules that you want to ensure are done consistently, such as if you have a soft-delete model (I.e. IsActive):

    public IQueryable<Student> GetStudents(bool includeInactive = false)
    {
        var query = Context.Students.AsQueryable();
        if(!includeInactive)
            query = query.Where(x => x.IsActive);

        return query;
    }

This ensures that by default only active students are ever returned. The same can be used to apply ownership checks against the current user to ensure the data returned is only that which they are allowed to see. (Such as in the case of a multi-tenant SaaS system)

The caller that wants the student data:

var students = StudentRepository.GetStudents()
    .Where(x => {insert where conditions})
    .OrderBy(...)
    ....

From here we can Skip, Take, ToList, Any, Count, etc. as well as using Include to eager load data if we want to work with the entities themselves. All of this adds considerable complexity to support with IEnumerable. I can call synchronous methods like ToList() or await asynchronous calls without doubling my effort in the repository or forcing everything to use one or the other.

By using IQueryable we are not "leaking" domain knowledge any more than using a more restrictive Generic Repository pattern because the act of composing details like that predicate requires knowledge of the domain and EF-isms as the passed predicate must conform to what EF can work with. (No calling methods, accessing non-mapped properties, etc.)

From there you can leverage AutoMapper's ProjectTo to project the IQueryable result to a ViewModel or DTO containing just the data the consumer needs that can be safely serialized without worrying about circular references or triggering lazy loads. Alternatively the projection can be done manually with Select. It avoids a lot of issues and provides a significant performance and resource utilization improvement as well.