Using a dirty flag with Core Data for synchronizing with a server

Question

I am working on syncing data between the client and a server. I am using MagicalRecord (a Core Data wrapper) for storing the data on the client. I have an entity called Dirty that contains a single attribute called dirty. This represents if there were changes on the client that have not been pushed up to the server yet. dirty gets set to [NSDate date] anytime an attribute is set on the class (of course, when setting dirty, the correct value is set). Every other entity created on the client inherits from Dirty. The idea is that we won’t fetch new data from the server until all client data has been pushed up (only fetch new data if all entities have dirty == nil).

When importing data from the server (using +[NSManagedObject MR_importFromObject:inContext:), each entity’s dirty attribute gets set to nil (because the client is up-to-date with the server).

Right before a save is kicked off (inside the +[MagicalRecord saveWithBlock:completion:] save block), dirty is still nil. However, in the completion block, fetching the entity that was just saved (on the main thread) has a value for dirty.

During the save, the entity is transferred to the main thread’s context. However, there is a problem because -[NSManagedObject didChangeValueForKey:] gets called for every attribute that is transferred from the localContext (background thread) to the main context (on the main thread). dirty gets set with [NSDate date] for each entity. Most of the time, dirty does not get set last, which means when another attribute is set, dirty is overwritten.

Is there a way to make sure dirty is the last attribute that is set when transferring the NSManagedObject instance to the main thread’s context? I’m even willing to set dirty when the object is saved (instead of when the properties are set).

I’ve tried all sorts of options, including checking against -[NSManagedObject isInserted] and -[NSManagedObject isUpdated] inside -[NSManagedObject didChangeValueForKey:]. One other thing that is kind of annoying is that the new object is inserted before the attributes are transferred over (I thought I could have some sort of flag to lock/unlock setting dirty).

Another thing to note is [NSManagedObject(_NSInternalMethods) _updateFromRefreshSnapshot:includingTransients:] is what gets called right before -[NSManagedObject didChangeValueForKey:] is called on the new object.

Any ideas? I’ve been facepalming over this for a couple days now.

score 1 · Answer 1 · edited May 23 '17 at 12:06

1

Have a look at the answer from Paul de Lange in SO 10723861

The 'TrackedEntity' over there would be your Dirty entity and the attribute lastModified would translate to your attribute 'dirty'

During a save (triggered by observing the NSManagedObjectContextWillSaveNotification) the -objectContextWillSave method will merge the inserted objects and the updated objects into a set. Then it goes through the set of object and updates the lastModified attribute with the timestamp.

--- Update (wrt. clientUpdatedAt)

You might want to look at this one, too. It explains on how to use some extra fields to assist with synchronization. Using an extra attribute sync_status should help in identifying if a entity needs to be uploaded. Hope that helps

edited May 23 '17 at 12:06

Community

1
1

answered Oct 31 '14 at 11:05

Olaf

3,042
1
16
26

So I am already doing this with `clientUpdatedAt` (and `clientCreatedAt`). `clientUpdatedAt` is not a good enough indication because when the server updates the entity on the client with the server's `updatedAt`time, the entity has to save and `clientUpdatedAt` is updated with `+[NSDate date]` once again. If I check if `clientUpdatedAt` is more recent than `updatedAt`, it will return `YES` every time. :/ – Jeff Oct 31 '14 at 12:21
I have updated my answer, you might need to introduce more attributes to handle synchronization – Olaf Oct 31 '14 at 13:17
Thanks for updating your answer. The challenge is this step: `set sync_status to 1 on your model object whenever something changes and needs to be synchronized to the server`. This is the same challenge I am having in my code. I tried using `didChangeValueForKey:` but this wasn't reliable for the reasons I mentioned in my original post. Do you have any suggestions for how to accomplish this step? Once this is figured out, everything else will be smooth sailing. – Jeff Oct 31 '14 at 13:33
I'm thinking about holding onto the dirty value inside the save block, and in the completion block making sure the values are the same. If not, then I will set them again and just save them on the main context. This is getting pretty nasty though. – Jeff Oct 31 '14 at 13:59

Jeff · Accepted Answer · 2014-10-31T15:43:34.067

After wasting a ton of time on potential solutions, I went back to take a look at solving this issue in the simplest way possible. Here is what I came up with:

1) Deleted -[Dirty didChangeValueForKey:]

2) Created a BTCoreDataService class, and added the following methods:

+ (void)saveClientChangesWithSaveBlock:(BTLocalManagedObjectContextBlock)saveBlock
                       completionBlock:(MRSaveCompletionHandler)completionBlock {
    [self saveAndSetDirty:[NSDate date] 
                saveBlock:saveBlock 
          completionBlock:completionBlock];
}

+ (void)saveServerChangesWithSaveBlock:(BTLocalManagedObjectContextBlock)saveBlock
                       completionBlock:(MRSaveCompletionHandler)completionBlock {
    [self saveAndSetDirty:nil 
                saveBlock:saveBlock 
          completionBlock:completionBlock];
}

#pragma mark - Internal

+ (void)saveAndSetDirty:(NSDate *)dirty
              saveBlock:(BTLocalManagedObjectContextBlock)saveBlock
        completionBlock:(MRSaveCompletionHandler)completionBlock {
    [MagicalRecord
     saveWithBlock:^(NSManagedObjectContext *localContext) {
         if (saveBlock) {
             saveBlock(localContext);

             [[localContext BT_insertedAndUpdatedAtObjectsKindOfClass:[Dirty class]]
              makeObjectsPerformSelector:@selector(setDirty:) withObject:dirty];
         }
     }
     completion:completionBlock];
}

And here is the implementation for NSManagedObjectContext+BTManagedObjectContext:

- (NSSet *)BT_insertedObjectsKindOfClass:(Class)cls {
    return
    [self.insertedObjects
     filteredSetUsingPredicate:[self BT_isKindOfClassPrediate:cls]];
}

- (NSSet *)BT_updatedObjectsKindOfClass:(Class)cls {
    return
    [self.updatedObjects
     filteredSetUsingPredicate:[self BT_isKindOfClassPrediate:cls]];
}

- (NSSet *)BT_insertedAndUpdatedAtObjectsKindOfClass:(Class)cls {
    return
    [[self BT_insertedObjectsKindOfClass:cls]
     setByAddingObjectsFromSet:[self BT_updatedObjectsKindOfClass:cls]];
}

#pragma mark - Internal

- (NSPredicate *)BT_isKindOfClassPrediate:(Class)cls {
    return [NSPredicate predicateWithFormat:@"self isKindOfClass:%@", cls];
}

The only thing now is to remember to use BTCoreDataService to save objects instead of using MagicalRecord directly.

Using a dirty flag with Core Data for synchronizing with a server

2 Answers2