Got duplicated data when subscribe multiple times

Question

I am using MongoDB aggregation in meteor. I got duplicated data when subscribe multiple times.

(The data in database are static, which means they are same all the time.)

// Server side

Meteor.publish('totalNumber', function () {
  let pipeline = [
    { $unwind: '$product' },
    { $group: {
      _id: {
        code: '$product.code',
        hour: { $hour: '$timestamp' }
      },
      total: { $sum: '$product.count' },
    }}
  ];

  Products.aggregate(
    pipeline,
    Meteor.bindEnvironment((err, result) => {
        console.log('result', result);  // at here every time subscribe, no duplicated data
        _.each(result, r => {
          this.added('totalNumber',
            // I use Random.id() here, because "Meteor does not currently support objects other than ObjectID as ids"
            Random.id(), {
              code: r._id.code,
              hour: r._id.hour,
              total: r.total
          });
        });
      }
    )
  );

  this.ready();
});

// Client side

this.subscribe('totalNumber', () => {
  // Correct result: [Object, Object] for example
  console.log(Products.find().fetch());
}, true);

this.subscribe('totalNumber', () => {
  // Wrong result: [Object, Object, Object, Object]
  console.log(Products.find().fetch());
}, true);

this.subscribe('totalNumber', () => {
  // Wrong result: [Object, Object, Object, Object, Object, Object]
  console.log(Products.find().fetch());
}, true);

So right now basically, the new results always include last time subscribe data.

How can I solve this problem? Thanks

You are calling `this.added` with `r._id`. Is `r._id`randomly generated, or will the same set of ids be produced each time the pipline is run? — David Weldon, Mar 15 '16 at 23:00
Sorry, u are right, I updated my question, I actually used `Random.id()` in my code, so they are different. — Hongbo Miao, Mar 15 '16 at 23:01
Yeah that's your problem. It's like telling the client "I added `{_id: 'a', total: 10}`" and then later telling it "I added `{_id: 'b', total: 10}`", and expecting there to be be only one object. The client will determine uniqueness by the value in the `_id` field, but if it's random, all of your documents will appear to be unique. I'd write this as an answer but I don't know what to suggest without knowing more about what you are trying to solve. — David Weldon, Mar 15 '16 at 23:14
@DavidWeldon I added more details in my question. I use `Random.id()` there, because `Meteor does not currently support objects other than ObjectID as ids`, so is there any other way to solve it? thanks — Hongbo Miao, Mar 15 '16 at 23:22

David Weldon · Accepted Answer · 2016-03-16T00:04:01.850

The problem is that you are using a random id each time in the call to added so the client always thinks all of the documents are unique. You need to devise a consistent id string generator. Using an answer to this question, you could imagine building a set of functions like these:

hashCode = function (s) {
  return s.split('').reduce(function (a, b) {
    a = ((a << 5) - a) + b.charCodeAt(0);return a & a;
  }, 0);
};

objectToHash = function (obj) {
  return String(hashCode(JSON.stringify(obj)));
};

So if you wanted a unique document for each combination of code and hour you could do this:

var id = objectToHash(r._id);
this.added('totalNumber', id, {...});

Got duplicated data when subscribe multiple times

1 Answers1

Linked