mongoose: sort and paginating the field inside $project

Question

$project: {
  _id: 1,
  edited: 1,
  game: {
    gta: {   
      totalUserNumber: {
        $reduce: {
          input: "$gta.users",
          initialValue: 0,
          in: { $add: [{ $size: "$$this" }, "$$value"] },
        },
      },
      userList: "$gta.users", <----- paginating this
    },
    DOTA2: {
        totalUserNumber: {
          $reduce: {
            input: "$dota2.users",
            initialValue: 0,
            in: { $add: [{ $size: "$$this" }, "$$value"] },
          },
        },
        userList: "$dota2.users", <------ paginating this
      },
    },
   .... More Games
  },

I have this $project. I have paginated the list of games by using $facet,$sort, $skip and $limit after $project.

I am trying also trying to paginate each game's userList. I have done to get the total value in order to calculate the page number and more.

But, I am struggling to apply $sort and $limit inside the $project. So far, I have just returned the document and then paginated with the return value. However, I don't think this is very efficient and wondering if there is any way that I can paginate the field inside the $project.

Is there any way that I can apply $sort and $limit inside the $project, in order to apply pagination to the fields and return?

------ Edit ------

this is for paginating the field. Because, I am already paginating the document (game list), I could not find any way that I can paginate the field, because I could not find any way that I can apply $facet to the field.

e.g. document

[
   gta: {
     userID: ['aa', 'bb', 'cc' ......],
   },
   dota: {
     userID: ['aa', 'bb', 'cc' ......],
   }
   ....
]

I am using $facet to paginate the list of games (dota, gta, lol and more). However, I did not want to return all the userID. I had to return the entire document and then paginate the userID to replace the json doc.

Now, I can paginate the field inside the aggregate pipeline by using $function.

thanks to Mongodb sort inner array !

const _function = function (e) {
    e // <---- will return userID array. You can do what you want to do.
    return {
     
    };
  };
   game
    .collection("game")
    .aggregate([
      {},
      {
        $set: {
          "game": {
            $function: {
              body: _function,
              args: ["$userID"],
              lang: "js",
            },
          },
        },
      },
    ])
    .toArray();

By using $function multiple time, you will be able to paginate the field. I don' really know if this is faster or not tho. Plus, make sure you can use $function. I read that you can't use this if you are on the free tier at Atlas.

score 1 · Accepted Answer · answered Jun 23 '21 at 16:38

1

What you are looking for is the $slice Operator.

It requires three parameters.

"$slice": [<Array>, <start-N>, <No-Of.elements to fetch>]

userList: {"$slice": ["$dota2.users", 20, 10]}  // <-- Will ignore first 20 elements in array and gets the next 10

answered Jun 23 '21 at 16:38

hhharsha36

3,089
2
12
12

1

P.S. Huge Dota fan here – hhharsha36 Jun 23 '21 at 16:39
How can I sort the sliced result with the alphabetical order? – chichi Jun 23 '21 at 17:38
1

I am there is no other way than to `$unwind`, `$sort` and then `$group`. Have a look at the `$facet` pipeline stage for a more optimized way, I can't give example since it's very application-specific. – hhharsha36 Jun 23 '21 at 17:55
1

I see. thanks mate anyway! Enjoy playing Dota! – chichi Jun 23 '21 at 18:00
Buddy. I've found the way. It is kinda work around. By using `$function`, I can use javascript. So, I merged everything into one array and then replaced the field by `$set`. Then using the filter to filter things out. Again with `$set`, I can paginate with the returned array and JS. – chichi Jun 24 '21 at 16:54
1

Wow, amazing bud. Please share the code snippet or link, so that I can learn. Thanks a lot for sharing, you are Awesome! – hhharsha36 Jun 24 '21 at 16:58
I just added in the question. I do not know if this will perform faster! – chichi Jun 24 '21 at 17:24
1

I have gone through the `$function` and `$where` documentation. It's a bit slow, but there is no other way of doing it (at least what I am aware of) – hhharsha36 Jun 24 '21 at 17:53
Do you think paginating the returned document will be faster? – chichi Jun 24 '21 at 18:52
1

@GIHYUNNAM Yes, Paginating the Result is much faster than performing the Aggregation every time. The reason is that for each Aggregation request, MongoDB performs the aggregation from scratch and won't reuse the results from the previous aggregation. But, If you perform pagination on your programming language side, depending on the language you are using, it's going to be heavy on RAM and CPU usage, so do keep this in mind. – hhharsha36 Jun 25 '21 at 04:20
Oh. I'd thought it was the other way around. I was so happy to find the `$function` so I can replace my post procssing node.js code lol. Learning new things everyday! Just one more question (sorry to bother you). If post processing is faster, what is the reason why people use aggregate()? – chichi Jun 25 '21 at 15:36
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/234205/discussion-between-hhharsha36-and-gi-hyun-nam). – hhharsha36 Jun 25 '21 at 16:11

mongoose: sort and paginating the field inside $project

1 Answers1