Include all existing fields and add new fields to document

Question

I would like to define a $project aggregation stage where I can instruct it to add a new field and include all existing fields, without having to list all the existing fields.

My document looks like this, with many fields:

{
    obj: {
        obj_field1: "hi",
        obj_field2: "hi2"
    },
    field1: "a",
    field2: "b",
    ...
    field26: "z"
}

I want to make an aggregation operation like this:

[
    {
        $project: {
            custom_field: "$obj.obj_field1",
            //the next part is that I don't want to do
            field1: 1,
            field2: 1,
            ...
            field26: 1
        }
    },
    ... //group, match, and whatever...
]

Is there something like an "include all fields" keyword that I can use in this case, or some other way to avoid having to list every field separately?

This feature is coming in the next major release 2.6. You can try it on the unstable dev branch - use "$$ROOT" to refer to the entire incoming document. See details in this ticket: https://jira.mongodb.org/browse/SERVER-5916 — Asya Kamsky, Oct 18 '13 at 08:48
This issue is also raised at http://stackoverflow.com/questions/20497499/mongodb-project-retain-previous-pipeline-fields, with some other useful and different answers. — Vince Bowdren, Aug 26 '16 at 08:47

score 239 · Answer 1 · edited Dec 29 '19 at 20:27

239

In 4.2+, you can use the $set aggregation pipeline operator which is nothing other than an alias to $addFieldsadded in 3.4

The $addFields stage is equivalent to a $project stage that explicitly specifies all existing fields in the input documents and adds the new fields.

db.collection.aggregate([
    { "$addFields": { "custom_field": "$obj.obj_field1" } }
])

edited Dec 29 '19 at 20:27

Kaspar Lee

5,446
4
31
54

answered Nov 25 '16 at 21:07

styvane

59,869
19
150
156

3

Any idea how to use it in C# driver? seems it does not exist – Homam Jul 16 '18 at 23:19
I'm not familiar with the C# driver but `$addFields` is new in MongoDB 3.4 which is supported by the [C# driver version 2.5+](https://docs.mongodb.com/ecosystem/drivers/csharp/#mongodb-compatibility) – styvane Jul 17 '18 at 00:59
I have been searching for a way to do this in C# driver, and so far it looks like the way to do it would be using something like `IAggregateFluent.AppendStage(new JsonPipelineStageDefinition("{ $addFields : { myField: 'myValue' }}")` – sboisse Oct 29 '18 at 21:36
7

I discovered that it can even replace a field if you specify the same field name – CME64 Dec 25 '18 at 09:58

score 95 · Answer 2 · answered Jan 02 '15 at 09:01

95

You can use $$ROOT to references the root document. Keep all fields of this document in a field and try to get it after that (depending on your client system: Java, C++, ...)

 [
    {
        $project: {
            custom_field: "$obj.obj_field1",
            document: "$$ROOT"

        }
    },
    ... //group, match, and whatever...
]

answered Jan 02 '15 at 09:01

Deka

1,333
12
14

27

But this will create an embedded doc called `document` an option to created one merged doc would have been nicer... – Alexander Suraphel Sep 01 '16 at 09:25
2

Does anyone know of the solution to pass through all the key value pairs without creating the embedded document? – ugotchi Sep 28 '16 at 09:28
14

Hold your breath. MongoDB 3.4 will come with [$addFields aggregation stage](https://jira.mongodb.org/browse/SERVER-5781) that does just this. See http://stackoverflow.com/a/24557029/4653485. – Jérôme Nov 15 '16 at 13:01
This seems to be the best current solution, as you can (at least in JS) then use the spread operator to rebuild your original object with the new `custom_field` attached. `const newObj = { ...result.document, custom_field: result.custom_field }` – ffritz Mar 13 '20 at 10:46

score 9 · Answer 3 · edited Nov 08 '19 at 08:03

9

To add new fields to your document you can use $addFields

from docs

and to all the fields in your document, you can use $$ROOT

db.collection.aggregate([

{ "$addFields": { "custom_field": "$obj.obj_field1" } },
{ "$group": {
        _id : "$field1",
        data: { $push : "$$ROOT" }
    }}
])

edited Nov 08 '19 at 08:03

Akj

7,038
3
28
40

answered Jul 24 '19 at 09:08

Deeksha Sharma

3,199
1
19
16

He wants all fields in root. This adds all fields under documents array. – Ataberk Feb 23 '22 at 22:52

score 8 · Answer 4 · answered Nov 26 '13 at 10:55

8

>>> There's something like "include all fields" keyword that I can use in this case or some another solution?

Unfortunaly, there is no operator to "include all fields" in aggregation operation. The only reason, why, because aggregation is mostly created to group/calculate data from collection fields (sum, avg, etc.) and return all the collection's fields is not direct purpose.

answered Nov 26 '13 at 10:55

Victoria Malaya

518
2
8

*...because aggregation is mostly created to group/calculate data from collection fields...* Best answer in fact! – felipsmartins Apr 01 '16 at 17:07
1

you have a collection `posts` with _id, title, body, likes fields. The likes field is an array of user _id who like the post. How could you list all posts with all the _id, title, body, likeCount? Returning all fields is a direct purpose in this case. – doc_id Oct 06 '17 at 02:22

score 3 · Answer 5 · answered Aug 21 '14 at 02:49

As of version 2.6.4, Mongo DB does not have such a feature for the $project aggregation pipeline. From the docs for $project:

Passes along the documents with only the specified fields to the next stage in the pipeline. The specified fields can be existing fields from the input documents or newly computed fields.

and

The _id field is, by default, included in the output documents. To include the other fields from the input documents in the output documents, you must explicitly specify the inclusion in $project.

score 0 · Answer 6 · answered Mar 05 '18 at 20:32

according to @Deka reply, for c# mongodb driver 2.5 you can get the grouped document with all keys like below;

var group = new BsonDocument
{
 { "_id", "$groupField" },
 { "_document", new BsonDocument { { "$first", "$$ROOT" } } }
};

ProjectionDefinition<BsonDocument> projection = new BsonDocument{{ "document", "$_document"}};
var result = await col.Aggregate().Group(group).Project(projection).ToListAsync();

// For demo first record 
var fistItemAsT = BsonSerializer.Deserialize<T>(result.ToArray()[0]["document"].AsBsonDocument);

Include all existing fields and add new fields to document

6 Answers6

Linked

Related