Remove all flow file attributes except the ones explicitly defined

Question

I want to remove all attributes from a given flow file except the ones I explicitly define to keep.

Given the following sample flow file attributes:

name: aaa
Place: bbb
Host: ccc
JsonAttribute: {
    "A": "a",
    "B": "b"
}
data: ddd

And I want to keep Host and JsonAttribute only.

Thus, the resulting flow file attributes should be:

Host: ccc
JsonAttribute: {
    "A": "a",
    "B": "b"
}

How can I achieve this using the standard processors NiFi provides?

renaming the attributes

Is it possible to rename the attributes using the same procedure? E.g. I would like to keep the attributes like above but rename JsonAttribute into customName

Endzeit · Accepted Answer · 2019-11-19T18:58:51.323

The UpdateAttribute processor does not only allow to add / set flow file attributes but also enables you to delete existing attributes from a flow file.

To do so you have to pass a regular expression matching all attributes you want to remove to the property Delete Attributes Expression.

In your case you can use a negative look-around to match every attribute other than the ones you want to keep.

^((?!Host)(?!JsonAttribute).)*$

So no additional processor is needed.

keeping Host attribute, moving JsonAttribute to a different attribute name

You should be able to also achieve the second behaviour using only a single UpdateAttribute processor.

Simply add a property, e.g. customName, to the processor and reference the old attribute using the NiFi Expression Language:

${JsonAttribute}

In that case you may also simplify your deletion regex to also delete the JsonAttribute flow file attribute:

^((?!Host).)*$

score 1 · Answer 2 · answered Nov 18 '19 at 07:31

1

AttributesToJSON Processor can filter data you don't need.You can set Attributes Regular Expression in this Processor.

answered Nov 18 '19 at 07:31

Cong

479
4
16

is there any way to customize the json name i.e.,instead of JsonAttributes: {"A":"a","B":"b"} --> customName:{"A":"a","B":"b"} – Prabhanj Nov 18 '19 at 09:20

score 1 · Answer 3 · answered Nov 18 '19 at 07:39

1

In the invokehttp processor there is an Attributes to Send property where you could define attributes to be used in request.

So you don't need to filter out attributes in another processor.

answered Nov 18 '19 at 07:39

daggett

26,404
3
40
56

by setting Attributes to Send --> JsonAttribute and given a try, but i am getting 400 bad request... – Prabhanj Nov 18 '19 at 09:19
That's another question. Read about httprequest - it sends to server flow file content as a body and attributes as http headers. – daggett Nov 18 '19 at 10:01

Remove all flow file attributes except the ones explicitly defined

renaming the attributes

3 Answers3

keeping Host attribute, moving JsonAttribute to a different attribute name