Apache NiFi evaluatejsonpath processor: JSONPath expression to concatenate 2 attributes

Question

I am using EvaluateJsonPath processor in NiFi to specify a composite primary key for writing my JSON data into elasticsearch. I have to create an attribute called 'key' by concatenating two attributes, lets say 'attr1' and 'attr2'. In EvaluateJsonPath configuration, I added a property 'key' with value '${attr1}${attr2}' (I'm using this key to avoid redundancy in elasticsearch). But I'm getting an error like : 'key' is invalid because specified expression was not valid: ${attr1}${attr2}.

What is the proper syntax for concatenating 2 attributes in a json record using EvaluateJsonPath processor?

Answer 1

I assume you mixed the term attributes with different contexts. I did that when I picked up NiFi for the first time and started working with JSON. To get things clarified, there are two types of attributes in this particular case:

1. FlowFile attributes - These are the attributes that are sort of like metadata of the FlowFile, like filename, filesize, mime.type, etc. When you add an attribute using a processor like UpdateAttribute they get added to the metadata of the FlowFile i.e. FlowFile attributes not to the FlowFile content itself.
2. JSON attributes - These are the fields that are in the FlowFile content. Ex: Name is an attribute here -> { "Name" : "Smith, John"}

Having said that, using expressions like ${projectName}, ${filename} are evaluated against the FlowFile attributes. This syntax is called NiFi Expression Language. they won't be evaluated against the content of the FlowFile. EvaluateJsonPath takes a JSON path expression like $.Name

So for example, lets assume you have the following JSON document in the FlowFile content:

{
    "attr1": "some value for attr1",
    "attr2": "some value for attr2"
}

In this case, you would have to configure the EvaluateJsonPath processor with new properties which look like the following:

attribute1 : $.attr1
attribute2 : $.attr2

Once done, flowfiles that pass through the EvaluateJsonPath will have the above two attributes added to the flowfile attributes which you can read using NiFi Expression like ${attribute1}. Then connect EvaluateJsonPath processor to UpdateRecord processor where you can add the combined composite key with the help of NiFi Expression Language to the FlowFile JSON content.

Useful Links

• NiFi Expression Language guide - https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html
• EvaluateJsonPath samples - https://community.hortonworks.com/questions/106210/how-to-use-evaluate-json-path-in-nifi.html
• UpdateRecord - https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.7.1/org.apache.nifi.processors.standard.UpdateRecord/additionalDetails.html