NiFi: UpdateAttribute to change filename
I use a GetHTTPS--> UpdateAttribute --> PutHDFS flow in order to read json files from an API every lets say 30 secs and put the files in HDFS. The 2nd step changes the filename property with the current date/timestamp so that we dont have same filename conflicts.
I have tried so far:
${filename: prepend(${now():format("yyyy-MM-dd-HH:mm:ss")})}
which leads to:
ERROR PutHDFS Failed to write to HDFS due to java.lang.IllegalArgumentException: java.net. URISyntaxException: Relative path in absolute URI: .2017-08-01-11:01:13-filename.json
I am not sure where this error comes from to be honest, plus there is a dot (.) before the date in the created filename in the error message that should not be there according to the expression used to prepend the filename. Without any filename manipulations in the 2nd step everything works as it should. Any help is greately appreciated, thanks in advance!
Answer
Your issue shows that your filename contains some invalid characters that may be '.' or ':'.
You have to use below expression for store the milliseconds with filename.
${filename:prepend(${now():toNumber()})}
toNumber which converts date into milliseconds.
or you can store like this if you want.
${filename:prepend(${now():format("yyyy-MM-dd-HH-mm-ss")})}
or you can use UUID() to be prepend with filename.
https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#uuid