How to clear NiFi queues?

Swati Sood picture Swati Sood · Sep 30, 2016 · Viewed 9.9k times · Source

We are creating some flows in NiFi and there might be some cases where the queues are being build up but due to some reason the flow doesn't work as expected.

At the end of the day, i would like to clear the queues and somehow would like to automate it. The question is how can we delete the queues from backend? Is there any way we can achieve that?

Answer

JDP10101 picture JDP10101 · Sep 30, 2016

In addition to the explicit "Drop Queue" function Bryan mentioned, a couple other features you may be interested are the "Back Pressure" and "FlowFile Expiration" settings on connections. These allow you to automatically control the the amount of data in any given connection. A simple explanation for each is below but for a more detail explanation I'd refer you to the Apache NiFi User Guide

Back Pressure: When a queue gets to a certain size, stop triggering the processor that feeds it.

FlowFile expiration: Once a FlowFile has spent X amount of time in the queue, it will be dropped.