I try to understand some aspects of AMQP protocol. Currently I have project with RabbitMQ and use python pika library. So question is about acknowledgement and message prefetching.
Consider we have a queue with only consumer (for sure this queue was declared as exclusive). So do I understand correctly: no matter if I consume with or with no ack flag? Anyway I should not be able to process several messages simultaneously and there are no another consumers that could take some of other still queued messages. Even better not to turn acknowledgement on, because possibly this may reduce AMQP server load.
Prefetch count doesn't mean anything if there is no acknowledgement. Correct?
I am not sure how prefetching works. I have a callback on new message and in its finally statement I acknowledge or reject the message. This is the only function and no matter how big prefetch count would be - anyway another message would not be processed until the current one is completed. So why should I need change prefetch_count value?
thanks in advance.
With autoack
flag unset, if your application failed during message processing all received messages will be lost. If such situation is quite rare and message lose is appropriate option in your application (for example, but no limited to, logs processing) you may turn autoack off.
And yes, having autoack
unset requires simpler broker logic, so it utilize less resources.
As to prefetch count
(and prefetch size
), these options tell broker how large maybe the payload that sent to client in advance to single message. Usually it is used to save time on network operations to wait for a new messages. When prefetch size
used client will receive one or more messages that has total size equal or less to preset pretch size (and/or count, which is less).
Both prefetch count
and prefetch size
rules is they set are applied. When one of them is set to zero (unset) it will not be applied.
The most important thing, prefetching defines behavior to send messages in advance of more message that client has unacked.
In conjunction these two settings produce something like this:
Prefetch with messages count limit and sending messages in advance:
Conditions:
prefetch-size=5kb, prefetch-count=4
off
The workflow:
prefetch-count=4
) to client. 4 messages will be marked as un-acked and moved out from queue (so they wont be delivered to other clients).Prefetch with large messages:
Conditions:
prefetch-size=5kb, prefetch-count=2
off
The workflow:
prefetch-size=5kb
) to client and that message marked as un-aked.prefetch-count=2
, note, that only first message was 5kb, the rest are 1kb) and these messages marked as un-acked.With auto-ack:
Conditions:
prefetch-size=5kb, prefetch-count=10
on
The workflow:
prefetch-size
and prefetch-count
ignored when no-ack
set to true (that is how auto-ack functionality called in RabbitMQ and in AMQP docs), messages will be sent to client one-by-one and removed from queue after successful sending.Note, that AMQP has asynchronous architecture, so under certain conditions two clients MAY receive single message at the same time. Also un-acked message MAY be delivered to the same client back (especially if you have single client).
Also, look at prefetch-size
and prefetch-count
official documentation and experiment a bit with these options.
P.S.: autoack
is basically no-ack
AMQP flag set to true
.