Mosquitto increasing maximum connection limit
I am trying to increase the maximum open file connection of Mosquitto broker. But I read that increasing concurrent connections are not controlled by Mosquitto only. As per our study we decided for 100k concurrent connection, we are targeting 1.6 GB RAM. But for testing I have to increase from default 1024 connections to 20000 Testing environment configurations: t2. micro AWS server with 64 MB 14.04 ubuntu operating system. Changing connection limit in the mosquitto configuration is not reflecting. What can be the reason? Do we need to change any configuration related to AWS Server?
My configurations:
Our system wide open connections is configured on /etc/sysctl.conf:
fs.file-max =99905
Running the command sysctl -p or cat /proc/sys/fs/file-max is reflecting the changes
In /etc/security/limits.conf:
ubuntu hard nofile 45000
ubuntu soft nofile 35000
Mosquitto is installed under the user 'Ubuntu' .
We also added below line of code on /etc/pam.d/common-session
session required pam_limits.so
Running ulimit -a is giving the below result:
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 7859
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 35000
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 7859
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
My init configuration file for mosquitto in /etc/init/mosquitto.conf
description "Mosquitto MQTTv3.1 broker"
author "Roger Light <[email protected]"
start on net-device-up
respawn
exec /usr/sbin/mosquitto -c /etc/mosquitto/mosquitto.conf
#limit nofile 4096 4096
limit nofile 24000 24000
Below is the configuration in /etc/mosquitto/mosquitto.conf:
# change user to root
user ubuntu
set_ulimit () {
ulimit -f unlimited
ulimit -t unlimited
ulimit -v unlimited
ulimit -n 24000
ulimit -m unlimited
ulimit -u 24000
}
start)
...
# Update ulimit config in start command
set_ulimit
...
;;
stop)
But running cat /proc/4957/limits is still showing default value 1024 open files:
Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 8388608 unlimited bytes
Max core file size 0 unlimited bytes
Max resident set unlimited unlimited bytes
Max processes 7859 7859 processes
Max open files 1024 4096 files
Max locked memory 65536 65536 bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 7859 7859 signals
Max msgqueue size 819200 819200 bytes
Max nice priority 0 0
Max realtime priority 0 0
Max realtime timeout unlimited unlimited us
4957 -is the process id of Mosquitto
Answer
The number of open files is limited by the user limits, see ulimit man page. I set the ulimit -n to 20000, and run mosquitto broker and it shows
% ps ax | grep mosquitto
9497 pts/44 S+ 0:00 ./mosquitto -c mosquitto.conf
9505 pts/10 S+ 0:00 grep --color=auto mosquitto
% cat /proc/9497/limits
Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 8388608 unlimited bytes
Max core file size 0 unlimited bytes
Max resident set unlimited unlimited bytes
Max processes 63084 63084 processes
Max open files 20000 20000 files
Max locked memory 65536 65536 bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 63084 63084 signals
Max msgqueue size 819200 819200 bytes
Max nice priority 0 0
Max realtime priority 0 0
Max realtime timeout unlimited unlimited us
Regardless, since mosquitto is single-threaded, we have not found it useable for anything more than about 1000 publisher clients with a reasonable payload rate at 1 / 10 seconds.