Spring Websocket in a tomcat cluster
In our current application, we use Spring Websockets over STOMP. We are looking to scale horizontally. Are there any best practices on how we should handle websocket traffic over multiple tomcat instances and how can we maintain session info across multiple nodes.Is there a working sample that one can refer to?
Answer
Horizontally scaling WebSockets is actually very different than horizontally scaling stateless/stateful HTTP only based applications.
Horizontally Scaling Stateless HTTP app: just spin up some application instances in different machines and put a load balancer in front of them. There are quite a lot different load balancer solutions such as HAProxy, Nginx, etc. If you are on a cloud environment such as AWS you could also have managed solutions such as Elastic Load Balancer.
Horizontally Scaling Stateful HTTP app: it would be great if we could have all applications being stateless everytime, but unfortunately that's not always possible. So, when dealing with stateful HTTP apps, you must care about the HTTP session, which is a basically a local storage for each different client where the web server can store data that is kept across different HTTP requests (such as when dealing with a Shopping Cart). Well, in this case, when scaling horizontally you should be aware that, as I said, it's a LOCAL storage, so ServerA will not be able to handle an HTTP session that is on ServerB. In other words, if for any reason Client1 that is being served by ServerA starts suddenly to be served by ServerB, his HTTP session will be lost (and his shopping cart will be gone!). The reasons could be a node failure or even a deployment. In order to address this issue, you can't keep HTTP sessions only locally, that is, you must store them on another external component. That are several components that would be able to handle this, such as any relational database, but that would be actually an overhead. Some NoSQL databases can handle this key-value behavior very well, such as Redis. Now, with the HTTP session being stored on Redis, if a client starts to be served by another server, it will fetch the client's HTTP session from Redis and load it into its memory, so everything will continue working and the user will not lost his HTTP session anymore. You can use Spring Session to easily store the HTTP session on Redis.
Horizontally Scaling WebSocket app: When a WebSocket connection is established, the server must keep the connection opened with the client so that they can exchange data in both directions. When a client is listening to a destination such as "/topic/public.messages" we say the client is subscribed to this destination. In Spring, when you use the simpleBroker approach, the subscriptions are kept in memory, so what happens for instance if Client1 is being served by ServerA and wants to send a message using WebSocket to Client2 being served by ServerB? You already know the answer! The message will not be delivered to Client2 because Server1 not even know about the Client2's subscription.
So, in order to address this issue, again you have to externalize the WebSockets subscriptions. As you are using STOMP as a subprotocol, you need an external component that can act as a external STOMP broker. There are quite a lot tools able to do this, but I would suggest RabbitMQ.
Now, you must change your Spring configuration so that it will not keep the subscriptions in-memory. Instead, it will delegate the subscriptions to a external STOMP broker. You can easily achieve this with some basic configurations such as enableStompBrokerRelay.
The important thing to note is that HTTP session is different than WebSocket session. Using Spring Session to store the HTTP session in Redis has absolutely nothing to do with horizontally scaling WebSockets.
I've coded a complete Web Chat Application with Spring Boot (and much more) that uses RabbitMQ as a Full External STOMP Broker and it's public on GitHub so please clone it, run the app in your machine and see the code details.
When it comes to a WebSocket connection loss, there's not much that Spring can do. Actually, the reconnection must be requested by the client side implementing a reconnection callback function, for instance (that's the WebSocket handshake flow, the client must start the handshake, not the server). There are some client side libraries that can handle this transparently for you. That's not SockJS case. In the Chat Application I also implemented this reconnection feature.