Hazelcast in multinode docker environments with TCPIP

Question

I am currently struggling with the following problem:

I have a Java EE application that shall be able to run in a cluster (it actually does). For the data exchange between the nodes I use Hazelcast 3.3.3. Since our customers are afraid of UDP we use TCP. So I configure 3 hazelcast parameters per node: The network interface to use, the port and the members of the cluster. In real life (not a docker environment) everything works. Now I want to put my Java EE servers into docker containers (one per node) and also link them via Hazelcast.

What I did was:

On the host machine running the docker containers I created several virtual network interfaces (172.21.0.X). for each docker container I gave network mappings like -p 172.21.0.X:5190:5190 (5190 is the port I use for Hazelcast)

In my Java EE application within the container I configured hazelcast to listen to interface 172.21.0.X, port 5190 members 172.21.0.X:5190,...... The problem now is: docker will assign an internal IP number to each cointainer. In advance I cannot be sure which IP number docker will assign. It definitly is not the IP I would like to use (172.21.0.X). Hazelcast starts up and tells me: no matching interface found. Of course not. Hazelcast will check the IP interfaces of the docker container which is something like 172.4.XX. This does not fit to the config I gave to hazelcast (use 172.21.X.X).

So far it seems not to be possible to specify the internal IP number of a docker container. I could query it from a running container, but that is too late since I need to do the configuration of my Java EE server BEFORE I start the container.

Any hints on that? Is it possible to get the Hazelcast running with TCPIP?

Answer 1

This thread provides an answer and an example: https://groups.google.com/d/msg/orient-database/ehfKcInWZXs/wxs8xLUOGB8J

Basically you use hazelcast variables in your hazelcast file, i.e.

<property name="hazelcast.local.localAddress">${hazelcast.ip}</property>

Then when you're running the docker contained you add:

-e IP=`ip route | awk '/default/ { print  $9 }'`

Which sets an environment variable in the docker container. Then in there you do something like this:

CMD ["java", "-Dhazelcast.ip=${IP}", ...

So it's taking the docker env variable, and smashing into java as a definition, which hazelcast then picks out and substitutes into it's config file at runtime.