Connect two instances of docker-compose

Question

I have a dockerized application with a few services running using docker-compose. I'd like to connect this application with ElasticSearch/Logstash/Kibana (ELK) using another docker-compose application, docker-elk. Both of them are running in the same docker machine in development. In production, that will probably not be the case.

How can I configure my application's docker-compose.yml to link to the ELK stack?

Answer 1

Update Jun 2016

The answer below is outdated starting with docker 1.10. See this other similar answer for the new solution. https://stackoverflow.com/a/34476794/1556338

Old answer

1. Create a network:

    $ docker network create --driver bridge my-net
   

2. Reference that network as an environment variable (${NETWORK})in the docker-compose.yml files. Eg:

   pg:
     image: postgres:9.4.4
     container_name: pg
     net: ${NETWORK}
     ports:
       - "5432"
   myapp:
     image: quay.io/myco/myapp
     container_name: myapp
     environment:
       DATABASE_URL: "http://pg:5432"
     net: ${NETWORK}
     ports:
       - "3000:3000"
   

Note that pg in http://pg:5432 will resolve to the ip address of the pg service (container). No need to hardcode ip addresses; An entry for pg is automatically added to the /etc/host of the myapp container.

3. Call docker-compose, passing it the network you created:

   $ NETWORK=my-net docker-compose up -d -f docker-compose.yml -f other-compose.yml
   

I've created a bridge network above which only works within one node (host). Good for dev. If you need to get two nodes to talk to each other, you need to create an overlay network. Same principle though. You pass the network name to the docker-compose up command.