docker-compose.yml for elasticsearch and kibana

elasticsearch docker docker-compose kibana
Richie picture Richie · Nov 2, 2016 · Viewed 24.9k times · Source

My aim is to get the elasticsearch and kibana images from DockerHub working locally using Docker.

This does the trick and works perfectly...

docker network create mynetwork --driver=bridge

docker run -p 5601:5601 --name kibana -d --network mynetwork kibana 
docker run -p 9200:9200 -p 9300:9300 --name elasticsearch -d --network mynetwork elasticsearch

Today a bird whispered in my ear and said I should learn docker-compose. So I tried to do all of what's above inside a docker-compose.yml.

Here is my attempt.

version: "2.0"
services:
  elasticsearch:
    image: elasticsearch:latest
    ports:
      - "9200:9200"
      - "9300:9300"
    networks:
      - docker_elk
  kibana:
    image: kibana:latest
    ports:
      - "5601:5601"
    networks:
      - docker_elk
networks:
  docker_elk:
    driver: bridge

Unfortunately this does not work. I've been racking my brains as to why I always get the ECONNREFUSED error as shown below when i run docker-compse up.

$ docker-compose up
Starting training_elasticsearch_1
Recreating training_kibana_1
Attaching to training_elasticsearch_1, training_kibana_1
elasticsearch_1  | [2016-11-02 22:39:55,798][WARN ][bootstrap                ] unable to install syscall filter: seccomp unavailable: your kernel is buggy and you should upgrade
elasticsearch_1  | [2016-11-02 22:39:56,036][INFO ][node                     ] [Caliban] version[2.4.1], pid[1], build[c67dc32/2016-09-27T18:57:55Z]
elasticsearch_1  | [2016-11-02 22:39:56,036][INFO ][node                     ] [Caliban] initializing ...
elasticsearch_1  | [2016-11-02 22:39:56,713][INFO ][plugins                  ] [Caliban] modules [reindex, lang-expression, lang-groovy], plugins [], sites []
elasticsearch_1  | [2016-11-02 22:39:56,749][INFO ][env                      ] [Caliban] using [1] data paths, mounts [[/usr/share/elasticsearch/data (/dev/vda2)]], net usable_space [54.8gb], net total_space [59gb], spins? [possibly], types [ext4]
elasticsearch_1  | [2016-11-02 22:39:56,749][INFO ][env                      ] [Caliban] heap size [990.7mb], compressed ordinary object pointers [true]
kibana_1         | {"type":"log","@timestamp":"2016-11-02T22:39:58Z","tags":["status","plugin:[email protected]","info"],"pid":11,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
kibana_1         | {"type":"log","@timestamp":"2016-11-02T22:39:58Z","tags":["status","plugin:[email protected]","info"],"pid":11,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
kibana_1         | {"type":"log","@timestamp":"2016-11-02T22:39:58Z","tags":["error","elasticsearch"],"pid":11,"message":"Request error, retrying -- connect ECONNREFUSED 172.20.0.2:9200"}
kibana_1         | {"type":"log","@timestamp":"2016-11-02T22:39:58Z","tags":["status","plugin:[email protected]","info"],"pid":11,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
kibana_1         | {"type":"log","@timestamp":"2016-11-02T22:39:58Z","tags":["warning","elasticsearch"],"pid":11,"message":"Unable to revive connection: http://elasticsearch:9200/"}
kibana_1         | {"type":"log","@timestamp":"2016-11-02T22:39:58Z","tags":["warning","elasticsearch"],"pid":11,"message":"No living connections"}
kibana_1         | {"type":"log","@timestamp":"2016-11-02T22:39:58Z","tags":["status","plugin:[email protected]","error"],"pid":11,"state":"red","message":"Status changed from yellow to red - Unable to connect to Elasticsearch at http://elasticsearch:9200.","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
kibana_1         | {"type":"log","@timestamp":"2016-11-02T22:39:58Z","tags":["status","plugin:[email protected]","info"],"pid":11,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
kibana_1         | {"type":"log","@timestamp":"2016-11-02T22:39:58Z","tags":["status","plugin:[email protected]","info"],"pid":11,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
kibana_1         | {"type":"log","@timestamp":"2016-11-02T22:39:58Z","tags":["status","plugin:[email protected]","info"],"pid":11,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
kibana_1         | {"type":"log","@timestamp":"2016-11-02T22:39:58Z","tags":["status","plugin:[email protected]","info"],"pid":11,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
kibana_1         | {"type":"log","@timestamp":"2016-11-02T22:39:58Z","tags":["status","plugin:[email protected]","info"],"pid":11,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
kibana_1         | {"type":"log","@timestamp":"2016-11-02T22:39:58Z","tags":["listening","info"],"pid":11,"message":"Server running at http://0.0.0.0:5601"}
elasticsearch_1  | [2016-11-02 22:39:58,515][INFO ][node                     ] [Caliban] initialized
elasticsearch_1  | [2016-11-02 22:39:58,515][INFO ][node                     ] [Caliban] starting ...
elasticsearch_1  | [2016-11-02 22:39:58,587][INFO ][transport                ] [Caliban] publish_address {172.20.0.2:9300}, bound_addresses {[::]:9300}
elasticsearch_1  | [2016-11-02 22:39:58,594][INFO ][discovery                ] [Caliban] elasticsearch/1Cf9qz7CSCqHBEEuwG7PQw
kibana_1         | {"type":"log","@timestamp":"2016-11-02T22:40:00Z","tags":["warning","elasticsearch"],"pid":11,"message":"Unable to revive connection: http://elasticsearch:9200/"}
kibana_1         | {"type":"log","@timestamp":"2016-11-02T22:40:00Z","tags":["warning","elasticsearch"],"pid":11,"message":"No living connections"}
elasticsearch_1  | [2016-11-02 22:40:01,650][INFO ][cluster.service          ] [Caliban] new_master {Caliban}{1Cf9qz7CSCqHBEEuwG7PQw}{172.20.0.2}{172.20.0.2:9300}, reason: zen-disco-join(elected_as_master, [0] joins received)
elasticsearch_1  | [2016-11-02 22:40:01,661][INFO ][http                     ] [Caliban] publish_address {172.20.0.2:9200}, bound_addresses {[::]:9200}
elasticsearch_1  | [2016-11-02 22:40:01,661][INFO ][node                     ] [Caliban] started
elasticsearch_1  | [2016-11-02 22:40:01,798][INFO ][gateway                  ] [Caliban] recovered [1] indices into cluster_state
elasticsearch_1  | [2016-11-02 22:40:02,149][INFO ][cluster.routing.allocation] [Caliban] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[.kibana][0]] ...]).
kibana_1         | {"type":"log","@timestamp":"2016-11-02T22:40:03Z","tags":["status","plugin:[email protected]","info"],"pid":11,"state":"green","message":"Status changed from red to green - Kibana index ready","prevState":"red","prevMsg":"Unable to connect to Elasticsearch at http://elasticsearch:9200."}
^CGracefully stopping... (press Ctrl+C again to force)
Stopping training_kibana_1 ... done
Stopping training_elasticsearch_1 ... done

Can someone please help me with why?

thanks

Answer

Tuan picture Tuan · Nov 2, 2016

You need to include the links.

version: "2.0"
services:
  elasticsearch:
    image: elasticsearch:latest
    ports:
      - "9200:9200"
      - "9300:9300"
    networks:
      - docker_elk
  kibana:
    image: kibana:latest
    ports:
      - "5601:5601"
    links:
      - elasticsearch
    networks:
      - docker_elk
networks:
  docker_elk:
    driver: bridge

UPDATED

When using the image elasticsearch:latest, it's Elasticsearch 5.0 and requires us to increase our Docker host virtual memory.

Before running the docker-compose, please make sure to run this command on your Docker host.

Linux:

su root
sysctl -w vm.max_map_count=262144

Windows (boot2docker)

docker-machine ssh default
sudo sysctl -w vm.max_map_count=262144

If you don't want to change your Docker host, just use the Elasticsearch 2.x image at elasticsearch:2

Related questions

docker-compose.yml for elasticsearch 7.0.1 and kibana 7.0.1

I am using Docker Desktop with linux containers on Windows 10 and would like to launch the latest versions of the elasticsearch and kibana containers over a docker compose file. Everything works fine when using some older version like 6.2.4. This is …

Read More
Docker - ELK - vm.max_map_count

I'm trying to use the docker's image elk-docker (https://elk-docker.readthedocs.io/) , using Docker Compose. The .yml file, is like this: elk: image: sebp/elk ports: - "5601:5601" - "9200:9200" - "5044:5044" When I run the command: sudo docker-compose up, the console shows: * …

Read More
Docker-compose links vs external_links

I believe it is simple question but I still do not get it from Docker-compose documentations. What is the difference between links and external_links? I like external_links as I want to have core docker-compose and I want to …

Read More