Logstash not creating index on Elasticsearch

Karthik Kamalakannan picture Karthik Kamalakannan · Mar 24, 2015 · Viewed 22.4k times · Source

I'm trying to setup a ELK stack on EC2, Ubuntu 14.04 instance. But everything install, and everything is working just fine, except for one thing.

Logstash is not creating an index on Elasticsearch. Whenever I try to access Kibana, it wants me to choose an index, from Elasticsearch.

Logstash is in the ES node, but the index is missing. Here's the message I get:

"Unable to fetch mapping. Do you have indices matching the pattern?"

Am I missing something out? I followed this tutorial: Digital Ocean

EDIT: Here's the screenshot of the error I'm facing: Logstash missing Indices in ES (Kibana4) Yet another screenshot:

Answer

DMH picture DMH · Mar 29, 2015

I got identical results on Amazon AMI (Centos/RHEL clone)

In fact exactly as per above… Until I injected some data into Elastic - this creates the first day index - then Kibana starts working. My simple .conf is:

input {
  stdin {
      type => "syslog"
    }
}
output {
  stdout {codec => rubydebug }
   elasticsearch {
          host => "localhost"
          port => 9200
          protocol => http
       }
}

then

cat /var/log/messages | logstash -f your.conf

Why stdin you ask? Well it's not super clear anywhere (also a new Logstash user - found this very unclear) that Logstash will never terminate (e.g. when using the file plugin) - it's designed to keep watching.

But using stdin - Logstash will run - send data to Elastic (which creates index) then go away.

If I did the same thing above with the file input plugin, it would never create the index - I don't know why this is.