Kibana 3 Milestone 4 and Graphite Integration

Question

I am having difficulties understanding integration of Graphite and Kibana 3 to monitor logs and system vitals. I am referring to figure in Log management system described here.

1. Considering the new features in Kibana 3 Milestone 4, can we collect system vitals and store it directly into elastic search instead of graphite and use a single kibana dashboard (What could be right choice to implement in a distributed system where emphasis is on performance and low memory foot print)?
2. Why must we use StatsD and graphite, when count and simple statistics are now supported by kibana - Elasticsearch combination?
3. In case, we decide to use both graphite and kibana, How do we integrate it into a single Dashboard?
4. Is there a tutorial to integrate Dashboards (kibana and graphitos/graph explorer/orion/pencil)?

Thanks in advance.

Answer 1

Why statsd-graphite:

1. Statsd and Graphite can help you visualize anything, not just logs and system vitals. It is very straightforward with the statsd-graphite stack, to measure say- number of users that hovered on the left bottom of your site for more than 10 seconds.

2. Because there is no in-between logging involved, the scalability that graphite provides is unparalleled from an IO point of view. Also consider the fact that statsd talks UDP, so collecting 300K metrics per minute is a breeze.

3. You don't have to log something in order to see it.

Integration:

As clearly shown in the architectural diagram you shared, you can filter the stats that you want to visualize, have them forwarded to statsd. This is in parallel with kibana visualizing directly from logstash-elasticsearch. Going redundant with data is a easier approach if you want to view both Graphite and Kibana data over Graphite, since the webapp would not query elasticsearch directly.

Vimeo's Graph Explorer is something you might want to look into. It queries elasticsearch.

---

Updates:

Not that Logstash catn't do it, but it isn't 'designed' for that role, whereas statsd et al, are.

I have been wondering if we have a simpler query language.

The inherent scheme of organization in graphite is tree-like and hence the searches do-not/ can-not yeild results from a different subtree. This makes it not-so-suitable for cross-dimensional searches. GE is the simplest, given you want the power.

Graph Explorer's flow-

Graph Explorer addresses this by adding tags to the metrics and integrating it with elasticsearch. So what GE actually does is that-

1. One time- It connects to your Graphite front-end, makes API calls to retrieve all metrics.

2. It then 'converts' the old style proto 1 metrics (A.B.C) into tag-based proto 2 metrics (host=A.app=B.username=C).

3. This is then exported to ES which maintains an index.

4. When you query GE front-end, it connects to ES to understand what you want.

5. GE then queries the Graphite-API, and delivers the results in GE front-end.

Moreover, does graph explorer assume we are using diamond for collection?

No.

How does it compare to pencil, orion and graphiti?

These are on-surface optimizations to visualization. They-

1. change the look and feel of the graphs.

2. make querying the API easier.

3. allow a better monitoring flow.

They DO NOT change the way you store or search the information. GE, embeds itself 'deeper' into the metric data and hence has a real edge over how you query metrics. (Cross dimensional search)

Heads up-

GE's metric-importing plug-in is far from perfect. It successfully imported 300 out of my 1000 metrics. It is also heavier to render, and the front-end eats more NW (because of the hoverable, zoomable features).

Update-

Grafana is out.