How to configure alerts in Prometheus for diskspace

prometheus prometheus-alertmanager
user2153844 picture user2153844 · Oct 16, 2018 · Viewed 10k times · Source

We have prometheus running on Win Server box, and WMI exporter on a separate box(client). Able to read client metrics in Prometheus. Now the requirement is the moment Diskspace =>90 % , send an email alert, so that we can run a job to clean up space using an automated job / manual job.

Could you please help on how to configure alert for diskspace >90

enter image description here

Answer

Prafull Ladha picture Prafull Ladha · Oct 23, 2018

To send email notification based on alert you need to setup alertmanager with prometheus. Here is the guide how to do that: https://github.com/prometheus/alertmanager

Also you can configure the alert rules. I am using node exporter to fetch node metrics and using the following rule

- alert: DiskSpace10%Free
     expr: node_exporter:node_filesystem_free:fs_used_percents >= 90
     labels:
       severity: moderate
     annotations:
       summary: "Instance {{ $labels.instance }} is low on disk space"
       description: "{{ $labels.instance }} has only {{ $value }}% free."

You can configure the above rule according to WMI exporter and you will be good to go. Hope this helps.

Related questions

Multiple Targets on prometheus

I've configured prometheus on Centos, version details are follows. prometheus-2.5.0.linux-386 I've added two targets on the prometheus.yml configuration file, all the servers node exporters are running. Config as follows, scrape_configs: - job_name: "node" scrape_interval: "15s" …

Read More
How to silence Prometheus Alertmanager using config files?

I'm using the official stable/prometheus-operator chart do deploy Prometheus with helm. It's working good so far, except for the annoying CPUThrottlingHigh alert that is firing for many pods (including the own Prometheus' config-reloaders containers). This alert is currently under …

Read More
scrape interval and evaluation interval in prometheus

My scrape interval and evaluation interval are way off from each other as whown below (15s vs 4m). When I feed metrics to the endpoint, I find that the rules are evaluated every 4m which is expected. However, what I …

Read More