How to ignore empty dataseries in prometheus

eventhorizon picture eventhorizon · Dec 19, 2017 · Viewed 8.7k times · Source

Calculating the maximum quantile over all dataseries is a problem for me:

query

http_response_time{job=~"^(x|y)$", quantile="0.95",...}

result

http_response_time{job="x",...} 0.26
http_response_time{job="y",...} NaN

This is how I would try to calculate the maximum:

avg(http_response_time{job=~"^(x|y)$",...})

Now the result will be "NaN". How can I ignore the "NaN" result (from the result section)?

UPDATE 0

The metric is a self made summary-metric.

UPDATE 1

Using prometheus version 1.8.

Answer

svenwltr picture svenwltr · Dec 19, 2017

I didn't try this one with NaN, but you can simply filter by values with binary operators. Since NaN mathematically doesn't equal NaN you could try this trick (since a response time should be always positive):

avg(http_response_time{job=~"^(x|y)$",...} >= 0)