Count unique values in aws cloudwatch metric

amazon-cloudwatch amazon-cloudwatchlogs amazon-cloudwatch-metrics
postelrich picture postelrich · Jun 26, 2017 · Viewed 13.3k times · Source

I have a set of cloudwatch logs in json format that contain a username field. How can I write a cloudwatch metric query that counts the number of unique users per month?

Answer

ImperviousPanda picture ImperviousPanda · Oct 4, 2019

You can now do this! Using CloudWatch Insights.

API: https://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_StartQuery.html

I am working on a similar problem and my query for this API looks something like:

fields @timestamp, @message
| filter @message like /User ID/
| parse @message "User ID: *" as @userId
| stats count(*) by @userId

To get the User Ids. Right now this returns with a list of them then counts for each one. Getting a total count of unique can either be done after getting the response or probably by playing with the query more.

You can easily play with queries using the CloudWatch Insights page in the AWS Console.

Related questions

AWS Cloudwatch Filter and Pattern Syntax

I'm following the instructions here https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/FilterAndPatternSyntax.html but it's not working as i'm expecting it to. I currently have the following cloudwatch log subscription filter pattern: ? "UNKNOWN_TOPIC_OR_PARTITION" ? " SEVERE " ? " severe " ? " …

Read More
send notification alert when AWS Lambda function has an error

I have a AWS Lambda function running some process in my infrastructure. The Lambda is triggered every 8 hours using a CloudWatch rule. I am trying to raise a notification if any error happens into the Lambda process. I tried to …

Read More
Amazon Cloudwatch Logs Insights with JSON fields

I am trying to use Logs Insights with data containing JSON in one of the fields, and to parse the JSON fields My data looks like the following when I put it in insights with the starter code fields @timestamp, @…

Read More