How to get metrics data from aws cloudwatch to csv

Question

i have been working on project, and i want to export metrics data from cloudwatch like CPU Utilization and Network Out data, is there any way to get that data? and convert it to csv?

Answer 1

Export CloudWatch Metrics

You can't do it directly but following is the step by step instructions:

Pre Reqs

1. AWS CLI (Command Line Interface) from here
2. jq is a lightweight and flexible command-line JSON processor from here

How to export

Use the following CLI:

aws cloudwatch get-metric-statistics 
--namespace AWS/EC2 --metric-name CPUUtilization 
--dimensions Name=InstanceId,Value=i-xxxxxxxxxxxxxxxxx 
--statistics Average 
--start-time 2020-03-01T00:00:00 
--end-time 2020-03-31T23:59:00 
--period 3600
--region us-east-1

Valid options for --metric-name depend on the --name-space parameter. For AWS/EC2, the full list can be seen by running the following CLI command:

aws cloudwatch list-metrics --namespace "AWS/EC2"

Valid options for --statistics are:

SampleCount
Average
Sum
Minimum
Maximum

--start-time and --end-time specify the range.

--period The granularity, in seconds, of the returned data points.

--region The region where the CloudWatch metric is being meatured (us-east-1, us-west-2 etc.)

Data output will look something like the following:

{
    "Label": "CPUUtilization",
    "Datapoints": []
}

To convert it to CSV we will use jq. For this you have two options:

Option 1

Pipe all the aws cli output to jq:

aws cloudwatch get-metric-statistics 
--namespace AWS/EC2 --metric-name CPUUtilization 
--dimensions Name=InstanceId,Value=i-xxxxxxxxxxxxxxxxx 
--statistics Average 
--start-time 2020-03-01T00:00:00 
--end-time 2020-03-31T23:59:00 
--period 3600
--region us-east-1
| jq -r '.Datapoints[] | [.Timestamp, .Minimum, .Unit] | @csv'

Option 2

Export the data to JSON:

aws cloudwatch get-metric-statistics 
--namespace AWS/EC2 --metric-name CPUUtilization 
--dimensions Name=InstanceId,Value=i-xxxxxxxxxxxxxxxxx 
--statistics Average 
--start-time 2020-03-01T00:00:00 
--end-time 2020-03-31T23:59:00 
--period 3600
--region us-east-1 >> data.json

Use jq to read the json to csv:

jq -r '.Datapoints[] | [.Timestamp, .Minimum, .Unit] | @csv' data.json

Output

Here is how the output will look like:

"2020-03-24T11:00:00Z",0.327868852454245,"Percent"
"2020-03-11T21:00:00Z",0.327868852454245,"Percent"
"2020-03-15T04:00:00Z",0.322580645156596,"Percent"
"2020-03-27T18:00:00Z",0.327868852478101,"Percent"