AWS Elasticsearch Service IAM Role based Access Policy

Question

I have been struggling to figure out how to communicate with the Amazon ES service from my EC2 instances.

The documentation clearly states that the Amazon ES service supports IAM User & Role based access policies. http://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-createupdatedomains.html#es-createdomain-configure-access-policies

However, when I have this access policy for my ES domain:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::123456789:role/my-ec2-role"
      },
      "Action": "es:*",
      "Resource": "arn:aws:es:us-west-2:123456789:domain/myDomain/*"
    }
  ]
}

I can't log into an ec2 instance and run a curl to hit my elasticsearch cluster.

Trying to do a simple curl of the _search API:

curl "http://search-myDomain.es.amazonaws.com/_search"

Produces an authentication error response:

{"Message":"User: anonymous is not authorized to perform: es:ESHttpGet on resource: arn:aws:es:us-west-2:123456789:domain/myDomain/_search"}

Just to be extra safe I put the AmazonESFullAccess Policy on my IAM Role, still doesn't work.

I must be missing something, because being able to programmatically interact with Elasticsearch from ec2 instances that use an IAM Role is essential to getting anything accomplished with the Amazon ES Service.

I also see this contradictory statement in the docs.

IAM-based Policy Example You create IAM-based access policies by using the AWS IAM console rather than the Amazon ES console. For information about creating IAM-based access policies, see the IAM documentation.

That link to IAM documentation, is to the home page of IAM and contains exactly zero information about how to do it. Anyone got a solution for me?

Answer 1

When using IAM service with AWS, you must sign your requests. curl doesn't support signed requests (which consists of hashing the request and adding a parameter to the header of the request). You can use one of their SDK's that has the signing algorithm built in, and then submit that request.

See: http://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/what-is-amazon-elasticsearch-service.html#signing-requests

You can find the SDKs for popular languages here: http://aws.amazon.com/tools/