How can I (securely) download a private S3 asset onto a new EC2 instance with cloudinit?

Christopher picture Christopher · Jul 6, 2012 · Viewed 19k times · Source

I'm using CloudFormation to manage a Tomcat webserver stack but am tired of doing raw AMI management for new application versions. I'd like to move in the direction of Chef but don't have the time right now. Instead, I'm trying to conquer a simple problem in webserver instantiation: How can I download a "current" WAR when new machines spin-up?

My thought was to utilize a private S3 bucket and cloudinit, but I'm a little stumped by what to do with IAM credentials. I could put them in the template's user data, but I'm loathe to do so, particularly because I'm version controlling that file. The only alternative I can think of is to use environment variables in the AMI itself. They'd have to be plaintext, but... eh, if you can break into my instance, you could zip up and download my entire webserver. As long as the IAM user isn't reused for anything else and is rotated regularly, it seems like a reasonable way to solve the problem. Am I missing anything? How can I securely download a private S3 asset using cloudinit?

Answer

Eric Hammond picture Eric Hammond · Jul 6, 2012

Amazon recently announced a new feature where you can give "IAM roles" to your EC2 instances. This makes it fairly easy to allow specific instances to have permission to read specific S3 resources.

Here's their blog post announcing the new feature:

http://aws.typepad.com/aws/2012/06/iam-roles-for-ec2-instances-simplified-secure-access-to-aws-service-apis-from-ec2.html

Here's the section in the EC2 documentation:

http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/UsingIAM.html#UsingIAMrolesWithAmazonEC2Instances

Here's the section in the IAM documentation:

http://docs.amazonwebservices.com/IAM/latest/UserGuide/WorkingWithRoles.html

IAM roles make the credentials available to the instance through HTTP, so any users or processes running on the instance can see them.