Going by a recent tutorial on setting up AWS Elastic Beanstalk for Ruby deployment using Git, I just set up a Elastic Beanstalk environment from my CI server. However, the application failed to start. I went through the logs to find that bundle install
was failing with an error message.
Fetching [email protected]:example/private-repository.git Host key verification failed. fatal: The remote end hung up unexpectedly [31mGit error: command
git clone '[email protected]:example/private-repository.git' "/var/app/ondeck/vendor/cache/ruby/1.9.1/cache/bundler/git/private-repository-e4bbe6c2b13bb62664e39e345c1b01d80017934c" --bare --no-hardlinks
in directory /var/app/ondeck has failed.[0m
Gemfile
of my Rails application contains references to gemified plugins hosted on a couple of my owned private repositories on Github. Something like
gem 'somegemname', :git => '[email protected]:example/private-repository.git'
I had faced similar issues with Capistrano deployments which were resolved by setting up ssh_options[:forward_agent] = true
.
AWS Elastic Beanstalk Ruby container supports custom configuration through custom .config
files placed under .ebextensions
. Would setting up an SSH forward agent help in this case? Are there any other alternatives to reach a private Github repository while starting an Elastic Beanstalk environment?
Update 1:
I just checked for the user with which a bundle install
is initiated. Found out that a script /opt/elasticbeanstalk/hooks/appdeploy/pre/10_bundle_install.sh
starts bundle install
as root
user. I tried creating an SSH Key under /root/.ssh
and added it's pub-key to Github Deploy keys for that repository. No luck so far. Will now try to add an SSH pub-key to my user account on Github so that it applies to all private repositories accessible through my Github account.
After a good day of effort, I finally enabled use of my organization's private GitHub repos with Elastic Beanstalk by just using a .config
file. I am using Python and pip
, but it should also work for other package installers on EB.
rhetonik's ssh-agent
+ssh-add
approach did not work for me at all, so I elected to set up an ssh configuration file instead.
Here is my .ebextensions/3-pip-install-from-github.config
file:
files:
"/root/.ssh/config":
owner: root
group: root
mode: "000600"
content: |
Host github.com
User git
Hostname github.com
IdentityFile /root/.ssh/github
commands:
01-command:
command: sudo ssh-keyscan -H github.com >> /root/.ssh/known_hosts
02-command:
command: sudo chmod 644 /root/.ssh/known_hosts
03-command:
command: sudo aws s3 cp s3://bucket-with-your-github-ssh-key/github /root/.ssh
04-command:
command: sudo chmod 600 /root/.ssh/github
Rough instructions:
Set up an S3 bucket accessible by your EB instance. Inside of that bucket, store the SSH key allowing access to the GitHub repository you want to access via pip
, npm
, bundle
, etc. Use sudo aws s3 cp
to copy that key onto your EB instance on deploy. sudo
is necessary because EB scripts use root
and not ec2-user
.
This ebextensions config file also creates 2 files on your EB instance. /root/.ssh/config
tells ssh
(invoked by pip
and git
) to use the key you copied from S3. Storing the output of ssh-keyscan -H github.com
into /root/.ssh/known_hosts
will pre-verify that ssh
on your EB instance is actually communicating with GitHub to avoid MITM attacks. This is better than disabling StrictHostKeyChecking
in /root/.ssh/config
.
Here is my requirements.txt
file for pip
:
Beaker==1.7.0
Flask==0.10.1
Jinja2==2.7.3
MarkupSafe==0.23
# [...]
git+ssh://[email protected]/myorganization/[email protected]
While running eb-deploy
, you can tail -f /var/log/eb-activity.log
to make sure everything runs smoothly.