Capistrano Deploy to dynamic AWS EC2
At Metal Toad we use Capistrano to deploy our projects to their respective servers.
At Metal Toad we use Capistrano to deploy our projects to their respective servers. Normally this is done with a configuration file for each stage (Dev, Staging, QA, and Production) that contains a list of servers. Below is an example of one of those files.
# Set the deployment directory on the target hosts. set :deploy_to, "/var/www/sites/virtual/<client>-stage.metaltoad-sites.com" # The hostnames to deploy to. role :web, "stg01-<client>.ec2.metaltoad.net", “stg02-<client>.ec2.metaltoad.net” # Specify one of the web servers to use for database backups or updates. # This server should also be running Drupal. role :db, "stg01-<client>.ec2.metaltoad.net", :primary => true # The path to drush set :drush, "cd #{current_path}/#{app_root} ; /usr/local/bin/drush" # The username on the target system, if different from your local username ssh_options[:user] = 'deploy'
This process has worked great, but now we are creating Custom Clouds in AWS, we need a more dynamic solution.
The biggest feature of AWS is the ability to pay for the servers you need, and only when you need them. That means we need to constantly change the lists of servers in our Capistrano configuration. These changes lead to deploys failing when a server that was there isn’t or when a server that should have gotten a deploy, didn’t.
Below is my solution to dynamically populate the server lists:
# Set the deployment directory on the target hosts. set :deploy_to, "/var/www/sites/virtual/#{application}" # The hostnames to deploy to. set :access_key, "" set :secret_key, "" set :ec2_servers, "" set :ec2_gateway, "" set :ec2_db_gateway, "" set :client, "" run_locally "echo `ec2-describe-instances -O #{access_key} -W #{secret_key} --filter \"instance-state-code=16\"| grep #{client} | grep Name | grep web | awk -F' ' '{ print $5\".ec2.metaltoad.net \" }'` | sed ':a;N;$!ba;s/\n/, /g' > /tmp/#{client}-servers" File.open("/tmp/#{client}-servers", 'r') do |f1| while line = f1.gets puts line ec2_servers=line.split(" ") end puts ec2_servers role(:web) { ec2_servers } end run_locally "echo `ec2-describe-instances -O #{access_key} -W #{secret_key} --filter \"instance-state-code=16\" --filter \"tag-key=Gateway\" | grep #{client} | grep Name | awk -F' ' '{ print $5\".ec2.metaltoad.net \" }'` | sed ':a;N;$!ba;s/\n/, /g' > /tmp/#{client}-gateway" File.open("/tmp/#{client}-gateway", 'r') do |f1| while line = f1.gets puts line ec2_gateway=line.split(" ") end set :gateway, ec2_gateway end # Specify one of the web servers to use for database backups or updates. # This server should also be running Drupal. run_locally "echo `ec2-describe-instances -O #{access_key} -W #{secret_key} --filter \"instance-state-code=16\" --filter \"tag-key=DB-Gateway\"| grep #{client} | grep Name | awk -F' ' '{ print $5\".ec2.metaltoad.net \" }'` | sed ':a;N;$!ba;s/\n/, /g' > /tmp/#{client}-database" File.open("/tmp/#{client}-database", 'r') do |f1| while line = f1.gets puts line ec2_db_gateway=line.split(" ") end role(:db) { ec2_db_gateway } end # The username on the target system, if different from your local username ssh_options[:user] = 'deploy' # The path to drush set :drush, "cd #{current_path}/#{app_root} ; /usr/local/bin/drush" namespace :deploy do desc "Notify New Relic" task :newrelic do run_locally 'curl -H "<newrelic key>" -d "deployment[application_id]=4214617" -d "deployment[user]=`whoami`" https://rpm.newrelic.com/deployments.xml' # end #end after "deploy", "deploy:newrelic"
This new script uses the AWS CLI to get a list of servers for our EC2 instances. It uses a combination of host names, and tags to identify the correct servers for the specific client. It then identifies our gateway, builds a list of web servers, and a list of servers with access to the database.
I hope other people will find this as useful as I do.