Deploying Shell Scripts with Capistrano 3

Introduction

We have a self developed and well established deployment procedure for all of our shell, awk and python scripts. So far we didn’t have any issues with it. So why bother replacing it with Capistrano?

During our build of the second disaster recovery site we realized, that deploying those scripts on multiple sites takes to much steps by a human. And since nobody is perfect we had to realize, that this might become an issues in the future.

We are using Ansible for our disaster recovery setup which enables us to get a full site, with multiple virtual machines, up and running in less than 30 min. And as you might have guessed from our previous blog posts, we have a couple of rails applications which we deploy with capistrano. Adding the new site was as easy as adding a new line with the right servers to the production.rb file. And we wanted to have the same luxury with all of our code

System Requirements

Capistrano requires that ruby version 2.0 or higher is installed on all servers. We deploy ruby with ansible and rbenv. This enables us to have fine grained control of the ruby version and all it’s gems used on the target machine. All of our code is further saved in a git repository on our internal gitlab server. So we are good to go!

Capifying your project

First we had to create the necessary directory structure and add the needed configuration files. This is can easily be done with

bundle exec cap install

This will create the following structure for us:

├── Capfile
├── config
│ ├── deploy
│ │ ├── production.rb
│ │ └── staging.rb
│ └── deploy.rb
└── lib
└── capistrano
└── tasks

The files of interest for us are config/deploy.rb and the files under config/deploy/*.rb. The first file contains the steps that are executed during the installation and the files in the deploy folder define the deployment environments where our software will be deployed.

Once everything is setup correctly the software will be deployed as following:

bundle exec cap production deploy

Deployment Environments

Let’s take a quick look at the production.rb in the deploy folder.

set :stage, :production

server 'dbserver-site-a', user: 'dbuser', roles: %w(app db), deploy_env: "production"
server 'webserver-site-a', user: 'webuser', roles: %w(app), deploy_env: "production"
server 'dbserver-site-b', user: 'dbuser', roles: %w(app db), deploy_env: "production"
server 'webserver-site-b', user: 'webuser', roles: %w(app), deploy_env: "production"

set :branch, ENV["REVISION"] || ENV["BRANCH_NAME"] || 'master'

When you now run capistrano our software is deployed on two sites with each two servers.

You can define multiple deployment environments like development, stating, testing and have different versions deployed in each environment.

The Installation Steps

The deploy.rb files consists of multiple task which will be run in parallel on all servers. Let’s have a look at an example tasks which will give you an idea what’s done in the background.

We run all our python code in an virtual environment which we setup as following:

task :pyenv do
   on roles(:app) do
      source="source #{venv_dir}/bin/activate;"
      execute "python3 -m venv #{venv_dir}"
      execute "#{source} pip3 install --upgrade pip"
      execute "#{source} pip3 install -r #{deploy_to}/current/adm/pqPyRequirements.txt"
   end
end
after 'deploy', 'deploy:pyenv'

What if you want to run a task on all servers in sequence. As we are accustomed by ruby we can write our code almost as natural language. So our sequential version of the same task looks as following:

 task :pyenv do
    on roles(:app), in: :sequence do
      source="source #{venv_dir}/bin/activate;"
      execute "python3 -m venv #{venv_dir}"
      execute "#{source} pip3 install --upgrade pip"
      execute "#{source} pip3 install -r #{deploy_to}/current/adm/pqPyRequirements.txt"
    end
end
after 'deploy', 'deploy:pyenv'

Happy Deployment

We went a step further and integrated capistrano in our continuous build in gitlab. This means once we pushed our changes to gitlab they are deployed to all our servers without any human interaction. Capistrano is configured to keep 5 versions on the servers. So in case anything went wrong we can easily rollback to the previous version.