« Back to blog

Blog: Automating deployment with Fabric

Graham Weldon

22 December, 2011

Lately as part of my work, and as part of trying to make changes faster on my personal site, I have been trying to really ease deployment. I encourage people to release early, release often, but I have often found the systems and processes required to get a release out there difficult, or cumbersome.

For various reasons, I never got onto the chef / capistrano / whatever else bandwagon. it never felt like a great solution, and to a certain extent it abstracts the system administration stuff that I love. I’m a freak for keeping logs. Whenever I jump onto a server and make changes, or configure something, the bash log is retained, and stored off-server, and the configuration files are committed to a repository and pushed off-server also. With this process, I have been able to rebuild servers safely and cleanly (albeit slowly).

What I was looking for was a nice system that allowed me complete flexibility, scalability, ease of administration and hopefully in the process exposed me to some new technology. A mate of mine, Mark Story suggested looking into Fabric. You can head over to their site and read the description if you like. But it's essentially a system that provides a way to script commands to be run on servers. It's built on Python, and it's so simple and powerful it was just the right solution for me.

I’m now using ‘fabfiles’ for everything I deploy in a progressive manner. If I need to deploy an update to an old site, I will spend a few minutes to create a fabfile for it on a standard base that I have been building up, meaning that all future deployments are as simple as typing a single command.

First, let's talk about how I deploy updates for my site.

$ fab

Yep. That's it. Given I am in the directory containing the fabfile.py for my site, typing fab on the command line is all that is needed to update my site.

Of course, more complex systems, and sites that require multiple servers require a slightly different configuration, and staging sites etc. So I have developed the following script which is completely configurable and extensible over time to allow me to deploy any site, with configuration options at the top of each fabfile.

Here is the deployment script in a Gist on Github.

What you will notice is that I have broken it down into various tasks and configuration sections to keep things simple.

There are 3 “environments”:

  • __base__ is the parent for all other environments, and contains default settings for all environments
  • production is for the production site
  • stage is for the staging site

Within the base I have listed a few special sub-sections:

  • configs are configuration files that are linked. Consider it a special case for links described below
  • links are from / to configurations for files and directories to be symlinked into the site structure
  • delete defines files and folders to be deleted from the deployment. In this case I delete the test.php file in the webroot of my CakePHP app.

The rest should be reasonably self explanatory. It's got some way to go to be completely generic for all sites, but so far it has saved myself and my job many many hours of toiling on servers with configs trying to remember deployment steps and special cases for various sites.

Detailed usage

Given that I have a number of environments that I can configure, this allows me to deploy more than one site configuration with the same fabfile.

Deploying stage is easy, since its the default environment. This is specified simply by putting 'default': true in the environment configuration in the fabfile. Because it's the default, I can do one of the following commands to deploy:

$ fab
or
$ fab deploy:environment=stage

Deploying the production environment is a little more long winded, like the second command option for staging. It's ordered this way so that if I accidentally run fab while the repo is in a state thats not stable, only the stage site is affected. Production is deployed with:

$ fab deploy:environment=production

It's really that simple.

Benefits

One of the features of this deployment, and the way fabric is built is that if any of the commands that are attempted fails, then the deployment stops. Given it's checking out the repo into its own directory, this means that the production site is unaffected while you attempt a deployment. It's only after all steps are successful that the symlink for the production site is adjusted.

Directory Structure

As I mentioned above, It's checking stuff out into a separate directory, and symlinking upon success. This means I have old deployments lying around to rollback to if I need to, and it also means that I keep the production environment sane during deployment.

Here’s the directory structure for my personal site:

/var/sites/grahamweldon.com/
    - current (symlink) -> 20111222.1512
    - config
    - uploads
    - 20111219.1865  (old)
    - 20111220.2231  (old)
    - 20111220.2238  (old)
    - 20111221.0954  (old)
    - 20111222.1512  (symlinked by 'current')

Conclusion

This is not perfect, and I am dubious about the security of posting my deployment script on github. But given I have no passwords, and key based authentication for all user accounts, I am sure this will be okay.

I hope this helps people out there trying to manage their own deployment, and I hope that you take a look at Fabric as you explore solutions to simplify your deployment process. I’d also welcome any and all comments on the fabfile I am currently using. Thanks!

Comments