Automate your setup and deploy with a build script!

Build Script

One of the most influential articles I have ever read in my career is this one. In it, James Shore talks about how your team can automate the setup of and deployment of your software in each of your environments (development, testing, staging, production, etc..).

When I first read the article, I was hooked because I was working at a company that had their developers spend the first week or two setting up their machine with all the dependencies needed to run the software. The company had a fairly high turn over rate of six developers (not counting me) in the just over two years I was there.

It wasn't like they had a horrible process. They created an image of the operating system with all of the necessary software. When you got your machine with the image, you had to run a bunch of manual steps and manual builds to get everything configured and setup. The image wasn't in a repository and it didn't get updated regularly. I remember one developer had a spot on the white board behind him where he would just write down all the changes that needed to be made the next time he worked on the image. He worked on the image every few months.

The build script is a valuable asset

I would like to contrast that with the job I took right after. The company wanted me to work on their biggest and oldest client's website. It took me well over a week to get everything setup and configured just so I could make my first change.

I was still inspired by the above mentioned article and I decided to take some action making the setup of the environment easier. Myself and a few developers took notes on what was still manual and what needed to be improved.

It started out with just little steps (if I recall correctly, we started with things like checking if some software was installed and then downloading and installing it). As we kept making changes to the system we kept making changes to the build script of the system. This would get committed to the same repository where this client's code lived. It grew with the system we were building.

Overtime, this script included a lot of steps, improvements, fixes and checks that made the process so much smoother. It did things like run unit tests before the code went to production, ran an update index on our search index software, pulled down a copy of the production database to the development machine and then changed all the production settings to development ones on our development machines. It was one of those tools that really made the process better.

The results

In fact, on my last day, the developer that took over that client's website had his development machine ready to make changes. Also, when he made his first commit his change ran through our gauntlet of checks and tests in the pre-deploy script. On the first day he was able to get his machine running and we had an automated pre-deploy process that helped him make fewer errors. The script took just a few hours to get everything setup for him. It would have been faster but the database pull took a lot of time since it was very large.

The deploy to production was only 9 minutes and it automated a lot of steps and quirks.

The process for creating this software isn't hard to do. I followed the advice in the above article and slowly, overtime, big items on the list became little items which then in time became non-events because of the automation. In other words, start with one item on your list and automate it as much as you can.

If you can't finish it completely, do as much as you can so that you are doing only a few steps manually (don't worry, you can come back and automate these steps later). The point is to have an automated script that handles as many steps and quirks in your setup and deploy as you can. We even included place holders in the script for things we wanted to add in time.

Bonus

I wanted to add this bonus section of what the script did in development and production. It is a high level view and not meant to be exhaustive (we had many small steps and checks because of the technology stack used). I am certain I am missing some things because it has been over a year since I have seen this script:

In development:

  • Create a virtual environment (if it didn't already exist)
  • Check current versions of dependencies
  • Install all dependencies with the correct version
  • Run automated tests
  • Pull copy of production database
  • Change the production settings in the database to development settings
  • Re-index the fuzzy search database (our shop didn't work without this being indexed)

In production:

  • Create a virtual environment (if it didn't already exist)
  • Check current versions of dependencies
  • Install all dependencies with the correct version
  • Run automated tests
  • Backup the database
  • Pull in the new code on the server
  • Run database migrations
  • Verify the production settings are in place in the database