Johntron

I use Subversion to deploy web applications using a simple svn update command on a production server. This is really convenient, but something has always bothered me. Domain names for production servers are never the same as the development servers. This is true regardless of whether you use PHP, Python, Ruby, or something else. For this reason, deploying is never as easy as typing svn update.

This specific problem is fairly new to computer science, because server-side scripting languages are relatively new in the whole scheme of thing. I’ve done some research, and I’ve documented the most elegant solution I could come up with to minimize the possibility of human error from the deployment process.

Here are the highlights:

• Machine independent
• Language independent
• Easy to deploy
• Easy to setup
• Maintenance free (no tweaking during the deployment phase)
• Standard (available on most systems)

This system uses m4 and make. m4 is tool used to generate code from macros (among other things). Using m4 means we can do things similar to preprocessor directives in C languages. Make is a build tool used for generating non-source files from source files. Using make means we can generate one set of files for a development environment, and one set of files for a production environment. This system is very common, and has been around for years, so there should be plenty of documentation out there.

After setting up this build system, deployment would work like this:

1. From a development server, commit changes to a Subversion repository
2. From a production server, update a working copy
3. Run `make`

I will not discuss Subversion, as this is covered elsewhere. I’ll assume you know this, so let’s get started by creating an m4 file.

m4 as a Preprocessor

Ideally, all of the machine-specific settings should be stored in a single file. This is often called a config file or a settings file. In my case, the file is named “settings.py”, because I’m writing some software using Django. To decouple the machine-specific code, I will need to be able to write macros in settings.py. To do this, I rename the file to “settings.py.m4″. The m4 extension is used to denote files that should be processed with m4 (clearly). After renaming the file, I can use whatever macros I want. The most useful macro is a simple if…else… function called ifdef().

The most common ifdef() macro will look like this:

ifdef(`debug', `True', `False')

m4 would replace this statement with True or False based on whether or not debug is defined. The true power of this is made apparent when you start assigning URLs to variable.

For instance, imagine a file that contains this line of code:

DEBUG = True

This DEBUG variable could be used to control output and logging. Clearly, debug should never be enabled on a production server (gross). So how would you account for two different environments? The most straightforward solution would be to manually edit this file after each svn update or perhaps have Subversion ignore this file. By manually editing this file each time, you are assuming you’ll remember to do this. By ignoring this file from Subversion, you’re throwing out all the benefits of version control.

So here is where m4 comes in. Replace the line of code listed above with this:

DEBUG = ifdef(`debug', `True', `False')

Notice that debug is not defined yet (not to be confused with DEBUG). If we were to run m4 on this file, this line would end up looking like this:

DEBUG = False

So now we just need to define “debug” at a higher level. The best place for this is at the command line. With m4, you can define variables on the command line. Use the -D flag to accomplish this. In our case we would type something like this (from the command line): m4 -D debug settings.py.m4. After hitting enter, you would see the entire settings.py.m4 file output to the shell. If you look carefully, you should see the following:

DEBUG = True

Try running the same command again, but without the -D debug this time. You should see:

DEBUG = False

When you’re done learning about m4, run the command one last time, but this time save the output to a file by running this command: m4 -D debug settings.py.m4 > settings.py Now we’ve decoupled our machine-specific code and can control it from the command line. The last step is to make things a little easier to remember. make is great for this, because it allows us to create build targets.

Deploying web software with make

A target is simply a final copy of an application. In our case, we will have a development target and a production target. Begin simplifying things by creating a file name “Makefile” in the highest level directory of your source code. This file is used by make to determine what needs to be done for any given build target. The syntax of this file is pretty simple.

My Makefile looks like this:

live:
    m4 placethings/settings.py.m4 > placethings/settings.py

development:
    m4 -D debug placethings/settings.py.m4 > placethings/settings.py

This file defines two targets: live and development. To build a target, we type: make [target]. Make assumes the first target is the default, so we can just type make, and “settings.py.m4″ would be processed for a production environment and saved to “settings.py”. Alternatively, we could type make development and “settings.py.m4″ would be processed for a development environment and saved to “settings.py”.

The last step is to put these files under version control along with everything else ( using svn add). If you wanted to, you could even go one step further and add svn update to your Makefile. There are advantages and disadvantes to this. I prefer not to do this, because anything that happens during the update process can affect the rest of the build process.

Clearly, I’m passionate about removing the possiblity of human error. Some might say this is obsessive, but I say it’s what makes good software great. Now, instead of potentially spewing code traces at your users without realizing it everytime you change your code, you just have to make sure your build process works next time you use an ifdef() in your settings.py.m4 file. This is much easier to do than trying to trigger an exception on the production server.

Tags: development, howto, software

This entry was posted on Monday, March 9th, 2009 at 1:19 pm and is filed under Advice, mobile. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.