Monday, October 25, 2010

Continuous Deployment

I listened to a podcast recently about a company doing continuous deployment. I was interested because my company does continuous deployment, and not just of a web application, but of a desktop app. From what I can tell, this is still quite rare.

They were asked how frequently they deploy, and they said every two weeks. I was a little surprised.  Obviously it's a long way from yearly releases, but to me, every two weeks is not exactly "continuous".

Part of the issue is that "continuous" often gets mixed up with "automated". For example, continuous build or integration systems are as much about the automation as they are about continuous. But the primary goal is the "continuous" part. The automation is a secondary requirement needed to satisfy the primary goal of continuous.

Of course, "continuous" seldom actually means "continuous". It usually means "frequent". Continuous builds might happen nightly, or might be triggered by commits to version control.

Our continuous deployment means daily, Monday to Thursday nights. We don't deploy Friday in case there are problems and we don't have anyone working on the weekends.

Our big nightmare is that something goes wrong with an update and all of our roughly 500 installations will be down. Of course, we have a big test suite that has to pass, but tests are never perfect. To reduce the risk we deploy to about ten "beta" sites first. Everyone else gets that update two days later. Having ten beta sites down is something we can handle, and they're aware they're beta sites so they're a little more understanding. In practice, we've had very few major problems.

We have a single central version control (similar to Subversion). Anything committed to version control automatically gets deployed. The problem is when we're working on a bigger or riskier change, we can't send it to version control until it's finished. But not committing regularly leads to conflicts and merge issues, and also means we're only tracking the changes with a large granularity and can't revert back to intermediate steps. Plus, version control is the main way we share code. If the changes haven't been sent to version control, it's awkward for other people to get access to them for review or testing. I think the solution would be a distributed version control system like Git or Mercurial where we can have multiple repositories.

I'm looking forward to reading Continuous Delivery although I think the focus is on web applications.

1 comment:

Jen said...

Although, a few of us have been pretty good at making the changes in smaller steps to that they can be sent to version control (and it's easier to find when we break things). Or to make the changes in copies of the original records so that they can be kept in version control until finished and ready to be used.

Knowing that whatever changes you make are going right out to customers definitely makes you think more about how you write your code and the testing (written and manual).

It definitely helps to have standards in place, such as pair programming and getting your code reviewed before sending, to help keep the code that is getting sent out working and bug free. The hard part is getting all the programmers to actually follow the standards :o)