This post was mentioned on Twitter by mfisher5kavika: New blog post: Continuous Deployment http://bit.ly/QRN2U...

While not universally true, smaller releases have proven in many companies to result in smaller more manageable issues upon release (as compared to the larger issues that tend to be happen within very large releases). More frequent releases, however, tend to create more frequent (albeit) smaller issues. Once again – these points don’t hold true everywhere but as a generalization they are consistent with our past experiences while running tech organizations and companies and advising our current clients.

More frequent releases also offer the benefit of faster time to market. If a company releases once a quarter and a feature is ready one week after the last release date, it might wait another 12 weeks to release as compared to a company that releases weekly in which the average delay for a release will be 3.5 days (assuming an even distribution of probability of completion across the 7 day week).

The question then should be what is the value of the process as compared to the cost and risk? We’ve always pushed our clients to more frequent releases to reduce the magnitude of a potential issue and decrease time to market. But continuous releases take this to an extreme! I suspect that the comments are correct and that this process is great in some areas (assuming that you have appropriate QA automation and risk mitigation) and potentially not appropriate for others.

Questions I would look to have answered in determining release frequency/velocity:
1) Quality cost of the frequency/velocity? Do we need more or less QA? What’s our cost of quality assurance?
2) Relative quality – does the actual AND customer perceived quality go up or down? Note – these are 2 separate issues in many cases. Sometimes customers don’t like things to change underneath their feet daily and while your absolute quality may go up – customers may perceive something different. Customers may also like the rapid change as the magnitude of any change is imperceptible (v. a release of Office 2007 for instance..>)
3) Cost of development – including and excluding cost of quality – what happens to your overall development costs? Do your engineers have more available hours for development or less? Do they get more done or less? Do they create more value or less?
4) Value creation – similar to (3) above – but are engineers getting more features out the door and are they generating more revenue overall?
5) Risk and magnitude of impact – Are you causing more problems or less? Are the problems bigger or smaller?
6) Time to market – are you faster, the same or slower with the changes? From ideation to launch – are you getting things out the door faster or more slowly?

Some of these obviously overlap – and I’m sure there are more. Any more ideas on the right questions by which to evaluate this or any other release process?

Thanks for the comments!

I agree with Julian’s comments that continuous deployment probably depends highly on the specific industry in which it is applied. However, even in highly non-fault-tolerant industries (medicine, nuclear, NASA, etc.) continuous deployment can be implemented, as long as it is done in stages.

You can continuously deploy to development servers, and even QA servers and realize a lot of the benefit of the practice. There just has to be even more that goes into the production release.

Thanks for the post!

Mr. Hericus