Continuous deployment and continuous delivery are sometime used as synonyms – but they are actually quite different.

The difference between the two is business considerations – continuous deployment doesn’t care about the business aspects only technical considerations around deployment into a new environment. On the other hand continuous delivery takes into account business considerations around deployment, and uses them as a gate for deployment decisions into production.

Continuous deployment is the technical capability of being able to automatically move artifacts from one environment to the next and ensure that the new environment is set up and configured properly. That is the focus of most DevOps activity and tools.

Continuous delivery is deployment from staging environments to the production environments – aka release. The difference between deployment and delivery are the unique attributes of the production environment. Even though the technical capability for automatic promotion to production may be the same as for continuous deployment  – there are considerations for deployment to production that are different than those for any other environment. One example is that production has a unique set of provisioning and configuration needs not used by any other environment; another example is business gates always exist when promoting to production, and may not exist for any other environment.

Automation is enough to for continuous deployment, but continuous delivery is another matter. Production assurance is the link between the technical capabilities of continuous deployment to the business goal of enhanced delivery velocity and availability – aka continuous delivery.

For long term success, DevOps teams will need to provide the business with more than just DevOps automation, they’ll also need to provide production assurance to lower the overall risk of release, not just increase release velocity. The current DevOps focus is on the automation needed to provision and configure environments, and to deploy application artifacts into those environments. Modern deployment paradigms (virtualization, containers) are making application deployment topologies and configuration more complex, which is causing DevOps automation complexity to increase as well.

The role of assurance in developing application code is clear, no one releases application features that haven’t been thoroughly tested – unit tests, integration tests, regression tests etc. The only app testing requirement driven by DevOps is that application testing at all levels must be automated – otherwise testing becomes a bottleneck and may be circumvented to achieve velocity.

Not the same can be said for DevOps automation, which is essentially infrastructure and configuration as code. Achieving continuous delivery requires confidence that the deployment code will work correctly in production, and when it breaks the problem will be easy to diagnose and repair. But this just isn’t the case – very little testing is done to release automation code and what little is done is only at the unit test level. There has been some effort to introduce Test Driven Development for DevOps automation tools – but because of the unique attributes of the production environment even when adopted that isn’t enough.

Traditional testing falls short for DevOps automation because it is impossible (or at least uneconomical) to test deployment automation in production – companies do not have a “spare” production environment for deployment testing. This means production deployment is exposed to human errors during the coding of the automation (aka bugs) which will only increase along with the adoption of DevOps and new, more complex, deployment architectures (micro-services and containers).

Because of its importance, and the difficulty of testing deployment in a production environment, DevOps needs production assurance – a method to validate production deployment to ensure the correctness of the production topology and configuration, the ability to differentiate between application and deployment bugs, and a feedback loop that links production deployment problems to the artifacts and staging environments that caused the problem. There are four steps to enabling Production Assurance:

  1. DevOps automation based on an explicit deployment model.
  2. Replicable, versioned staging environments.
  3. DevOps automation development using the same development and testing rigor as application code.
  4. Validated production deployment topologies.
  5. A way to quickly differentiate application bugs from deployment bugs and a fast, easy mapping of deployment problems back to the specific staging environments that created them.

Adopting DevOps without Production Assurance increases the risk of deployment issues stemming from mistaken configuration or insufficient testing. As DevOps matures and deployment architectures become more complex (e.g. containers) automated deployment (aka configuration as code) is becoming more like programming  – and configuration issues become a complex interleaving of human error and automation bugs.

Businesses adopt DevOps in order to achieve increased delivery velocity, but it can’t come at the expense of availability. Velocity can be measured in a variety of ways, from a purely technical measure like time-to-null-deploy to more business oriented measures like the ratio between number-of-deliveries per quarter, month, day or even hour and the number of rollbacks (or roll-forwards). Availability is the ratio between uptime and operation time and needs to take into account mean time to detect, triage and repair a problem (triage is the gross ability to assign the problem to Dev or Ops). DevOps is focused on velocity, production assurance – availability.

Production assurance is complementary to DevOps, extending DevOps methodology during and after application production deployment, aka release. DevOps focuses on automation of all environments while production assurance focuses the unique aspects of the production environment and how it relates to previous environments. Production assurance provides visibility into and assurance of complex deployments in order to minimize incorrect deployment and the time to detect, triage, diagnose and repair deployment issues.