Technical note

History of DevOps

From the chaos of pre-DevOps releases to Puppet, Docker, and Kubernetes — how DevOps came to be.

Part 1 of a DevOps and Site Reliability series.

To master DevOps, it's important to understand how it emerged in the first place. So let's begin at the beginning, and start with understanding the history, need, and birth of DevOps.

The Pre-DevOps Era: 2007

Before DevOps came into the picture, releases and deployment were chaotic events. Last-minute issues and rollbacks on important features were common. Operations engineers didn't have much insight into what went into the product, and mostly went into release days blindsided. Build infrastructure was inconsistent and relied on a bunch of snowflake servers. Changes made by the team were reactive and ad hoc. Configuration changes were sometimes missed in certain places, leading to system failures and major production issues.

Although tools like CruiseControl and Hudson were used for continuous integration, they were still in very early stages, and weren't technologically advanced enough to handle a lot of the complications that occurred. Nagios was broadly used for monitoring, but its effectiveness lost meaning since the tool couldn't identify high-priority warnings from the haystack of alerts.

All of these issues combined, along with more, made the whole process a chaotic environment. This is where the need for DevOps was felt.

Winds of Change: 2008

In 2008, three major tools emerged and gained a name for themselves: Puppet, Borg, and LXCs.

Puppet was a widely used configuration management tool. Puppet drove change management through a centralized configuration management system. To establish this, developers wrote infrastructure as code and published it to a centralized server. Puppet made sure all systems automatically pulled changes from this centralized server and configured themselves — eliminating the possibility of missed configuration updates, and ensuring absolute consistency.

LXCs, or Linux Containers, also emerged around the same time. Built on top of cgroups and namespaces, the Linux kernel tried to offer process resource isolation to the world. Meanwhile, Google had already migrated to Borg, its own cluster manager for containers.

Around the same time, Patrick Debois and Andrew Clay Shafer, who coincidentally met at an agile developers' conference, formed the Agile System Administration group, which would later gain a lot of recognition in the world of DevOps.

Inception Year: 2009

In 2009, cloud computing was new and a cutting-edge technology in the industry. AWS, with its VPCs, Elastic Block Store, auto-scaling, and other appealing features, had become globally recognized. A number of companies started leveraging these services and dynamically building solutions on top of them.

Around the same time, a series of events, conferences, and discussions led Patrick Debois and a few more enthusiasts to start a conversation about achieving collaboration between Development and Operations. In the midst of that conversation, the term "DevOps Day" was coined, and people started actively looking forward to it.

In terms of persistent tools, there was also a visible subset of Puppet developers who migrated from Puppet to Chef, a more sophisticated, complex, yet upgraded version of Puppet — which probably paved the way for Ansible to emerge.

Era of Chaos: 2010–15

The years 2010 to 2015 are said to be the Era of Chaos, since a number of tools emerged, became popular, and declined from the industry in quick succession. There was a spike in the number of choices available for development, deployment, tooling, and orchestration.

  • Hudson and Jenkins split, each claiming to be the parent of the other. After the split, Jenkins emerged as the more popular choice, and came to dominate continuous integration tools.
  • Ansible took a completely different approach to configuration management — simple yet sophisticated — which beat both Puppet and Chef in the race to become more popular.
  • Linux Containers (LXC) came out of namespaces, cgroups, and the Linux kernel, which enticed a lot of development lifecycles.
  • Docker was open-sourced by dotCloud as a result of low revenue and no funds, and later went on to become one of the most popular technologies of all time.
  • Kubernetes, initially known internally as Seven of Nine, was open-sourced by Google for the world to use. It was a friendlier version of Borg, and remains the most popular orchestration system to date.
  • Swarm was released as Docker's own orchestration system.
  • Mesos, a hybrid orchestration system developed by Apache, was also released, with a framework named Marathon.

The unusually large number of options available made standard choices disappear, and increased the chaos around DevOps between 2010 and 2015.

Maturity Years: 2016–18

Amid all the chaos, we saw Kubernetes grow to become the most preferred orchestration engine of all. One contributing factor was that Kubernetes constantly evolved — adding features including scaling, presets, and Windows support early on, while also collaborating and becoming friendlier with its competitors, Docker and Mesos. This added a lot of value to the technology. We saw platforms like AWS and Microsoft Azure release their own versions of Kubernetes, and it made sense for giant organizations like GitHub to fully migrate to it.

On the other hand, Ansible emerged as the most preferred tool for configuration management. Since Docker and Kubernetes were now in use, the need for a configuration management system in that layer was reduced, since most configurations were pre-built. The need for configuration management remained for configuring underlying systems — installing Kubernetes, network and system configuration, patching, and compliance. This is where Ansible emerged as a simple yet efficient tool that supported container orchestration engines. Red Hat acquired Ansible in 2015, giving it the final boost it needed to become one of the most dominant tools in configuration management.

Lastly, when it came to continuous integration, Jenkins has always been the industry standard, even today.


Reference and credits for facts: Gourav Shah, DevOps Trainer, Author & Consultant at School of DevOps.

← All writing