What Is DevOps?

DevOps is “Developers” and “Operations” smashed together and reflects a new movement that tears down the wall that traditionally separated developers from system administrators. At its core, DevOps is all about automating your infrastructure using tools and practices that previously were used only by developers to manage code, such as source control, automated testing and recovery. You will see how the DevOps movement arose as a consequence of several other prominent trends in computing such as Agile development, web/mobile development, big data, NoSQL, virtual machines and cloud computing.

DevOps and the Agile Movement

You can equate the DevOps phenomenon with the rise of the Agile movement or even see it as an extension of Agile development into the infrastructure domain. Developers were traditionally restricted to coding and possibly some architectural decisions about the organization of the software. The time scale and releases were measured in months if not years. Agile brought a different cadence and also direct involvement of the developers with the customers. All this software needs hardware to run on, to be configured and connected to other pieces of software. Required changes to infrastructure outpaced the traditionally heavy process of request and approval.

DevOps and Web/Mobile Development

The switch from shrink-wrapped software, or even user-controlled download of software, to web-based software and automatic updates of desktop software was another big factor. Think about the last time you installed software on your computer and had to go through a horrible wizard with multiple windows and a EULA of thousands words. It doesn’t happen anymore. In those days, the bulk of the software an organization developed was run, and the artifacts stored, on the user’s machine. The organization kept a database of users, sales, etc., but it didn’t really need to scale much as more and more users bought or downloaded the software. Even if the software required more resources when new versions were developed, it was taken care of by Moore’s Law — as users bought newer and stronger machines and not by the organization having to acquire and configure more hardware. In the web/mobile era the trend reversed itself. The users’ machines, while still very capable of running computations and storing data, often depended on a central web site or web application that collected and measured every interaction and provided seamless access to any device. Again, this required the organization to expand its infrastructure as the user base grew and as the software became more sophisticated.

By the way, this trend is slowing down or even reversed these days with the advent of strong, fast browsers and powerful mobile devices that can again run the bulk of computation of the client-side, but the data still more often than not remains on the back end.

DevOps and Big Data

The rise of big data is a no brainer motivator for DevOps. Orders of magnitude more data is not just a quantitative change. It is a qualitative change. For example, if it formerly took 10 minutes to backup some data, then you could trust a person to do it as part of a check list and if the backup failed for some reason, you would just try it again. Now, if the same backup takes 10 days, the data is distributed across multiple machines and can almost never be completed in one run, you have to think differently. You will have to develop a distributed program to backup in parallel, you will worry about consistency and network partition. This is a whole different ball game. DevOps was pretty much the only way to manage the rising sophistication of operations.

DevOps and NoSQL

For a long time important data was stored in heavy expansive relational databases such as Oracle, IBM DB2 and Microsoft SQL. For various reasons I will not get into here, this has changed and many companies started to delve in other Non-SQL databases. The complexity of storage and data management shot up significantly. No more DBAs to manage, fine tune and take care of the data. Now, it was the developers’ job and they needed some tools to do it, especially when exploration into the uncharted waters of NoSQL databases often meant switching from one solution to another or even utilizing multiple solutions at the same time. DevOps was a necessity to manage the beast.

DevOps and Virtual Machines

Virtual machines were yet another big trend that ushered DevOps into the mainstream. With virtual machines you could easily and cheaply create and experiment with complicated topologies and have a full production environment simulated on one machine or replicated to a staging environment. That elevated the quality and testability of complex distributed systems, but required a more sophisticated approach for configuration management. DevOps was the just the right thing.

DevOps and the Cloud

Enter the cloud. The success of the cloud development took away the drudgery of manually dealing with servers, but brought with it a slew of other considerations. While, you could provision a small deployment just by using the cloud management console, and it was often better than traditional home-grown scripts and tools for monitoring, to really utilize the cloud at larger scale you want to use APIs, which fits DevOps like a glove. Traditional sys admins are comfortable SSH-ing into a machine and fine-tuning it or writing a shell/per/python script that can do it over multiple machines, but interacting with REST APIs is another story.

Do You Need DevOps?

This is the big question. If your organization uses traditional operational procedures where a sys admin/IT person has full control over everything and you need to submit a form and wait for 3 weeks to open a port on a server, then it might be a good idea to consider DevOps. Whether you are in a decision making position or just a junior developer, the way forward is risk-free experimentation. Read up, play with the tools and if the organization decides to pursue DevOps then do it carefully. It is pure and simple software development in a new domain — the domain of software infrastructure. If your business is the next “Uber for turtles” you may want to make sure you are ready to take the plunge and have the ability to make DevOps work. This requires at least one person with deep system administration skills and decent programming skills.

Popular Tools and Frameworks

DevOps is so hot right now (yes, Zoolander). New tools, and even companies dedicated to support these tools or manage your infrastructure, for you pop up every day. The DevOps software itself is always open source. Puppet and Chef are two Ruby-based frameworks that are considered the incumbents. Both have been around for a while (Puppet first) and manage the infrastructure of some big companies. I’ve tried to work with Chef and came out a little disappointed. It felt very heavy-handed and required installing agents on each machine you want to manage. My favorite DevOps framework is the Python-based Ansible. Ansible operates by default in push-mode over SSH and supports provisioning, configuration management and orchestration.

The Future of DevOps

DevOps will continue to rise and make its way into more and more organizations. A new trend in the DevOps world is using containers like Docker to abstract the actual deployment site even farther than virtual machines. This opens up opportunities for a lot of innovation. Another cool trend is ChatOps, which uses chat bots like Hubot to take administrative actions, share infrastructure status and notifications via chat rooms. So, whether you adopt DevOps today or not, pay attention and stay tuned.