DevOps engineers are essentially the backbone of their IT departments — creating integrated, user-friendly systems quickly.
DevOps
DevOps is a software development method that emphasizes communication, collaboration (information sharing and web service usage), integration, automation, and measurement of cooperation between software developers and other IT professionals. The method acknowledges the interdependence of software development, quality assurance (QA), and IT operations, and aims to help an organization rapidly produce software products and services and to improve operations performance.
"DevOps" is meant to denote a close collaboration and cross-pollination between what were previously purely development roles, purely operations roles, and purely QA roles. Because software needs to be released at an ever-increasing rate, the old "waterfall" develop-test-release cycle is seen as broken. Developers must also take responsibility for the quality of the testing and release environments.
The increasing scope of responsibility of the "developer" (whether or not that term is even appropriate anymore is debatable) has given rise to a chimera-like job candidate: the "full-stack" developer. Such a developer is capable of doing the job of developer, QA team member, operations analyst, sysadmin, and DBA. Before you accuse me of hyperbole, go back and read that list again. Is there any role in the list whose duties you wouldn't expect a "full-stack" developer to be well versed in?
Devops is exciting for developers, and can also be scary. It will change what you need to know and the skills you need in order to succeed. Doing devops requires that you learn new tools and embrace deep cultural changes to the way that you think and work.
You'll have to adapt to new processes in the shorter term, while also anticipating long-term organizational changes. Adopting devops means you'll learn to work differently than you have before, alongside other developers and sysadmins who are also making this big shift.
Below why developer should learn DevOps:
Making the shift
Devops replaces the change control bureaucracy and firefighting that systems administrators are accustomed to with automated continuous deployment and infrastructure as code. The sysadmin becomes an operations engineer, using new tools like Puppet and Chef (and Ansible and Salt and Vagrant and Docker) to design and rapidly spin up standardized, preconfigured systems in a public or private cloud. As a result, wait times go down and reliability goes up.
A similar transformation is at work for developers. You'll have an easier time adjusting to the changes if you are already familiar with Agile methods, but there is still a learning curve.
Devops is more agile than Agile
Devops extends Agile ideas and values and practices to operations as well as development. Devops embraces the same collaboration and transparency, high-bandwidth informal communications, and team-based approach to problem solving that you might be familiar with if you've worked in Agile teams. But in a devops shop you you will end up working with more people, and your responsibilities will broaden.
Agile teams are cross-functional within development; devops teams are cross-functional across development and operations. A devops team includes developers and testers and the product owner, as well as specialists in network engineering and systems engineering, DBAs, information security, application support, and the help desk. Everyone who will be involved in running the complete system in production is part of the development effort.
Just like Agile tears down the walls between the dev team and the customer, and the walls within the team, devops tears down walls between developers and operations. In the same way that Agile requires developers to work with the customer to understand real business needs and priorities and to solve business problems together, devops requires devs to work closely with ops and understand their needs and priorities. Developers and sysadmins solve operations problems together.
Just like Agile brings more decisions about how the system needs to work down to the level of the development team, devops brings more decisions about how the total system is set up and run to the level of the dev-and-ops team.
Continuous delivery vs continuous deployment
Devops is about getting changes to code and operations infrastructure into the hands of users faster. This focus on increasing release velocity means that developers and operations have to find new, safer, and more efficient ways to push changes into production.
Agile development teams use continuous integration with automated testing to make sure that code changes are done correctly. Devops extends this to continuous delivery. Every check-in (meaning code changes, as well as changes to application or infrastructure configuration) runs through a pipeline of tests with feedback along the way.
If a change passes initial unit and functional tests, the system is automatically re-deployed to a testing environment for integration testing, security testing, performance and operational system testing, and other checks. This can mean rebuilding and reconfiguring the operating system and the runtime stack as well as the application from the ground up for each check-in.
Continuous delivery makes sure not only that the code builds and works but that the entire system is integrated and deployed successfully with each change. Continuous deployment takes this one step further. A shop practicing continuous deployment automatically runs every code check-in through the entire test pipeline. If all the tests pass, the changes are automatically deployed to production. This type of rapid deployment happens hundreds or thousands of times daily at places like Amazon and Google.
Done means 'working in production'
In devops, your definition of done needs to change. A feature isn't done just because it has been unit tested and demo'ed to a product owner. It's done when it has been successfully integrated and deployed into production and is being used successfully. Agile stops with working code. Devops takes this to production and back.
Release velocity isn't measured by how many story points the team delivers in a sprint. It's measured by cycle time, which is the time it takes to turn a problem report or change request around to code working in production. In devops, changes tend to become smaller, simpler, and more focused, which minimizes turnaround time.
In many devops shops, instead of working in feature branches or release branches, developers make changes directly to the code mainline, using conditional logic and “feature flippers” to do “branching in code." They then “dark launch” their changes to production, even before the feature is complete, leaving the feature flag off so that the new code doesn't execute. When the feature is ready, it can be turned on incrementally for a few customers in a “canary release," and then progressively rolled out to more customers if the feedback is positive. Another option is to turn on the new feature for some customers and not others and run “A/B tests," monitoring and comparing customer behavior to measure the impact of the change.
Measuring success
Creating feedback loops between operations and development is a critical part of devops. Developers use production feedback to drive decisions, improvements, and changes to the system: not just problem reports and user suggestions, but measurable data collected on how the system is working, on conversion rates or whatever metrics the business uses to determine success.
This means that you need to think about how to instrument your application, what metrics you need to collect, and make sure that the system is hooked into operations monitoring and management interfaces.
Becoming a full-stack developer
In devops you'll spend a lot of time thinking about and doing things other than designing, coding, and unit-testing application features. Some examples:
Packaging and planning feature rollouts (including how to measure success) and working with ops to roll-out system changes including changes to infrastructure.
Building your continuous delivery or continuous deployment pipelines, including automating system tests as much as possible: integration testing, operations testing, security testing, and performance testing.
Data analytics and data science to support A/B testing, designing metrics and analyzing the results.
Monitoring and troubleshooting the system in production, making sure that it is working correctly, and fixing it if it isn't. Amazon's CTO describes this as “You build it, you run it." This can – and probably will – include being on call for code that you wrote in case something goes wrong.
Working through failure and recovery scenarios with ops, and joining in “blameless post mortem” reviews when something goes wrong in production. Everyone involved with the system gets together to understand what happened, what went wrong, why it went wrong, what you and others can do to prevent a problem like this from happening again, and to make it easier to fix and recover from if it happens again. This includes everything from how to improve your design or your testing or the information that the system provides to ops, to how to improve information sharing between people in an incident, and what training or tools people need to do their jobs better. You then feed all of this back into how you work as part of a continuous improvement process, just as you would with Agile retrospectives.
All of this means that you may need to become more of what Facebook calls a full-stack developer: someone who knows a lot more not just about the app, but about the runtime environment, about the network and the operating system, and about how the system is configured, run, and monitored. You'll need to get closer to the metal, to understand more about the technology, how it works, how it scales, and how to make sure that it is secure. This can be a tall order, especially if you're more business focused and less technical.
Your work will also become more interrupt-driven and variable. Devops is a lot more like working in a startup than in an enterprise; a lot more like maintenance than project-based development. Just as Agile breaks big projects down into work that can be done in sprints, devops brings your focus down to piece work: getting a fix or change into production as quickly as possible, making sure that it works, getting feedback, and iterating. You'll also need to find ways to effectively balance support and operations tasks with development. You'll probably find that Kanban is a much better fit for how to manage your work than Scrum.
If you're already working in an Agile/Lean environment on a live system then devops will make perfect sense – you might already be doing devops without knowing it. But if you're used to designing and building a big system and then handing it off to ops and a sustaining engineering team, then devops is going to turn your world inside out.