Don’t be a DevOps Engineer

I imagine that after reading the title some people might ask “Why? Isn’t DevOps a good thing? Aren’t you a DevOps guy yourself?”. These are all good and fair questions. So, yeah, my current job title is DevOps Evangelist. I promote the use of DevOps principles and help people adding said principles to their skill set. DevOps isn’t a bad thing either, actually it’s a great thing, which helps many people produce better software at a much faster speed. So why do I recommend not being a DevOps Engineer? Here is why.

What does that mean?

To explain this, I would like to tell a little bit about how I got into this whole DevOps world. In my last job I was a jack-of-all-trades. I did everything, literally: setting up servers, installing and configuring software, planing and implementing networks, preparing and maintaining databases, message queues, caches, and other backend services, developing and testing own software, handling day2day IT operations, preparing backups, setting up monitoring, creating reports, and so on and so forth. Since it was a small company there were never enough people who could do this. So I had to be smart in order to maximize my performance. One of the things I did to improve my output was automatization. While I tried to automatize as much as possible I slowly entered the territory of what is known as DevOps today, though I didn’t realize it myself.

Only after the company went out of business and while I was looking for a new job I stumbled over this area. At some point around new year 2018 I got an email from a company I had never heard about, saying they saw my profile and would like me to join them as a team leader for their DevOps department. Being a team leader sounded very tempting, so I started investigating what companies think this DevOps thing is and what the job requires. But instead of focusing my investigations around the department and it’s needs I kept it technical and just read everything about DevOps I could get my eyes on. It wasn’t much different from what I did, actually it sounded exactly like my jack-of-all-trades gig I had before. Take care of everything from planning to delivering and running IT solutions. So I accepted and joined the bexio AG in Rapperswil, Switzerland.

Entering the realms of a DevOps team

I had only 2 people in my team to take care of, but those 2 guys were among the most knowledgable and skilled professionals in my field and I felt honored to be their boss. But
my job wasn’t what I had expected. Even though my guys knew their way around the IT landscape much better than me (obviously, I was the new guy), they easily lost focus on business needs, preferred spending time in their little digital worlds, solving complex but small problems, and had a hard time cooperating with the dev team. So I had to do more business tasks than actual technical tasks. Which was OK since I wanted to be a leader, but I admit I missed the tech part very, very much. And then slowly, while I got more and more into my job, I started to see the problems.

In a perfect and (maybe traditional) world you have a couple of teams working together to produce software. You have the product owners, defining requirements. You have the developers creating code. You have the QA team making sure the code fulfills the requirements. You have an Ops team running the code and maintaining the systems the code runs on. That used to work well, when implemented right. But then someone screamed “DevOps is the best” and all of a sudden you find all companies needing this new thing called DevOps, even though nobody understands it, but the promised results sound nice. Better quality, faster production, risk reduction. Who doesn’t like that? OK, but how do you implement that?

The company tried it by creating a DevOps team, sitting between the Dev and Ops. But the results were a disaster. The dev team stopped worrying about how their code performed. The ops team didn’t care about the code. Both teams expected the DevOps guys to take the code and run it. When problems occurred, nobody wanted to handle them, relying on the DevOps team to safe the day. We saved many days. But it was taking a toll. We wanted to help the company become better, so we did what was necessary. But that wasn’t the DevOps I heard about. That was just a bad way of using some tools mentioned in some DevOps books.

Only if you understand the problem, you can solve it

When I finally realized this I talked to my supervisors and the other team leaders. Every. Single. One. And every time I was greeted warmly, my words were heard and pretty much everyone I talked to agreed that we needed to change. But we didn’t. The Dev team was too busy, producing new features as fast as possible. The QA team had zero coding skills and relied on manual testing. Operations was handled by an external company offering web hosting and managed services.

I came up with a plan. We would slowly train the Dev team into taking responsibility for their code again. The QA team started restructuring and learned about automated testing. Operations promised to improve, but unfortunately they just kept doing what they always did: listen to the DevOps team for commands. The progress was slow. Exceptional slow. While we did our best in pushing the plan, we were constantly held up by fire fighting small problems. Devs couldn’t keep their development environments running. QA broke their own automated tests. Operations decided to do harddisk maintenance on a production database during the middle of a business day without notifying anyone. Meanwhile the user base increased daily, so we struggled with performance issues and spent much time on analyzing to find the bottle necks and remove them.

It was too much. Soon the first guy of my team told me officially, that he was on the lookout for a new job and recommended finding a replacement. A month later the second guy just quit. Two days later I found myself talking to someone from middle management, trying to explain to him that the developers should deploy their code themself. He didn’t agree and started the dance with his fellow members, which would lead to more emails, more meetings, and less progress. So I sat down and realized that this wouldn’t be working. Then I wrote my letter of resignation, went to the CEO (CTO was on holiday), and stepped down.

Making changes often requires drastic methods

Try imagine being a manager and seeing this: the only team in the entire company being able to handle updates to your product and handling operations announces its disappearance. Two thirds within a week, the other third already trying to find something else. This was quite a shock. But it had a good effect. I was asked into several meetings with upper management where I explained the reasons behind my decision. I offered suggestions on how to proceed. My final suggestion was to dissolve the entire DevOps team, handing its responsibilities over to the Dev team and create a dedicated Ops team (they changed the name to SRE, another buzzword, invented by Google AFAIK).

Now my plan wasn’t an option anymore. It became a requirement to keep the company functional. That made the difference. Attitudes changed. Procedures changes. Everything changed. But too little, too late for the DevOps team. And probably not fast enough for a safe transition. So I was asked to stick around for some more time. But I refused to just keep doing what I was doing. Instead I offered to become a consultant and help the company change. My offer was accepted. So I became a DevOps Evangelist. Today I teach people and help them helping themself, showing developers and testers how to create CI/CD pipelines, use containerization and automatization to move forward. All job offers for DevOps engineers disappeared and were replaced by offers for Software or SRE engineers. The DevOps team disappeared from the company’s organizational chart. My team left. I remained.

While in the middle of all this, I found a very interesting article online: “If you have a team named DevOps, then you’re doing it wrong” by Amy Harms. I showed this to my team mates and we all agree with her. We weren’t a DevOps team nor were we doing DevOps. We were just a glorified team with a nice name, trying to get this DevOps for the company to work out. Ever since I started first, I got a ton of job offers and contact requests from job hunters, trying to fulfill the high demand for DevOps engineers. After I read Amys article I started to check these offers from a different angle. Most of the time these companies were only looking for someone handling operations or doing things their own developers couldn’t or wouldn’t do. So basically the same thing I just had been trough.

Baseline

If you want to do DevOps, then don’t be a DevOps engineer. Be a developer. Be a tester. Be a system engineer. These are the people that benefit from using DevOps principles. They can improve their work so much but using all these things people connect with DevOps. Think of DevOps not as a job, but rather a way of doing things. And keep in mind, most people out there don’t really understand anything about DevOps, but just want it for their benefit. So don’t let them decide how to do this. Or you become a DevOps engineer.