Software complexity. It’s one of the greatest impediments to DevOps culture adoption, according to a 2018 survey by Quali. Other significant obstacles include “stagnant organizational cultures,” the “jumble of legacy processes, IT infrastructure and newly created cloud environments,” and “lack of standardized mechanisms for continuous testing.”
But forewarned is forearmed; and if you’re aware of the risks, you’ll be better equipped to handle them and ensure they don’t pose a threat to your planned organizational changes. Let’s explore how exactly software complexity can complicate the adoption of a DevOps culture and what challenges you might face, including those specific to DevOps-related technologies.
The “Tech Is Easy” Fallacy
One of the greatest challenges of managing software complexity in a DevOps transformation is that the tooling involved often doesn’t align with the organizational culture and structure. Achieving such a transformation requires many changes, software being just one.
When it comes to creating a DevOps culture, techies generally tend to focus on the purely technological aspects of the adoption process. They often ignore or downplay the complexity that can result when implementing changes across an organization made up of non-technical staff and when dealing with new technical challenges.
For example, one of the key organizational changes achieving a DevOps culture requires is the “you build it, you run it” approach. But this can lead to unexpected complications if the team that developed the software lacks the expertise to actually operate it in production.
Suddenly, a team that’s used to understanding the intricacies of Javascript or database schema upgrades can find itself needing to address operational issues such as the provisioning of infrastructure or the management of network ports, load balancers, and firewalls. This can lead to unexpected delays and confusion as the team learns and adapts to these new concerns.
This is precisely why so many DevOps transformation guides emphasize the importance of team composition. “You build it, you run it” brings with it great freedom, but as the previous R&D and operations silos no longer exist, there is a greater responsibility. And you can no longer blame the “other team,” so it’s vital to have the necessary skills to collaboratively solve the problems close at hand or else be faced with overwhelming technical challenges.
“Bias for the New” and Legacy
When contemplating introducing technical changes, there’s an inherent bias in favor of new technologies, which often promise to solve all your problems without the gnarly complexity that existing operations have built up over time.
While these new generations of technology often genuinely do solve the problems of traditional technologies, this bias can lead to a severe underestimation of the challenges involved in adopting new technologies.
It’s also worth bearing in mind that the early adopters of new technologies within companies tend to be both technically stronger and more experienced in them than the majority of employees who will end up using them. It’s easy to underestimate the difficulties of training and mentoring the company’s broader, non-tech staff to use this new tech.
In addition to the technological difficulties this brings as well as the need to provide ongoing support for those legacy applications that are still running, this also poses organizational challenges.
If long-standing employees are not given the chance to take part in this new endeavor and to contribute their knowledge, this can result in resentment and lack of cooperation. Worse, this can lead to a new “them and us” silo in place of (or even in addition to) the old R&D and operations silos. It is therefore extremely important to get all your staff on board to make the transition as smooth as possible and to guarantee success.
The New Technologies
Now on to the specifics of the new technologies associated with DevOps transformations and how they can contribute to the issue of complexity.
Kubernetes
Kubernetes has undoubtedly been the greatest transformational technology to come about in recent years. Originally sponsored by Google as an open source project, Kubernetes has come to dominate the technical landscape. It seeks to leverage another new technology—containers—to create a platform for running applications across entire data centers.
However, it is very challenging to implement Kubernetes within an organizations for a number of reasons. First, as a relatively new and fast-growing tech, just keeping up with all its latest developments is a full-time job. Entire sub-projects and initiatives come and go on a monthly basis, while trends emerge and disappear.
And that’s just the ecosystem surrounding Kubernetes. Its actual operation is a whole nother serious technical challenge, and one for which standards and tooling are still emerging. Early adopters of Kubernetes have especially struggled with networking, the software supply chain, and upgrades.
So great are these difficulties, RedHat even made an early strategic bet with their OpenShift platform that businesses would pay a significant premium to have most of these challenges managed for them. Arguably, OpenShift was a significant factor in IBM’s $34 billion acquisition of RedHat.
The Kubernetes technology contains many other technologies within it that all need to be understood. In addition, the adoption of Kubernetes itself requires adopting new microservices architectures. While the trail has been blazed in the microservices space by lighthouse companies such as Netflix and Monzo, many organizations have still failed to make the shift. This is in large part due to the complex problems that emerge when an organization needs to shift from a monolithic architecture.
Conway’s Law states that an organization’s hierarchy tends to reflect the technical architecture of its principal product. It follows from this that microservices architectures demand a distributed team structure, and trying to retrofit a microservices organization structure onto one built around a monolithic organization structure leads to all kinds of complexities that can significantly slow down adoption.
GitOps
GitOps is another technology where the mainstream lacks the cutting edge. While many technical staff are entirely sold on the benefits of having their entire infrastructure represented in code, significantly more engineers struggle to understand Git itself; and they feel less than comfortable automating their entire build system, despite the benefits it offers in terms of agility and change management.
Cloud Technologies
And, finally, there is the sheer complexity and richness of the cloud technology space to contend with. In addition to the many challenges of moving your operations, code, and data from data centers under your control to virtual resources operated by Amazon, Google, or Microsoft, you have to navigate all the services they offer.
In 2019, AWS offered 165 distinct services, with almost daily announcements. Just keeping up to date with AWS news is hard enough, let alone having to follow what’s happening on Google Cloud Platform and Azure as well.
Configuring and managing all these services can be overwhelming. There have been several high-profile cases of sensitive data exposed on the public internet. The work required to prevent such cases from happening under your watch can also result in the growth of significant organizational complexity.
Overcoming Complexity
Overcoming the complexity that can arise when adopting a DevOps culture is no easy feat. As discussed previously, many of these challenges stem from two main issues. First, your organizational structure may need to change in order to reflect the new technical structure. Second, new technologies present multiple challenges that require either external expertise or investment in training internal staff.
Despite the challenges, adopting a DevOps culture is an achievable goal. By implementing changes in your organizational structure as new technologies are introduced, your team is sure to overcome the obstacles along the way and guarantee a successful DevOps transformation.
