Navigating the path towards Modern IT with XOps

Hi! My name is Chris Alleaume, and I work for Dell Technologies as an advisory consulting solutions principal, focused on all things "Automation".

Fairly recently, Dell Technologies held Dell Technologies World.
I was incredibly excited to be sharing some knowledge and meeting customers in person.

Since my primary focus is on the evolution of DevOps and “bridging the gap” between Application Developers and IT Ops, I really wanted to hear first-hand perspectives. 

I experienced many interactions with IT Operations personae who seemed to be looking for advice on either 1) how to use / adopt Kubernetes or 2) how to encourage their administrators and operations teams to learn how to adopt a “DevOps culture”. This is indeed cultural in nature. There are two different types of teams that exist in organizations; those who are forward-looking and those who are stuck in a traditional IT culture. Naturally, given today's cyber security climate, there is now, more than ever, a much larger focus on security AND automation.

This brings me to the subject of DevSecOps.

Before we get there, however, let's scratch the surface on XOps. There are many terms which all fall under the primary blanket "DevOps" term. When we think about some of the constituents of DevOps, there are encompassing elements which are common to the XOps terms. 

First, adopting a DevOps process typically involves an agile methodology. Think of this approach as planning quick iterations (sprints) to produce functionality towards a minimally viable product (MVP).

Source control and pipelines come to mind, as well, which brings me to GitOps. When we employ Infrastructure-as-Code (IaC), along with Configuration-as-Code, we can adopt GitOps, where we store declarative configurations for infrastructure and applications in source control.
The benefits of adopting GitOps include (obviously) a declarative approach to defining application and infrastructure configurations as code, version history and automation through pipelines, triggered by changes made to those configuration files. GitOps falls under the "XOps" umbrella, so to speak, as does DevSecOps. 

DevSecOps involves rolling security in your DevOps process. There are various permutations to what "security" means in a DevSecOps process, depending on what our customers see as the most pertinent elements of security. 

What is DevSecOps?

An iteration of DevOps which includes Security aspects - this helps to bridge the gap between Developers (or engineers), IT Operations and IT Security teams.

Why is it important?

Considering the proliferation in hacks, ransomware and cyber attacks, security is top of mind for the majority of our customers – with due reason. Instead of being dangerously reactive to security compliance, the idea is to adopt security compliance best practices in the earlier stages of automated deployments, leaning towards a more proactive approach.

How is it implemented?

There are multiple areas to which the implementation or adoption of DevSecOps applies, some of which I'll elaborate on below.

Applications and Automation

Applying best practices to code principles, dependencies and standards involves automating linting and scanning by using CI/CD Pipelines. This means that whenever code is committed to a repository, scans are executed to ensure that the code and dependencies are within a certain threshold of compliance.

For example, when writing an application with library / package dependencies, committing code to repository initiates a series of scan jobs. These scan jobs could be configured to perform the following types of scans:

•  Linting (static code analysis)
  • Validate code structure, syntax and organizational best practices
  • Check for plain-text passwords or hard-coded values which are subject to serve as reference points for exploiters

• Dependency scanning
  • When referencing dependent libraries or packages, any known vulnerabilities linked to these packages and libraries can be black-listed or flagged for additional security verification

 

Container images

Images can be built from scratch, however, in order to save time, many organizations and developers leverage pre-built images as the foundation for their own images.

In either case, Image Repositories can be embellished to adopt image scanners which scan for  known vulnerabilities against Operating Systems, known images, libraries, packages, applications, practices and a series of security guidelines particular to containerized applications.

This means that all images uploaded (pushed) to an image repository will be flagged if not compliant with certain thresholds and disallowed from reaching production until remediated.

Virtual Machines

Virtual Machines are typically cloned from Templates in order to simplify and accelerate deployments. This opens up two possible phases for security scanning:

• When building clean, new VM templates:

• Just before a workload deployment is handed over to the requestor:

When scanning for security compliance on Linux machines, for example, a tool like OpenScap is installed dynamically and executed to perform a scan of the workload. The scan itself uses a set of rules which can be customized beforehand to suit security needs. Once complete, the scan process generates a report indicating compliance, which is weighted against the severity of each failed rule.
When a scan falls below a certain threshold, remediation can be dynamically generated and applied. Should a second scan fail after remediation attempts, the pipeline or process fails, disallowing the creation of Templates or Workloads which fall below a certain threshold.

If template VMs have passed security scans, this means that each workload deployed from a template will be secure by default when booting up for the first time.

Once the workload has been configured, perhaps with applications installed, an additional scan is executed to ensure that other processes have not undone any of the preceding checks, maintaining acceptable levels of security / hardening compliance.

The Journey

Many customers tend to ask, "how do we harness DevOps to improve our operational engineering?", or, "how do we get our teams to adopt a DevOps approach?".

This brings us to two particular challenges:

1. Culture
2. Operating Model

In order to adopt any kind of process, there are multiple factors to consider. How do I excite my teams and brighten their perception? How do I prepare my teams to adopt a more modern mindset? How do I teach my operations teams how to code?

In a fairly recent delivery for a customer, we assisted a large enterprise in adopting DevOps and promoting certain operations staff to Site Reliability Engineers.

Those members who were willing to advance their careers and "stay relevant" were keen to learn, and became SREs.

This ultimately resulted in a 30% reduction in Operational Staff members, since the SRE role was far more streamlined and reduced the need for as many traditional operations staff members.

From an operating model perspective, Dell Technologies has a team dedicated to this. We assist our customers in the process of aligning operating models to a specific business objective. For example, if customers want to adopt a multi-cloud approach, we help them establish gaps in skills, opportunities for enablement, strategies in budget realignment and, ultimately, equipping the business to support the adoption of new or refined processes.

Where to from here?

Dell Consulting Services has a profound level of experience with the implementation of DevOps practices, including Infrastructure-as-Code, GitOps, DevSecOps, SRE enablement etc.

Our teams have delivery models where we not only assist in automation best practices, but also consult and "teach to fish" while seeding those best practices, enabling our customer teams to take their next steps with confidence and knowledge.

Did you know?

For Infrastructure-as-Code implementations for Dell-specific hardware (eg VxRail, Redfish, iDRAC, PowerMax, etc.), we have a collection of Ansible modules and other code repositories on github.com/dell

Informative publications and demos:

Storage Automation Platform: Powerstore

Automation Services for Storage

GitOps drives new IT infrastructure automation agilities