The rise of the DevOps team methodology in modern software development has had a profound effect on application security in both good and bad ways. For example, shortened release cadences have introduced opportunities to both increase and decrease the security of release candidates.
In the case of increased or improved security, members of DevOps teams have greater insight into the overall code and structure of their projects, pull requests invite openness and improve code quality, and automated testing catches problems early. Meanwhile, the opposite effect can also be seen where rapid releases may attract short-sighted coding practices for the sake of speed, poorly vetted automated scanners may not provide the depth of security coverage required for a product or service, and teams may even become desensitized to the overabundance of data available leading to missed or ignored alerts and issues.
Unfortunately, for many small to medium-sized businesses (SMBs) these opportunities often weigh heavily on the side of decreased or diminished security. It is not difficult to see the attraction of sacrificing a strong, tested security posture for increased delivery speeds and features when time is short and small teams are overtaxed. But SMBs should not (and do not) have to make this choice. There are common best practices to follow that fuse security into the DevOps process without forfeiting the agile performance of a well-oiled DevOps team. While these practices are applicable to organizations and teams of all sizes, this article intends to present these practices of a Secure Development Lifecycle in a manner appropriate for smaller businesses and teams.
Shift Left 
If you follow a DevOps methodology, you are already familiar with the term “Shift Left.†It is commonly used in reference to defect management. It is easier, cheaper, and more effective to fix issues early in the software development process (on the left side of the process) than it is later in the process. This applies to security as well since, after all, poorly implemented security is indeed a defect. So, start your security process at the beginning. It should be well integrated into the project and product initiation functions such as market analysis, cost evaluation, project planning, design, and requirements gathering.
The following practices are not one-time checklist items. They are all part of the continuous feedback loop evaluated and updated throughout the lifecycle. However, when the bulk of plan, design, and architecture is shifted to the left, each subsequent iteration should be targeted to only new features, processes, and factors making the ongoing process efficient and operationally viable even for small teams.
Define Application Security Standards
Know the security and privacy requirements of your business and technology. Depending on your industry, data, and even location of your target audience this may refer to regulatory compliance requirements such as PCI-DSS, HIPAA/HITECH, FedRAMP, GDPR, or CCPA. But security requirements do not end with those enforced by law or regulation. You should also define your expectations for internal code quality, cryptography standards, management of known infrastructure threats, and associated weaknesses inherent to a specific platform or technology. Make sure to identify these standards up front and architect toward them rather than modifying them later.
Perform Threat Modeling
Simply put, you cannot protect against threats you do not know. Threat Modeling is a process of visualizing threats and vulnerabilities to a system, application, or component to identify mitigations and appropriate protections. Attempting to apply standards such as OWASP TOP 10 or SANS 25 to an application environment, the PCI or HIPAA standard to a system, or secure coding practices to a component without first envisioning what you are building is like throwing darts at a dartboard blindfolded. You may know the direction you are aiming, but you will only hit a bullseye accidentally.
“You cannot build what you cannot draw.†I don’t know who coined the phrase, but I have heard it over and over throughout my career in many ways and referring to many subjects. It is as true for security as well.
Provide Application Security Training for Team Members
Security is everyone’s responsibility in DevOps teams. There is a pattern emerging in DevOps-oriented organizations (and especially in smaller businesses) to eliminate or restrict collaboration with siloed security teams during the development of an application relegating their participation to scheduled interactions, such as penetration testing, on a finished product. This is, of course, a dreadful decision.
As a product evolves, new risk factors are introduced. As platforms emerge or are updated, new risk factors are introduced. Role-based security training is necessary for a healthy secure development lifecycle because people are your eyes and ears to these risk factors before they become a problem.
Recovering from a single security incident that could have been avoided with proper knowledge and training can be very damaging to both the finances and the reputation of any organization. You do not need the security department to bless every commit, but team members should possess the skills to apply security best practices at their level of involvement.
Define Meaningful Application Security Metrics
Purposeful and meaningful metrics are essential to the enduring security of your system or application. These are the rules and conditions used to evaluate security effectiveness, the ability of a team to respond to threats and incidents, and the assurance of meeting regulatory standards. Effective security metrics are commonly referred to as meeting the SMART (Specific, Measurable, Attainable, Repeatable, and Timely) criteria and should drive action to address both correction and improvement.
The security metrics allow an organization to measure the key performance indicators and security posture of a system to know if it is working as it should, when to make changes, and how to continuously improve. It is also useful in filtering out unnecessary or redundant information and targeting data to the proper audience to prevent data fatigue.
Manage Supply Chain Risk
How do you trust third-party and open-source components and tools used in your software development? You don’t.
It is highly unlikely that your DevOps team is completely detached from any external supply chain. Cryptography libraries, authentication and identity providers, deployment tools, intrusion detection, and prevention platforms, and even language frameworks are all part of your external security supply chain. Whether they are expensive, commercially sourced products or free, open-source products, there are innate risks to relying on someone else for your security.
Managing your supply chain risk involves creating an inventory and components and their dependencies, identifying security risks associated with the components, assessing and mitigating the risks, and monitoring for changes. Maintain a list of approved tools to use, regularly reassess them, and monitor for the use of any unapproved components and tools.
Test, Test, Test
Testing for security vulnerabilities provides a means of assessment, validation, and discovery and should occur throughout the software lifecycle using three different methods:
- Static Analysis Security Testing (SAST) includes the pre-compilation evaluation of source code to identify security vulnerabilities. While often used for code analyzing code quality and adherence to coding policies, SAST may also be used to identify code smells that jeopardize the security of an application. This testing is often included in the integrated development environment as a set of tools that evaluate in real-time as code is written and may also be present in the code repository commit pipeline.
- Dynamic Analysis Security Testing (DAST) is executed by tools that perform a run-time assessment of an application to validate that security functionality runs as designed. This may include diagnostic analysis for operating system and memory level vulnerabilities, authentication processes, misuse of authorized functions and privilege systems, and many other tests such as stress tests and fuzzing to test for data exposure weaknesses. This testing is performed in several stages including unit tests, post-build pipelines, and vulnerability management schedules.
- Penetration Testing is like DAST in that it occurs post-compilation on a running application, but while some penetration testing occurs internally, the most valuable penetration test is completed externally by security professionals who simulate the actions of external bad actors, and in some cases, internal ones as well. The goals are two-fold; (1) to expose system vulnerabilities and operational threats so they can be addressed, and (2) to validate and/or certify the effectiveness of security controls in the production environment of the system which is often required for regulatory compliance.
Plan to Respond
Your application is now tested and deployed, the system is being monitored and your team members are on a continuous learning path to build and operate secure software. The problem is new vulnerabilities are uncovered, bugs are reported, operating systems and third-party components are updated, and compliance requirements change. An Incident Response Plan is needed to establish a protocol for assessing and addressing new security-related events. This plan is a testable framework for evaluating an incident, categorizing and prioritizing the response, and setting the strategy for resolving and reporting the issue.
Rinse and Repeat
You have successfully shifted left and established the bulk of tools, procedures, training, and practices to integrate security into the DevOps process. Now as a new feature is added it fits naturally into the continuous improvement workflow.
- Does it have new security requirements?
- Does it present new threats?
- Do team members know how to secure it?
- Does it have new metrics to evaluate?
- Does it rely on new external components?
- How do you test it?
- How do you respond to problems?
You have the security machinery in place, and the DevOps feedback loop turns the wheel to put it in motion.
Cultivate Application Security Champions
Finally, every DevOps team should have the role of security champion, and in a smaller organization, this role may be entirely different than one in a larger organization. Essentially, the security champion is tasked with keeping track of the security needs of a project. They are responsible for ensuring that security is considered at every phase of the lifecycle, conducting security reviews at project milestones, informing other team members of secure coding practices, reviewing security tagged pull requests, and liaising with the larger security principals as applicable.
What do you look for in a team member to fill this role? They are intimately familiar with the technical aspects of the project in development, they are knowledgeable of secure coding practices, they are in tune with industry-related security developments, and they are inclined to speak when something is wrong. Identify them, train them and cultivate their desire to apply security best practices to the project.
Conclusion
Security in DevOps is sometimes referred to as DevSecOps (or SecDevOps, or Secure DevOps ), but I believe this is a bad idea because it places the two in opposition as if DevOps and secure DevOps are two different concepts. The truth is security is a natural fit into the traditional DevOps process. It is just a matter of applying the security principles and best practices to the phases you already have in place.
We hope you found this post informational. If there are specific challenges you and your team face, the architects and engineers here at Clear Measure would be happy to work with you. Reach out to us now to find out all the ways we can help you!
