If you’re here, you know the basic DevSecOps practices like incorporating proper encryption techniques and embracing the principle of least privilege. You may be entering the realm of advanced DevSecOps maturity, where you function as a highly efficient, collaborative team, with developers embracing secure coding and automated security testing best practices.
This blog is intended to move your team beyond basics to more advanced DevSecOps techniques (such as audit logging and fault tolerance) to detect and respond to the increasing intensity and volume of security attacks to applications and infrastructure. According to the 2022 Cost of a Data Breach study, the longer a security threat lingers, the costlier it gets. Organizations that contain a data breach in 200 days or less save an average of $1.12 million.
The techniques detailed below can help you secure your applications and infrastructure, investigate threats faster, and decrease the potential attack surface for bad actors.
1. Audit logging
What is audit logging?
Think of audit logging as documenting the activity within software systems. An audit log records the occurrence of an event, the time it happened, the responsible user or service, and the entity impacted. An audit trail shows events in order, enabling teams to get a sequential view of what happened on their system. Advanced DevSecOps teams often review audit logs to investigate security breaches and maintain regulatory compliance standards.
Why it’s important
An audit trail can help your organization find out how a breach happened. According to NIST, audit trails can help you accomplish several security-related objectives, including maintaining individual accountability, reconstructing events or actions that happen on your system, detecting intrusion and analyzing problems.
Pro tip
Regularly collecting and retaining logs is critical to your audit logging program. In fact, you probably need to retain logs beyond the typical 30-day retention period to detect advanced persistent threats (APTs). According to UC Berkeley’s information security resources, your audit logging program should, at a minimum, include the following checklist:
- Operating System(OS) Events
- Start up and shut down of the system
- Start up and down of a service
- Network connection changes or failures
- Changes to, or attempts to change, system security settings and controls
- OS Audit Records
- Logon attempts (successful or unsuccessful)
- The function(s) performed after logged on (e.g., reading or updating critical file, software installation)
- Account changes (e.g., account creation and deletion, account privilege assignment)
- Successful/failed use of privileged accounts
- Application Account Information
- Successful and failed application authentication attempts
- Application account changes (e.g., account creation and deletion, account privilege assignment)
- Use of application privileges
- Application Operations
- Application startup and shutdown
- Application failures
- Major application configuration changes
- Application transactions, for example,
- Email servers recording the sender, recipients, subject name, and attachment names for each email
- Web servers recording each URL requested and the type of response provided by the server
- Business applications recording which financial records were accessed by each user
2. Fault tolerance
What is fault tolerance?
Fault tolerance is the ability of a system (such as a computer, network, cloud cluster, etc.) to continue operating without interruption when one or more of its components fail. Load balancing and failover solutions can prevent system outages and ensure high availability. However, closely monitoring your system’s fault tolerance can help you detect an increasing volume of sophisticated attacks.
Why it’s important
Threat actors are launching more and more attacks that impact system operations. These may include:
- Direct path DDoS attacks that target individual organizations
- DNS tunneling attacks that attempt to use malicious domain names or DNS servers to bypass protections
- SQL injection attacks that allow attackers to interfere with queries from applications to databases
- And more.
Pro tip
Event logs contain detailed information regarding state changes in your environment. First, detecting these changes can help you pick up on potentially suspicious activity, outages, or failures on the network caused by malicious actors. Second, it’s critical to secure the sensitive data contained within these logs, such as passwords or access permissions. In Amazon S3, for example, services like AWS Trusted Advisor will help you check for misconfigurations or open access privileges in your S3 buckets that may provide a front door to attackers.
3. Threat hunting
What is threat hunting?
Threat hunting is a purposeful and structured search for evidence of malicious activities that have not yet generated security alerts. It’s a proactive, human-centric security measure that pushes the boundaries of automated detection methods.
Why it’s important
Threat hunting can help detect APTs in the network that mask themselves as legitimate activity. These threats can linger and become very costly and damaging. There are many threat hunting methodologies that provide a well-defined, research-based structure to the approach. Most threat hunters assume the cloud environment has already been compromised and the threat already exists.
Pro tip
One of the most important ways to determine a security organization’s threat hunting ability is the quantity and quality of the log data it collects and makes available to the SecOps team. Most security professionals believe that enriching the systems in their security operations center (SOC) with additional data sources is the most important step they can take to enhance threat hunting capabilities.
Broadly speaking, threat hunters need access to both host and network data sources, as well as cloud application logs. Host logs can be collected via an agent or through native logging applications like Windows Event Forwarding, the Sysmon utility, auditing services for Linux architectures or unified logging for MacOS.
These logs should provide visibility into how configuration management utilities like PowerShell are being used within the environment, since these tools are commonly exploited by attackers seeking to maintain persistence, while keeping a low profile.
Read: Log Analytics and SIEM for Enterprise Security Operations and Threat Hunting
4. Fuzzing
What is it?
According to OWASP, fuzzing (or fuzz testing) is a black box software testing technique that involves finding implementation bugs using automated malformed/semi-malformed data injection. In other words, fuzzers inject data so application testers can watch how an application acts in the presence of malicious and/or random code in the real world.
Why it’s important
For many teams, fuzzing is an important, proactive security check before an application is shipped into production. Fuzzing can show you the quality of the target system and software. Using fuzz testing, you can check the robustness and security risk posture of the system and software application you’re testing.
Fuzzing also is the primary technique attackers use to find software vulnerabilities. Incorporating fuzzing into your SOC can potentially help prevent zero-day exploits from unknown bugs and weaknesses in your system. In many cases, fuzzing can uncover vulnerabilities that otherwise wouldn’t be detected through manual audits or conventional security testing.
Pro tip
Fuzzing can be done at low cost and doesn’t require much human intervention. There are many open source tools and frameworks available to help teams accomplish their fuzz testing goals, including (but not limited to) the following resources:
Open Source Mutational Fuzzers
- American fuzzy lop
- Radamsa - a flock of fuzzers
- APIFuzzer - fuzz test without coding
- Jazzer - fuzzing for the JVM
- ForAllSecure Mayhem for API
Fuzzing Frameworks
5. Automated testing (SAST, DAST, IAST)
What is it?
There are a variety of automated security testing techniques that can help your DevSecOps team build security into the CI/CD pipeline. A few examples include:
- Static application security testing, or SAST: A white box testing technique that scans an application before code is compiled.
- Dynamic application security testing, or DAST: Like fuzzing, DAST is a black box testing technique that mimics a malicious attack in the real world.
- Interactive application security testing, or IAST: Unlike SAST or DAST, IAST works inside the application in real time.
- Mobile application security testing (MAST): Designed specifically for mobile application security, MAST methods include pentesting or automated static and dynamic analysis of code.
Why it’s important
Certain DevSecOps tools can catch bugs you weren’t anticipating in your software applications. Automated security checks are usually worked into the DevOps pipeline, and become an integral part of application development and delivery.
Pro tip
Most organizations (90%) use open-source software in some way. As a result, it’s important to incorporate software composition analysis (SCA) into your automated testing routine. These tools can scan all of your open-source components and dependencies for vulnerabilities. Open source vulnerabilities have grown in 2022 by 30%. In fact, many major issues, such as the infamous Log4J security vulnerability, could have been avoided by updating open-source software and dependencies regularly.
Embracing a DevSecOps approach
A DevSecOps approach that involves secure platform design, automation, and culture changes can ensure that security becomes a shared responsibility throughout the organization. This approach is more important than ever before, given that the cybersecurity industry is facing a global shortage of 3.4 million workers. As a result, DevOps teams must take ownership of security using some of the techniques described above.
The good news is that a variety of tools that already exist within the DevOps toolchain can help with security use cases. For example, centralizing logs within an analytics solution like ChaosSearch can help create a security data lake using existing cloud object storage resources, such as Amazon S3 or Google Cloud Platform. A holistic view into logs with unlimited retention can help provide faster threat detection and incident response capabilities to DevSecOps teams.
