DevSecOps build and test process

April 6, 2022  |  Keith Thomas

In the previous article about the coding process, we covered developers using secure coding practices and how to secure the central code repository that represents the single source of truth. After coding is complete, developers move to the build and test processes of the Continuous Integration (CI) phase. These processes use automation to compile code and test it for errors, vulnerabilities, license conformity, unexpected behavior, and of course bugs in the application.

The focus of DevSecOps is to help developers follow secure-coding best practices and open-source licensing policy that were identified in the planning process. In addition, DevSecOps helps testers by providing automated scanning and testing capabilities within the build pipeline.

What is in a build pipeline?

Build pipelines run on highly customizable platforms like Microsoft Azure DevOps, Jenkins, and Gitlab. The build pipeline pulls source code from a repository and packages the software into an artifact. The artifact is then stored in a different repository (called a registry) where it can be retrieved by the release pipeline. Jobs in the build pipeline perform the step-by-step tasks to create an application build. The jobs can be grouped into stages and run sequentially every time the build process is run. Jobs need a build server, or pools of build servers to run the pipeline and return a built application for testing.

Pipeline DevSecOps

DevSecOps partners with developers by inserting additional source code scanning tools as jobs into the build pipeline. The tools used depend on what is being built and is usually determined through DevSecOps collaboration with the development team to understand the architecture and design of the code. For most projects, DevSecOps should implement at a minimum, the scanning tools that look for vulnerabilities, poor coding practices and license violations.

Source code scanners

Pipelines allow automated application security (AppSec) scans to be run every time a new build is created. This capability allows DevSecOps to integrate static analysis (lint) tools like source code scanners that can run early in the software development lifecycle. Security scanners come in two forms: static application security testing (SAST) and dynamic application security testing (DAST).

SAST is run early in the development lifecycle because it scans source code before it is compiled. DAST runs after the development cycle and is focused on finding the same types of vulnerabilities hackers look for while the application is running.

SAST can look for supply chain attacks, source code errors, vulnerabilities, poor coding practices, and free open-source software (FOSS) license violations. SAST speeds up code reviews and delivers valuable information early in the project so developers can incorporate better secure coding practices. Picking the right SAST tool is important because different tools can scan different coding languages. By automating scanning and providing feedback early in the development process, developers are empowered by DevSecOps to be proactive in making security related code changes before the code becomes an application.

Container image scanners

Application builds that create containers for microservices like Docker are stored in a registry as an image artifact. These images have application code, additional software packages, and dependencies that are needed to run the application. Sometimes the images are built by the developers and other times are pulled from a public repository like Github.

Source code scanners review the source code, image scanners review the built application, packages, and dependencies. Image scanners look for container vulnerabilities and exploits like supply chain attacks and crypto jacking software.

Image scanners should be run during the build process so that vulnerabilities are identified and remediated by the development team quickly. Keeping an image small (fewest needed packages and dependencies) is a great (and easy) way for developers to reduce the attack surface of the image and speed up security scanning and remediating vulnerabilities.

In addition to image scanning, DevSecOps recommends the following criteria to protect the application. Images should be configured to not run on the host system using the admin (root) account. This protects the host from privilege escalation if the application is compromised.

Images should be signed by a trusted certificate authority so they have a trusted signature that can be verified when the image is deployed to an environment. Images should be stored in a dedicated image repository so that all internal microservices platforms (Docker and Kubernetes) only pull “approved” images.

Test process

Testing is one of the first environments that an application build is deployed into. Testing teams use tools like Selenium and Cucumber to help automate as much of the testing as possible. Automated test plans can benefit from iterative improvements that increase the test plan quality every time a build is created. DevSecOps has open-source tools like ZAP that support proxying and can sit between the testing tools to perform security scanning as the tests are examining the application. Bringing DevSecOps and the testing teams together helps builds trust and collaboration while speeding up testing and reducing the number of scripts and tools necessary to complete the testing process.

Bending the rules

Outages, quality issues, and common mistakes can happen when there is pressure to deliver in a compressed timeframe. Building and testing is where bending the rules may be accepted or even the current norm within the teams. Security scanners are designed to stop the build process if audits and compliance fail. If the development and testing teams are unaware of this risk, it will appear as builds and tests breaking. They will complain to their leaders who will come to the DevSecOp team and demand the tools get out of the way of the success of DevOps.

DevSecOps overcomes these concerns by being an integral part of the team with developers and testers. Coordination between DevSecOps and developers is also promoted by adding the findings from these tools into the same bug tracking tools used by testers. DevSecOps integrates by speaking about the changes and listening to incorporate the feedback loop, create inclusiveness, and collaborate to help everyone understand what the tools are doing, how they work, and why they are important.

Next steps

Security scanners help developers follow secure-coding and license compliance practices. Scanners and feedback work best when performed as early as possible in the build pipeline so adjustments can be made quickly and with minimal development impact. Using automation encourages developers and testers not to bend the rules. With the application built and tests complete, the software is ready to be packaged as a release.

Share this with others

Get price Free trial