Introduction
As a part of the Software Security Assurance guidelines, we are required to perform various types of security scanning. Security scanning is a specialized kind of testing that attempts to identify potential software bugs that could make a deployed application vulnerable to attack and compromise. As with any testing, the tests can be manual or automatic, and they wan be performed at various points in the development cycle. We can classify security scanning into three general categories:
- Static Application Security Testing (SAST) - performed against source code / configuration data a static scan looks for common logic errors that might lead to system compromise.
- Dynamic Application Security Testing (DAST) - Performed against a running system, a dynamic scan looks for vulnerable software / configurations that might lead to system compromise.
- Security Monitoring - Deployed as a part of the system, a security monitor continuously evaluates the security status relative to a set of rules or policies and reports anomalies.
Static and Dynamic scanning are typically integrated into the development process; security monitoring is delivered as a feature component of the system. In the OC-CNE project, we integrating static and dynamic scans into our build pipeline and enforce policies to ensure that we meet the OSSA security guidelines. The OC-CNE project implements security scans both to ensure the compliance of our delivered software and to provide an example implementation for 5G NF development teams.
Static Scanning
The OC-CNE project currently implements two kinds of static scanning, with other planned for future development. Static scanning is performed against source repositories or recently built products; these scans do not require a functional deployment/execution environment.
Fortify Scanner (Microfocus.com)
The fortify scanner is one of several OSSA approved scanning tools; the OC-CNE project does not contain any scannable source code and no CI/CD examples have been developed. For initial releases of 5G NFs, projects shall perform manual scanning using an approved tool; we will explore CI/CD integration in future sprints.
OWASP Dependency Checker (jeremylong/DependencyCheck)
The OWASP Dependency Checker will examine project source and identify known vulnerabilities in 3rd party libraries. While the OC-CNE project does not contain any scannable source, a simple proof of concept example was created showing how to integrate the OWASP Dependency Checker into your gradle builds. Here are the details:
Configure gradle to include the OWASP Dependency Check Plug-in:
plugins {
id "org.owasp.dependencycheck" version "4.0.2"
}
apply plugin: 'org.owasp.dependencycheck'
dependencyCheck {
failBuildOnCVSS=7
cveValidForHours=1
format='ALL'
suppressionFile='scanners/owasp_supress.xml'
}
|
And here is a sample invocation:
> Configure project :
Reckoned version: 0.1.0-beta.4.411+41b3893
> Task :dependencyCheckAnalyze
Verifying dependencies for project occne
Checking for updates and analyzing dependencies for vulnerabilities
Generating report for project occne
Found 0 vulnerabilities in project occne
Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https:
BUILD SUCCESSFUL in 30s
1 actionable task: 1 executed
|
When run, a full set of the scan details will be generated in the "build" directory. These include html, cvs, and json reports. The html reports are probably the easiest to view. If you need to suppress any false alerts, you can view the build/reports/dependency_check_reports.html report and press the "suppress" button to capture suppression file fragment, which you can add to a false alert suppression file. Here is an example:
<?xml version="1.0" encoding="UTF-8"?>
<suppress>
<notes><![CDATA[
file name: docker-java-shaded-3.1.0-rc-5-cglib.jar (shaded: com.fasterxml.jackson.core:jackson-databind:2.6.7)
]]></notes>
<gav regex="true">^com\.fasterxml\.jackson\.core:jackson-databind:.*$</gav>
<cpe>cpe:/a:fasterxml:jackson</cpe>
</suppress>
<suppress>
<notes><![CDATA[
file name: docker-java-shaded-3.1.0-rc-5-cglib.jar (shaded: com.google.guava:guava:19.0)
]]></notes>
<gav regex="true">^com\.google\.guava:guava:.*$</gav>
<cpe>cpe:/a:google:guava</cpe>
</suppress>
<suppress>
<notes><![CDATA[
file name: docker-java-shaded-3.1.0-rc-5-cglib.jar (shaded: com.fasterxml.jackson.core:jackson-databind:2.6.7)
]]></notes>
<gav regex="true">^com\.fasterxml\.jackson\.core:jackson-databind:.*$</gav>
<cpe>cpe:/a:fasterxml:jackson-databind</cpe>
</suppress>
</suppressions>
|
McAfee Malware Scanner (www.mcafee.com)
The McAfee malware scanner use has been mandated by Oracle; the OCCNE team has packaged it as a docker container which can be used to scan selected directories for malware:
# Select the latest scan image to get the latest virus defs
export MCAFEE_IMAGE=cgbu-dev-docker.dockerhub-den.oraclecorp.com/occne/mcafee_scan:latest
# To scan a directory other that ${PWD}, adjust the volume selection
docker run -v ${PWD}:/root --rm ${MCAFEE_IMAGE} ./uvscan -v --unzip /root
|
The (cgbu-dev-docker.dockerhub-den.oraclecorp.com) is used to hold images build by the occne team.
In the OC-CNE project, we build and use this container to scan all of the containers created by our Jenkins pipeline. Here the Jenkins stage that performed this scan:
stage('StaticScan') {
when { expression { params.SCAN } }
steps {
script {
sh """
mkdir -p scandir
# extract all of the built images from docker as tar files
docker save ${env.OS_IMAGE} > scandir/os_image.tar
docker save ${env.K8S_IMAGE} > scandir/k8s_image.tar
docker save ${env.CFG_IMAGE} > scandir/cfg_image.tar
docker save ${env.MCAFEE_IMAGE} > scandir/mcafee_image.tar
# scan all of the built images (clean exit only if no malware found...)
docker run -v \${PWD}/scandir:/root --rm ${env.MCAFEE_IMAGE} ./uvscan -v --sub --unzip /root
# cleanup
rm -rf scandir
"""
}
}
}
|
Anchore Engine Vulnerability Scanner (anchore/anchore-engine)
Anchore Engine is a vulnerability scanner that will look a docker containers for known vulnerabilities; the OC-CNE team has added support for anchore-engine scanning in the deployment pipeline as it is in the deployment pipeline that we assemble our list of 3rd party containers (which we then scan). It should be possible for 5G NFs to scan in their CI pipelines as their container list is well known and container tagging should be computed at that point in time.
#!/bin/bash
export AEVOLUME=~/scanners/aevolume
# the IP address of your private repository
export HOSTIP=${HOSTIP:-10.75.216.10}
# default to the lab proxy
# if the directory exists, then we assume that the scanner is already running
if [ -d ${AEVOLUME} ]; then
exit 0
fi
mkdir ${AEVOLUME}
cd ${AEVOLUME}
docker pull docker.io/anchore/anchore-engine:latest
docker create --name ae docker.io/anchore/anchore-engine:latest
docker cp ae:/docker-compose.yaml ${AEVOLUME}/docker-compose.yaml
sed -i \
-e '/^ environment:/a\ \ \ \ - http_proxy='${HTTP_PROXY} \
-e '/^ environment:/a\ \ \ \ - https_proxy='${HTTPS_PROXY} \
-e '/^ environment:/a\ \ \ \ - no_proxy='${HOSTIP}',localhost,127.0.0.1,us.oracle.com,oracle.com,labs.nc.tekelec.com,oraclecorp.com,engine-catalog,engine-policy-engine,engine-simpleq' \
docker-compose.yaml
docker rm ae
# start it up
docker-compose pull
docker-compose up -d
|
Here is a script that will pull an image from a docker registry and initiate a scan:
#!/bin/bash
export IMAGE=${1}
# this should work but (unfortunately) winterfell maps its own name to loopback rather than static ip
# export HOSTIP=$(ping -c1 ${HOSTNAME} | head -1 | awk -F "[()]" '{print $2}')
# name of the private repository host
export HOSTNAME=${HOSTNAME:-$(hostname)}
# IP address of the private repository host
export HOSTIP=${HOSTIP:-10.75.216.10}
# port of the private repository
export HOSTPORT=${HOSTPORT:-5000}
# where did you start the scanner?
export AEVOLUME=~/scanners/aevolume
export LOG=${AEVOLUME}/AE.log
# these are the default credentials
export ANCHORE_CLI_URL=http:
export ANCHORE_CLI_USER=admin
export ANCHORE_CLI_PASS=foobar
anchore-cli()
{
(
cd ${AEVOLUME}
docker-compose exec engine-api anchore-cli ${@}
)
}
fiximage()
{
# the private repository doesn't have DNS resolution - manually resolve it here
echo $1 | sed "s/^${HOSTNAME}:${HOSTPORT}/${HOSTIP}:${HOSTPORT}/"
}
uncomment()
{
cat $* | grep -v '^[ ]*#' | grep -v '^[ ]*$' | grep -v '^[`][`][`]'
}
scanit()
{
image=$(fiximage $1)
echo "Scanning: ${image}" >&2
# restore this line if you want to force a rescan
# anchore-cli image del --force ${image} 2>/dev/null
anchore-cli image add ${image}
anchore-cli image wait ${image}
anchore-cli image vuln ${image} all
}
scanit ${IMAGE} 2>&1 | tee -a ${LOG}
exit 0
|
And here is an example of the kinds of vulnerabilities that this tool can detect:
$ anchore-cli image vuln 10.75.191.98:5000/aschnabl/occne/mcafee_scan:latest all
Vulnerability ID Package Severity Fix Vulnerability URL
ELSA-2019-0049 libgudev1-219-62.0.1.el7 High 0:219-62.0.4.el7_6.2 http:
ELSA-2019-0049 systemd-219-62.0.1.el7 High 0:219-62.0.4.el7_6.2 http:
ELSA-2019-0049 systemd-libs-219-62.0.1.el7 High 0:219-62.0.4.el7_6.2 http:
ELSA-2019-0201 libgudev1-219-62.0.1.el7 Low 0:219-62.0.4.el7_6.3 http:
ELSA-2019-0201 systemd-219-62.0.1.el7 Low 0:219-62.0.4.el7_6.3 http:
ELSA-2019-0201 systemd-libs-219-62.0.1.el7 Low 0:219-62.0.4.el7_6.3 http:
ELSA-2019-0194 bind-libs-lite-9.9.4-72.el7 Medium 32:9.9.4-73.el7_6 http:
ELSA-2019-0194 bind-license-9.9.4-72.el7 Medium 32:9.9.4-73.el7_6 http:
|
Dynamic Scanning
Dynamic scanning is performed against the software running in a deployment / execution environment. The dynamic scans are packaged as a collection of Ansible playbooks delivered in a Docker container image. The scans can be controlled by passing tags; the following scans are available:
- tls - run all the tls related scans
- ciphers - scans for open ports supporting TLS and confirms OSSA TLS cipher conformance
- cis - run all the cis related scans
- docker-bench-security - run the docker cis benchmark scan
- kube-bench - run the K8s cis benchmark scan
- scap - run all the scap related scans
- oval - run the scap oval scan to look for reported vulnerabilities (CVEs)
- xccd - run all of the xccdf related scans
- xccdf-standard - run the Oracle Linux 7 Hardening scan
- xccdf-rht-ccp - run the standard Oracle Linux 7 Cloud Provider Security Scan
- xccdf-stif-rhel7-disa - run the Oracle Linux 7 DOD Security Scan
All of these scans target a cluster - they use a hosts.ini file to identify cluster hosts and resources and then either run scans on specific cluster hosts or sniff open ports on cluster hosts to perform port based-scanning.
Scan Execution using cne_scan
Like all of the OCCNE tools, cne_scan requires that the environment is defined in a kubespray compatible hosts.ini file and access to the SSH keys for the cluster; the scan is invoked using docker; the following script shows how we invoke this tool from of CI/CD pipeline:
set -x
TMP=/tmp/cne_scan\$\$
CLUSTER=${params.CLUSTER}
SSHCREDS=${params.BASTION_USER}@${params.BASTION_HOSTNAME}
REPOHOST=${params.REPO_HOSTNAME}:5000
# clean out any old logs before the new run
rm -rf logs/k8s* logs/scan
ssh \${SSHCREDS} sudo rm -rf /var/occne/cluster/\${CLUSTER}/k8s*
ssh \${SSHCREDS} sudo rm -rf /var/occne/cluster/\${CLUSTER}/scan
for img in aquasec/kube-bench:latest docker/docker-bench-security:latest ; do
# Pull scan containers from public repo to central repo
docker pull \${img}
docker tag \${img} \${REPOHOST}/\${img}
docker push \${REPOHOST}/\${img}
# Pull containers from central repo to cluster repo
ssh -t -t \${SSHCREDS} docker pull \${REPOHOST}/\${img}
ssh -t -t \${SSHCREDS} docker tag \${REPOHOST}/\${img} '`hostname`:5000'/\${img}
ssh -t -t \${SSHCREDS} docker push '`hostname`:5000'/\${img}
done
mkdir \${TMP}
# now fetch the latest oval rules
wget https:
# and push the rules to the bastion
scp \${TMP}/com.oracle.elsa-all.xml.bz2 \${SSHCREDS}:/var/occne/cluster/\${CLUSTER}/com.oracle.elsa-all.xml.bz2
# get rid of the tmp dir
rm -rf \${TMP}
# run the scans
ssh -t -t \${SSHCREDS} 'docker run -it --rm \
--name DEPLOY_${params.CLUSTER}${BUILD_NUMBER} \
--network host -v /var/occne/cluster/${params.CLUSTER}:/host \
-v /var/occne/cluster/${params.CLUSTER}/com.oracle.elsa-all.xml.bz2:/tmp/com.oracle.elsa-all.xml.bz2 \
-v /var/occne:/var/occne:rw -e OCCNEARGS=${params.CNE_SCAN_ARGS} ${params.CNE_SCAN_IMAGE} '
# copy back the results
rsync -a \${SSHCREDS}:/var/occne/cluster/\${CLUSTER}/k8s* logs || true
rsync -a \${SSHCREDS}:/var/occne/cluster/\${CLUSTER}/scan logs || true
|
OSSA Cipherscan
Cipherscan performs OSSA TLS cipher security compliance and vulnerability assessment for hosts. We run this against our deployed OC-CNE environment as a part of our CI/CD self test processes.
The OSSA Cipherscan tools performs the following actions against all nodes in the cluster:
- use nmap to identify all open ports on all hosts (returned as
.../<host>/portscan.xml) - use nmap to identify all open ports on all hosts that accept TLS; for each host / TLS port:
- perform cipherscan to identify TLS ciphers accepted by the host/port
- categorize the cipher selections using OSSA criteria
- Return results as
../<host>/<port>/cipherscan.html
OpenSCAP/openscap
Open SCAP performs security compliance and vulnerability assessment for hosts. We run this against our deployed OC-CNE environment as a part of our CI/CD self test processes. We are currently installing openscap via YUM from the OL7 repository - this is provided by Red Hat.
docker/docker-bench-security
Docker Bench Security checks for dozens of common best-practices for deploying docker containers in production. We run this against our deployed OC-CNE environment as a part of our CI/CD self test processes.
aquasecurity/kube-bench
Kube Bench checks CIS Kubernetes Benchmark conformance. We plan to run this against our deployed OC-CNE environment as a part of our CI/CD self test processes.
Comments
Post a Comment