Skip to main content

CIS Benchmark Kubernetes - PodSecurityPolicies

CIS Benchmark Kubernetes - PSP

The CIS document provides prescriptive guidance for establishing a secure configuration posture for Kubernetes. The Pod Security Policies (PSP) enable fine-grained authorization of pod creation and updates.  In this article, we'll review the CIS benchmark items for Pod Security Policies and provide implementation details on how to enforce them on Kubernetes cluster. Let us first review the CIS benchmark guidance for Pod Security Policies. 

CIS Benchmark Overview

The following items are checked for this benchmark item:

  • 1.7 PodSecurityPolicies
    • 1.7.1 Do not admit privileged containers 
    • 1.7.2 Do not admit containers wishing to share the host process ID namespace 
    • 1.7.3 Do not admit containers wishing to share the host IPC namespace 
    • 1.7.4 Do not admit containers wishing to share the host network namespace 
    • 1.7.5 Do not admit containers with allowPrivilegeEscalation 
    • 1.7.6 Do not admit root containers 
    • 1.7.7 Do not admit containers with dangerous capabilities

In practical terms, these best practices may not apply to each and every pod being deployed in the system.    When deployed on our Kubernetes cluster, we will use this as the default policy across the cluster, and will selectively grant permissions on a targeted basis.

Design Overview

This benchmark requires that Kubernetes PodSecurityPolicies be enforced for all pods running in the system.   The following diagram shows the basic pod security policy configuration data model:


Practically, we may have some pods that will require some elevated permissions, but most will not.  To cater to the requirements, let us define two basic pod security policies: 

  • restricted
  • privileged

restricted. This is a policy that simply prevents the creation of privileged pods.  


apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: restricted
  annotations:
    seccomp.security.alpha.kubernetes.io/defaultProfileName:  'docker/default'
    seccomp.security.alpha.kubernetes.io/allowedProfileNames: 'docker/default'
  labels:
    kubernetes.io/cluster-service: 'true'
    addonmanager.kubernetes.io/mode: Reconcile
spec:
  privileged: false
  allowPrivilegeEscalation: false
  requiredDropCapabilities:
    - ALL
  volumes:
    - 'configMap'
    - 'emptyDir'
    - 'projected'
    - 'secret'
    - 'downwardAPI'
    - 'persistentVolumeClaim'
  hostNetwork: false
  hostIPC: false
  hostPID: false
  runAsUser:
    rule: 'MustRunAsNonRoot'
  seLinux:
    rule: 'RunAsAny'
  supplementalGroups:
    rule: 'MustRunAs'
    ranges:
      - min: 1
        max: 65535
  fsGroup:
    rule: 'MustRunAs'
    ranges:
      - min: 1
        max: 65535
  readOnlyRootFilesystem: false
  forbiddenSysctls:
  - '*'

privileged. This is a policy that allows the creation of privileged pods.  

apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: privileged
  annotations:
    seccomp.security.alpha.kubernetes.io/allowedProfileNames: '*'
  labels:
    kubernetes.io/cluster-service: "true"
    addonmanager.kubernetes.io/mode: Reconcile
spec:
  privileged: true
  allowPrivilegeEscalation: true
  allowedCapabilities:
  - '*'
  volumes:
  - '*'
  hostNetwork: true
  hostPorts:
  - min: 0
    max: 65535
  hostIPC: true
  hostPID: true
  runAsUser:
    rule: 'RunAsAny'
  seLinux:
    rule: 'RunAsAny'
  supplementalGroups:
    rule: 'RunAsAny'
  fsGroup:
    rule: 'RunAsAny'
  readOnlyRootFilesystem: false
  # This will fail if allowed-unsafe-sysctls is not set accordingly in kubelet flags
  allowedUnsafeSysctls:
  - '*'
PodSecurityPolicy enforcement is performed by an admission controller.   This controller must be pre-configured and enabled in Kubernetes deployments.  To check if PodSecurityPolicy admission controller is enabled, grab a shell into kube-apiserver pod and search for enabled admission plugins as shown below.

> kubectl exec -ti kube-apiserver -n kube-system -- sh
# kube-apiserver -h | grep enable-admission-plugins
...
     MutatingAdmissionWebhook, NamespaceAutoProvision, NamespaceExists, NamespaceLifecycle, NodeRestriction, OwnerReferencesPermissionEnforcement, PersistentVolumeClaimResize, PersistentVolumeLabel, PodNodeSelector, PodPreset, PodSecurityPolicy, PodTolerationRestriction, Priority, ResourceQuota, RuntimeClass, SecurityContextDeny, ServiceAccount, StorageObjectInUseProtection, TaintNodesByCondition, ValidatingAdmissionWebhook. 

Enforcement is performed whenever a new pod is created, usually by a Deployment, ReplicaSet, or some other templated controller.    The authorization for the pod creation action is configured using RBAC Roles (or ClusterRoles) which are bound to a User or ServiceAccount.  Thus PodSecurityPolicy enforcement can be configured and enforced at either a cluster or namespace granularity. 

Next, let us create the following two ClusterRoles that have verb use granted on the PodSecurityPolicy created above respectively.

  • psp:privileged. This ClusterRole is granted the verb use on PodSecurityPolicy privileged created earlier.
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: psp:privileged
rules:
- apiGroups:
  - policy
  resourceNames:
  - privileged
  resources:
  - podsecuritypolicies
  verbs:
  - use
  • psp:restricted. This ClusterRole is granted the verb use on PodSecurityPolicy restricted created earlier.
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: psp:restricted
rules:
- apiGroups:
  - policy
  resourceNames:
  - restricted
  resources:
  - podsecuritypolicies
  verbs:
  - use

Enforce Policy Use Cases

Now that we have created PodSecurityPolicies with different privileges and ClusterRoles that can use those policies, let us enforce these policies for different use cases outlined below.

Restrict system:authenticated subjects by default

To enforce restricted PodSecurityPolicy on all authenticated users by default, we will bind the psp:restricted ClusterRole on system:authenticated subjects.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: psp:any:restricted
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: psp:restricted
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: Group
  name: system:authenticated 

Allow nodes to create privileged pods

Here the user kubelet is allowed to create privileged pod by creating a RoleBinding object with the role psp:privileged.

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: psp:nodes:privileged
  namespace: kube-system
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: psp:privileged
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: Group
  name: system:nodes
- apiGroup: rbac.authorization.k8s.io
  kind: User
  name: kubelet

Allow kube-system namespace subjects to create privileged pods 

To enforce a PodSecurityPolicy on all subjects in a namespace kube-system, we can bind a ClusterRole that we created earlier to all the serviceaccounts in that namespace as shown below.

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: psp:kube-system:privileged
  namespace: kube-system
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: psp:privileged
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: Group
  name: system:masters
- apiGroup: rbac.authorization.k8s.io
  kind: Group
  name: system:serviceaccounts:kube-system

With above PSP in place, services that are run in there own namespace will be started with restricted permissions. If those services require higher privileges, we will see two kinds of PSP enforcement errors:

  • container has runAsNonRoot and image has non-numeric user (e.g., kibana), cannot verify user is non-root
  • container has runAsNonRoot and image will run as root

Ideally, we would define a custom PSP for use in the services namespace. 

Allow kube-system namespace subjects to create privileged pods

To allow services to run with increased privileges in a namespace svc-namespace, create a RoleBinding as shown below.

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: psp:svc-namespace:cs-restricted
  namespace: svc-namespace
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: psp:privileged
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: Group
  name: system:serviceaccounts

Conclusion

We reviewed CIS Kubernetes Benchmark, especially the guidance for Pod Security Policies. We created PSP to enforce those guidance. Practically, all services may not run with these restrictions. Some services may need to define a more relaxed pod security policy. We provided configurations for these different use cases.

Labels:

Comments

Post a Comment

Popular posts from this blog

Supporting OpenTracing jaeger in spring boot applications

This page describes code changes in a typical spring-boot based application to support OpenTracing and jaeger. Instrumenting a tracer If you are creating a simple spring boot application that uses  spring-boot-starter-web , by default, the application does not support writing traces to jaeger. To support jaeger tracing, the first thing is to modify the build.gradle to add dependency of jaeger: dependencies {      implementation  'org.springframework.boot:spring-boot-starter-web'      implementation  'io.opentracing.contrib:opentracing-spring-web-starter:3.0.1'      // support opentracing jaeger      implementation  'io.opentracing.contrib:opentracing-spring-jaeger-starter:3.1.2'      testImplementation( 'org.springframework.boot:spring-boot-starter-test' ) {          exclude group:  'org.junit.vintage' , module:  'junit...
Post a Comment
Read more

HOWTO on implementing scanning into a CI Pipeline

Introduction As a part of the Software Security Assurance guidelines, we are required to perform various types of security scanning.  Security scanning is a specialized kind of testing that attempts to identify potential software bugs that could make a deployed application vulnerable to attack and compromise.  As with any testing, the tests can be manual or automatic, and they wan be performed at various points in the development cycle.   We can classify security scanning into three general categories: Static Application Security Testing (SAST)   - performed against source code / configuration data a static scan looks for common logic errors that might lead to system compromise. Dy namic Application Security Testing  (DAST)  - Performed against a running system, a dynamic scan looks for vulnerable software / configurations that might lead to system compromise. Security Monitoring  - Deployed as a part of the system, a security monitor co...
Post a Comment
Read more

Fortify Tooling User Guide

  Introduction The fortify-tools container is located within a shared repository in OCIR and requires a JWT to be able to access.  The variable WF_JWT will need to be set to a valid MAT You will want to choose one of three ways to use the Fortify SCA tooling: Integration Description Using the Fortify Tools Standalone to Scan a Project This is for using the tooling without integration in GitLab CI or Jenkins CI. Using the Fortify Tools In GitLab CI to Scan a Project This is for using the tooling against a project that whose code is hosted in GitLab and whose CI engine is GitLab CI. Using the Fortify Tools In Jenkins CI to Scan a Project This is for using the tooling against a project that whose code is hosted in GitLab and whose CI engine is Jenkins CI. Using the Fortify Tools Standalone to Scan a Project Simple Usage Run the Fortify Tools in a container docker run -t --rm -v <path to project source root directory>:/var/fortify/src phx.ocir.io/oraclegbudevcorp/cn-shared/s...
Post a Comment
Read more