PCI-DSS v4.0.0 Control Baseline for Red Hat OpenShift Container Platform 4

  apiVersion: machineconfiguration.openshift.io/v1
  kind: KubeletConfig
  metadata:
     name: kubelet-config-$pool
  spec:
      machineConfigPoolSelector:
          matchLabels:
              pools.operator.machineconfiguration.openshift.io/$pool_name: ""
      kubeletConfig:
        tlsMinVersion: VersionTLS12
  

authentication:
  ...
  anonymous:
    enabled: false
  ...

authorization:
  mode: Webhook
  ...

authentication:
...
  x509:
    clientCAFile: /etc/kubernetes/kubelet-ca.crt
...

apiVersion: machineconfiguration.openshift.io/v1
kind: KubeletConfig
metadata:
   name: kubelet-config-$pool
spec:
    machineConfigPoolSelector:
        matchLabels:
            pools.operator.machineconfiguration.openshift.io/$pool_name: ""
    kubeletConfig:
        eventRecordQPS: 
       

  apiVersion: machineconfiguration.openshift.io/v1
  kind: KubeletConfig
  metadata:
     name: kubelet-config-$pool
  spec:
      machineConfigPoolSelector:
          matchLabels:
              pools.operator.machineconfiguration.openshift.io/$pool_name: ""
      kubeletConfig:
        tlsCipherSuites:
        - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
        - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
        - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
        - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
        - TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256
        - TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256
  

The configuration tlsSecurityProfile specifies TLS configurations to be used while establishing connections with the externally exposed servers. Though secure transport mode is used for establishing connections, the protocols used may not always be strong enough to avoid interception and manipulation of the data in transport. TLS Security profile configured should not make use of any protocols, ciphers, and algorithms with known security vulnerabilities.

tlsSecurityProfile can be configured to use one of custom, intermediate, modern, or old profile. Profile Old should be avoided at all times and when using custom profile one should be extremely careful as invalid configurations can be catastrophic. It is always advised to configure minimum TLS version to TLSv1.2 or latest when using Custom profile or to use predefined profiles Intermediate or modern. If a TLS security profile is not configured, the default TLS security profile is Intermediate.

To configure Custom tlsSecurityProfile for the Kubelet with TLSv1.2 as minimum TLS version, create a new or modify existing KubeletConfig object along these lines, one for every MachineConfigPool:

  apiVersion: machineconfiguration.openshift.io/v1
  kind: KubeletConfig
  metadata:
      name: kubelet-tls-config-$pool
  spec:
      tlsSecurityProfile:
          type: Custom
          custom:
              ciphers:
              - ECDHE-ECDSA-CHACHA20-POLY1305
              - ECDHE-RSA-CHACHA20-POLY1305
              - ECDHE-RSA-AES128-GCM-SHA256
              - ECDHE-ECDSA-AES128-GCM-SHA256
              minTLSVersion: 
      machineConfigPoolSelector:
          matchLabels:
              pools.operator.machineconfiguration.openshift.io/$pool_name: ""
  

For more information, follow OpenShift documentation: the relevant documentation.

...
rotateCertificates: true
...

featureGates:
...
  RotateKubeletClientCertificate: true
...

makeIPTablesUtilChains: true

serverTLSBootstrap: true

apiVersion: machineconfiguration.openshift.io/v1
kind: KubeletConfig
metadata:
   name: kubelet-config-$pool
spec:
    machineConfigPoolSelector:
        matchLabels:
            pools.operator.machineconfiguration.openshift.io/$pool_name: ""
    kubeletConfig:
        streamingConnectionIdleTimeout: 
       

Two types of garbage collection are performed on an OpenShift Container Platform node:

• Container garbage collection: Removes terminated containers.
• Image garbage collection: Removes images not referenced by any running pods.

Container garbage collection can be performed using eviction thresholds. Image garbage collection relies on disk usage as reported by cAdvisor on the node to decide which images to remove from the node.

The OpenShift administrator can configure how OpenShift Container Platform performs garbage collection by creating a kubeletConfig object for each Machine Config Pool using any combination of the following:

• soft eviction for containers
• hard eviction for containers
• eviction for images

To configure, follow the directions in the documentation

This rule pertains to the imagefs.available setting of the evictionHard section. Remediation will set field imagefs.available to {{ .var_kubelet_evictionhard_imagefs_available }} based on the variable var_kubelet_evictionhard_imagefs_available.

Two types of garbage collection are performed on an OpenShift Container Platform node:

• Container garbage collection: Removes terminated containers.
• Image garbage collection: Removes images not referenced by any running pods.

Container garbage collection can be performed using eviction thresholds. Image garbage collection relies on disk usage as reported by cAdvisor on the node to decide which images to remove from the node.

The OpenShift administrator can configure how OpenShift Container Platform performs garbage collection by creating a kubeletConfig object for each Machine Config Pool using any combination of the following:

• soft eviction for containers
• hard eviction for containers
• eviction for images

To configure, follow the directions in the documentation

This rule pertains to the memory.available setting of the evictionHard section. Remediation will set field memory.available to {{ .var_kubelet_evictionhard_memory_available }} based on the variable var_kubelet_evictionhard_memory_available.

Two types of garbage collection are performed on an OpenShift Container Platform node:

• Container garbage collection: Removes terminated containers.
• Image garbage collection: Removes images not referenced by any running pods.

Container garbage collection can be performed using eviction thresholds. Image garbage collection relies on disk usage as reported by cAdvisor on the node to decide which images to remove from the node.

The OpenShift administrator can configure how OpenShift Container Platform performs garbage collection by creating a kubeletConfig object for each Machine Config Pool using any combination of the following:

• soft eviction for containers
• hard eviction for containers
• eviction for images

To configure, follow the directions in the documentation

This rule pertains to the nodefs.available setting of the evictionHard section. Remediation will set field nodefs.available to {{ .var_kubelet_evictionhard_nodefs_available }} based on the variable var_kubelet_evictionhard_nodefs_available.

Two types of garbage collection are performed on an OpenShift Container Platform node:

• Container garbage collection: Removes terminated containers.
• Image garbage collection: Removes images not referenced by any running pods.

Container garbage collection can be performed using eviction thresholds. Image garbage collection relies on disk usage as reported by cAdvisor on the node to decide which images to remove from the node.

The OpenShift administrator can configure how OpenShift Container Platform performs garbage collection by creating a kubeletConfig object for each Machine Config Pool using any combination of the following:

• soft eviction for containers
• hard eviction for containers
• eviction for images

To configure, follow the directions in the documentation

This rule pertains to the nodefs.inodesFree setting of the evictionHard section. Remediation will set field nodefs.inodesFree to {{ .var_kubelet_evictionhard_nodefs_inodesfree }} based on the variable var_kubelet_evictionhard_nodefs_inodesfree.

-a always,exit -F dir=/var/log/kube-apiserver/ -F perm=r -F auid>=1000 -F auid!=unset -F key=access-audit-trail

-a always,exit -F dir=/var/log/oauth-apiserver/ -F perm=r -F auid>=1000 -F auid!=unset -F key=access-audit-trail

-a always,exit -F dir=/var/log/openshift-apiserver/ -F perm=r -F auid>=1000 -F auid!=unset -F key=access-audit-trail

$ sudo chmod 0700 /var/log/kube-apiserver/

$ sudo chmod 0700 /var/log/oauth-apiserver/

$ sudo chmod 0700 /var/log/openshift-apiserver/

/var/log/kube-apiserver/

$ sudo chown root /var/log/kube-apiserver 

$ sudo chown root /var/log/kube-apiserver/* 

/var/log/oauth-apiserver/

$ sudo chown root /var/log/oauth-apiserver 

$ sudo chown root /var/log/oauth-apiserver/* 

/var/log/openshift-apiserver/

$ sudo chown root /var/log/openshift-apiserver 

$ sudo chown root /var/log/openshift-apiserver/* 

$ sudo chmod 0600 /var/log/kube-apiserver/.*

$ sudo chmod 0600 /var/log/oauth-apiserver/.*

$ sudo chmod 0600 /var/log/openshift-apiserver/.*

$ sudo chgrp root /etc/cni/net.d/*

$ sudo chgrp root /etc/kubernetes/static-pod-resources/kube-controller-manager-pod-*/configmaps/controller-manager-kubeconfig/kubeconfig

$ sudo chgrp root /var/lib/etcd/member/

$ sudo chgrp root /var/lib/etcd/member/wal/*

$ sudo chgrp root /etc/kubernetes/manifests/etcd-pod.yaml

$ sudo chgrp root /etc/kubernetes/static-pod-resources/*/*/*/*.crt

$ sudo chgrp root /var/lib/cni/networks/openshift-sdn/.*

$ sudo chgrp root /etc/kubernetes/static-pod-resources/kube-apiserver-pod-*/kube-apiserver-pod.yaml

$ sudo chgrp root /etc/kubernetes/static-pod-resources/kube-controller-manager-pod-*/kube-controller-manager-pod.yaml

$ sudo chgrp root /etc/kubernetes/static-pod-resources/kube-scheduler-pod-*/kube-scheduler-pod.yaml

$ sudo chgrp root /etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/node-kubeconfigs/*.kubeconfig

$ sudo chgrp root /var/run/multus/cni/net.d/*

$ sudo chgrp root /etc/kubernetes/static-pod-resources/*/*/*/tls.crt

$ sudo chgrp root /etc/kubernetes/static-pod-resources/*/*/*/*.key

$ sudo chgrp root /var/run/openshift-sdn/cniserver/config.json

$ sudo chgrp root /run/ovn-kubernetes/cni/ovn-cni-server.sock

$ sudo chgrp root /var/lib/ovn/etc/*.db

/var/run/openvswitch/ovs-vswitchd.pid

openvswitch

hugetlbfs

/run/openvswitch/ovs-vswitchd.pid

openvswitch

hugetlbfs

/run/openvswitch/ovsdb-server.pid

openvswitch

hugetlbfs

$ sudo chgrp root /etc/kubernetes/static-pod-resources/kube-scheduler-pod-*/configmaps/scheduler-kubeconfig/kubeconfig

$ sudo chown root /etc/cni/net.d/* 

$ sudo chown root /etc/kubernetes/static-pod-resources/kube-controller-manager-pod-*/configmaps/controller-manager-kubeconfig/kubeconfig 

$ sudo chown root /var/lib/etcd/member/ 

$ sudo chown root /var/lib/etcd/member/wal/* 

$ sudo chown root /etc/kubernetes/manifests/etcd-pod.yaml 

$ sudo chown root /etc/kubernetes/static-pod-resources/*/*/*/*.crt 

$ sudo chown root /var/lib/cni/networks/openshift-sdn/.* 

$ sudo chown root /etc/kubernetes/static-pod-resources/kube-apiserver-pod-*/kube-apiserver-pod.yaml 

$ sudo chown root /etc/kubernetes/static-pod-resources/kube-controller-manager-pod-*/kube-controller-manager-pod.yaml 

$ sudo chown root /etc/kubernetes/static-pod-resources/kube-scheduler-pod-*/kube-scheduler-pod.yaml 

$ sudo chown root /etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/node-kubeconfigs/*.kubeconfig 

$ sudo chown root /var/run/multus/cni/net.d/* 

$ sudo chown root /etc/kubernetes/static-pod-resources/*/*/*/tls.crt 

$ sudo chown root /etc/kubernetes/static-pod-resources/*/*/*/*.key 

$ sudo chown root /var/run/openshift-sdn/cniserver/config.json 

$ sudo chown root /run/ovn-kubernetes/cni/ovn-cni-server.sock 

$ sudo chown root /var/lib/ovn/etc/*.db 

$ sudo chown openvswitch /etc/openvswitch/conf.db 

$ sudo chown openvswitch /etc/openvswitch/.conf.db.~lock~ 

$ sudo chown openvswitch /var/run/openvswitch/ovs-vswitchd.pid 

$ sudo chown openvswitch /etc/openvswitch/system-id.conf 

$ sudo chown openvswitch /run/openvswitch/ovs-vswitchd.pid 

$ sudo chown openvswitch /run/openvswitch/ovsdb-server.pid 

$ sudo chown root /etc/kubernetes/static-pod-resources/kube-scheduler-pod-*/configmaps/scheduler-kubeconfig/kubeconfig 

$ sudo chmod 0600 /etc/cni/net.d/*

$ sudo chmod 0600 /etc/kubernetes/static-pod-resources/kube-controller-manager-pod-*/configmaps/controller-manager-kubeconfig/kubeconfig

$ sudo chmod 0700 /var/lib/etcd

$ sudo chmod 0600 /var/lib/etcd/member/wal/*

$ sudo chmod 0600 /etc/kubernetes/manifests/etcd-pod.yaml

$ sudo chmod 0600 /etc/kubernetes/static-pod-resources/etcd-*/secrets/*/*.crt

$ sudo chmod 0644 /var/lib/cni/networks/openshift-sdn/*

$ sudo chmod 0600 /etc/kubernetes/static-pod-resources/kube-apiserver-pod-*/kube-apiserver-pod.yaml

$ sudo chmod 0600 /etc/kubernetes/static-pod-resources/kube-controller-manager-pod-*/kube-controller-manager-pod.yaml

$ sudo chmod 0600 /etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/node-kubeconfigs/*.kubeconfig

$ sudo chmod 0644 /var/run/multus/cni/net.d/*

$ sudo chmod 0600 /etc/kubernetes/static-pod-resources/kube-*/secrets/*/tls.crt

$ sudo chmod 0600 /etc/kubernetes/static-pod-resources/*/*/*/*.key

$ sudo chmod 0600 /run/ovn-kubernetes/cni/ovn-cni-server.sock

$ sudo chmod 0640 /var/lib/ovn/etc/*.db

$ sudo chmod 0640 /etc/openvswitch/conf.db

$ sudo chmod 0600 /etc/openvswitch/.conf.db.~lock~

$ sudo chmod 0644 /var/run/openvswitch/ovs-vswitchd.pid

$ sudo chmod 0644 /etc/openvswitch/system-id.conf

$ sudo chmod 0644 /run/openvswitch/ovs-vswitchd.pid

$ sudo chmod 0644 /run/openvswitch/ovsdb-server.pid

$ sudo chmod 0644 /etc/kubernetes/static-pod-resources/kube-scheduler-pod-*/kube-scheduler-pod.yaml

$ sudo chmod 0600 /etc/kubernetes/static-pod-resources/kube-scheduler-pod-*/configmaps/scheduler-kubeconfig/kubeconfig

$ sudo chmod 0444 /var/run/openshift-sdn/cniserver/config.json

$ sudo chgrp root /etc/kubernetes/kubelet.conf

$ sudo chgrp root /etc/kubernetes/kubelet-ca.crt

$ sudo chgrp root /var/lib/kubelet/kubeconfig

$ sudo chgrp root /etc/systemd/system/kubelet.service

$ sudo chown root /var/lib/kubelet/config.json 

$ sudo chown root /etc/kubernetes/kubelet.conf 

$ sudo chown root /etc/kubernetes/kubelet-ca.crt 

$ sudo chown root /var/lib/kubelet/kubeconfig 

$ sudo chown root /etc/systemd/system/kubelet.service 

$ sudo chmod 0644 /etc/kubernetes/kubelet.conf

$ sudo chmod 0644 /etc/kubernetes/kubelet-ca.crt

$ sudo chmod 0600 /var/lib/kubelet/kubeconfig

$ sudo chmod 0644 /etc/systemd/system/kubelet.service