PCI-DSS v3.2.1 Control Baseline for Red Hat OpenShift Container Platform 4

  apiVersion: machineconfiguration.openshift.io/v1
  kind: KubeletConfig
  metadata:
     name: kubelet-config-$pool
  spec:
      machineConfigPoolSelector:
          matchLabels:
              pools.operator.machineconfiguration.openshift.io/$pool_name: ""
      kubeletConfig:
        tlsMinVersion: VersionTLS12
  

authentication:
  ...
  anonymous:
    enabled: false
  ...

authorization:
  mode: Webhook
  ...

authentication:
...
  x509:
    clientCAFile: /etc/kubernetes/kubelet-ca.crt
...

apiVersion: machineconfiguration.openshift.io/v1
kind: KubeletConfig
metadata:
   name: kubelet-config-$pool
spec:
    machineConfigPoolSelector:
        matchLabels:
            pools.operator.machineconfiguration.openshift.io/$pool_name: ""
    kubeletConfig:
        eventRecordQPS: 

  apiVersion: machineconfiguration.openshift.io/v1
  kind: KubeletConfig
  metadata:
     name: kubelet-config-$pool
  spec:
      machineConfigPoolSelector:
          matchLabels:
              pools.operator.machineconfiguration.openshift.io/$pool_name: ""
      kubeletConfig:
        tlsCipherSuites:
        - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
        - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
        - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
        - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
        - TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256
        - TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256
  

...
rotateCertificates: true
...

featureGates:
...
  RotateKubeletClientCertificate: true
...

makeIPTablesUtilChains: true

serverTLSBootstrap: true

apiVersion: machineconfiguration.openshift.io/v1
kind: KubeletConfig
metadata:
   name: kubelet-config-$pool
spec:
    machineConfigPoolSelector:
        matchLabels:
            pools.operator.machineconfiguration.openshift.io/$pool_name: ""
    kubeletConfig:
        streamingConnectionIdleTimeout: 

Two types of garbage collection are performed on an OpenShift Container Platform node:

• Container garbage collection: Removes terminated containers.
• Image garbage collection: Removes images not referenced by any running pods.

Container garbage collection can be performed using eviction thresholds. Image garbage collection relies on disk usage as reported by cAdvisor on the node to decide which images to remove from the node.

The OpenShift administrator can configure how OpenShift Container Platform performs garbage collection by creating a kubeletConfig object for each Machine Config Pool using any combination of the following:

• soft eviction for containers
• hard eviction for containers
• eviction for images

To configure, follow the directions in the documentation

This rule pertains to the imagefs.available setting of the evictionHard section.

Two types of garbage collection are performed on an OpenShift Container Platform node:

• Container garbage collection: Removes terminated containers.
• Image garbage collection: Removes images not referenced by any running pods.

Container garbage collection can be performed using eviction thresholds. Image garbage collection relies on disk usage as reported by cAdvisor on the node to decide which images to remove from the node.

The OpenShift administrator can configure how OpenShift Container Platform performs garbage collection by creating a kubeletConfig object for each Machine Config Pool using any combination of the following:

• soft eviction for containers
• hard eviction for containers
• eviction for images

To configure, follow the directions in the documentation

This rule pertains to the memory.available setting of the evictionHard section.

Two types of garbage collection are performed on an OpenShift Container Platform node:

• Container garbage collection: Removes terminated containers.
• Image garbage collection: Removes images not referenced by any running pods.

Container garbage collection can be performed using eviction thresholds. Image garbage collection relies on disk usage as reported by cAdvisor on the node to decide which images to remove from the node.

The OpenShift administrator can configure how OpenShift Container Platform performs garbage collection by creating a kubeletConfig object for each Machine Config Pool using any combination of the following:

• soft eviction for containers
• hard eviction for containers
• eviction for images

To configure, follow the directions in the documentation

This rule pertains to the nodefs.available setting of the evictionHard section.

Two types of garbage collection are performed on an OpenShift Container Platform node:

• Container garbage collection: Removes terminated containers.
• Image garbage collection: Removes images not referenced by any running pods.

Container garbage collection can be performed using eviction thresholds. Image garbage collection relies on disk usage as reported by cAdvisor on the node to decide which images to remove from the node.

The OpenShift administrator can configure how OpenShift Container Platform performs garbage collection by creating a kubeletConfig object for each Machine Config Pool using any combination of the following:

• soft eviction for containers
• hard eviction for containers
• eviction for images

To configure, follow the directions in the documentation

This rule pertains to the nodefs.inodesFree setting of the evictionHard section.

$ sudo chmod 0700 /var/log/kube-apiserver/

$ sudo chmod 0700 /var/log/oauth-apiserver/

$ sudo chmod 0700 /var/log/openshift-apiserver/

/var/log/kube-apiserver/

$ sudo chown root /var/log/kube-apiserver 

$ sudo chown root /var/log/kube-apiserver/* 

/var/log/oauth-apiserver/

$ sudo chown root /var/log/oauth-apiserver 

$ sudo chown root /var/log/oauth-apiserver/* 

/var/log/openshift-apiserver/

$ sudo chown root /var/log/openshift-apiserver 

$ sudo chown root /var/log/openshift-apiserver/* 

$ sudo chmod 0600 /var/log/kube-apiserver/.*

$ sudo chmod 0600 /var/log/oauth-apiserver/.*

$ sudo chmod 0600 /var/log/openshift-apiserver/.*

Partitioning Red Hat CoreOS is a Day 1 operation and cannot be changed afterwards. For documentation on how to add a MachineConfig manifest that specifies a separate /var/log/kube-apiserver partition, follow: https://docs.openshift.com/container-platform/latest/installing/installing_platform_agnostic/installing-platform-agnostic.html#installation-user-infra-machines-advanced_disk_installing-platform-agnostic

Note that the Red Hat OpenShift documentation often references a block device, such as /dev/vda. The name of the available block devices depends on the underlying infrastructure (bare metal vs cloud), and often the specific instance type. For example in AWS, some instance types have NVMe drives (/dev/nvme*), others use /dev/xvda*. You will need to look for relevant documentation for your infrastructure around this. In many cases, the simplest thing is to boot a single machine with an Ignition configuration that just gives you SSH access, and inspect the block devices via e.g. the lsblk command. For physical hardware, a good best practice is to reference devices via the /dev/disk/by-id/ or /dev/disk/by-path links.

Partitioning Red Hat CoreOS is a Day 1 operation and cannot be changed afterwards. For documentation on how to add a MachineConfig manifest that specifies a separate /var/log/oauth-apiserver partition, follow: https://docs.openshift.com/container-platform/latest/installing/installing_platform_agnostic/installing-platform-agnostic.html#installation-user-infra-machines-advanced_disk_installing-platform-agnostic

Note that the Red Hat OpenShift documentation often references a block device, such as /dev/vda. The name of the available block devices depends on the underlying infrastructure (bare metal vs cloud), and often the specific instance type. For example in AWS, some instance types have NVMe drives (/dev/nvme*), others use /dev/xvda*. You will need to look for relevant documentation for your infrastructure around this. In many cases, the simplest thing is to boot a single machine with an Ignition configuration that just gives you SSH access, and inspect the block devices via e.g. the lsblk command. For physical hardware, a good best practice is to reference devices via the /dev/disk/by-id/ or /dev/disk/by-path links.

Partitioning Red Hat CoreOS is a Day 1 operation and cannot be changed afterwards. For documentation on how to add a MachineConfig manifest that specifies a separate /var/log/openshift-apiserver partition, follow: https://docs.openshift.com/container-platform/latest/installing/installing_platform_agnostic/installing-platform-agnostic.html#installation-user-infra-machines-advanced_disk_installing-platform-agnostic

Note that the Red Hat OpenShift documentation often references a block device, such as /dev/vda. The name of the available block devices depends on the underlying infrastructure (bare metal vs cloud), and often the specific instance type. For example in AWS, some instance types have NVMe drives (/dev/nvme*), others use /dev/xvda*. You will need to look for relevant documentation for your infrastructure around this. In many cases, the simplest thing is to boot a single machine with an Ignition configuration that just gives you SSH access, and inspect the block devices via e.g. the lsblk command. For physical hardware, a good best practice is to reference devices via the /dev/disk/by-id/ or /dev/disk/by-path links.

$ sudo chgrp root /etc/cni/net.d/*

$ sudo chgrp root /etc/kubernetes/static-pod-resources/kube-controller-manager-pod-*/configmaps/controller-manager-kubeconfig/kubeconfig

$ sudo chgrp root /var/lib/etcd/member/

$ sudo chgrp root /var/lib/etcd/member/wal/*

$ sudo chgrp root /etc/kubernetes/manifests/etcd-pod.yaml

$ sudo chgrp root /etc/kubernetes/static-pod-resources/*/*/*/*.crt

$ sudo chgrp root /var/lib/cni/networks/openshift-sdn/.*

$ sudo chgrp root /etc/kubernetes/static-pod-resources/kube-apiserver-pod-*/kube-apiserver-pod.yaml

$ sudo chgrp root /etc/kubernetes/static-pod-resources/kube-controller-manager-pod-*/kube-controller-manager-pod.yaml

$ sudo chgrp root /etc/kubernetes/static-pod-resources/kube-scheduler-pod-*/kube-scheduler-pod.yaml

$ sudo chgrp root /etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/node-kubeconfigs/*.kubeconfig

$ sudo chgrp root /var/run/multus/cni/net.d/*

$ sudo chgrp root /etc/kubernetes/static-pod-resources/*/*/*/tls.crt

$ sudo chgrp root /etc/kubernetes/static-pod-resources/*/*/*/*.key

$ sudo chgrp root /var/run/openshift-sdn/cniserver/config.json

$ sudo chgrp root /run/ovn-kubernetes/cni/ovn-cni-server.sock

$ sudo chgrp root /var/lib/ovn/etc/*.db

/var/run/openvswitch/ovs-vswitchd.pid

openvswitch

hugetlbfs

/run/openvswitch/ovs-vswitchd.pid

openvswitch

hugetlbfs

/run/openvswitch/ovsdb-server.pid

openvswitch

hugetlbfs

$ sudo chgrp root /etc/kubernetes/static-pod-resources/kube-scheduler-pod-*/configmaps/scheduler-kubeconfig/kubeconfig

$ sudo chown root /etc/cni/net.d/* 

$ sudo chown root /etc/kubernetes/static-pod-resources/kube-controller-manager-pod-*/configmaps/controller-manager-kubeconfig/kubeconfig 

$ sudo chown root /var/lib/etcd/member/ 

$ sudo chown root /var/lib/etcd/member/wal/* 

$ sudo chown root /etc/kubernetes/manifests/etcd-pod.yaml 

$ sudo chown root /etc/kubernetes/static-pod-resources/*/*/*/*.crt 

$ sudo chown root /var/lib/cni/networks/openshift-sdn/.* 

$ sudo chown root /etc/kubernetes/static-pod-resources/kube-apiserver-pod-*/kube-apiserver-pod.yaml 

$ sudo chown root /etc/kubernetes/static-pod-resources/kube-controller-manager-pod-*/kube-controller-manager-pod.yaml 

$ sudo chown root /etc/kubernetes/static-pod-resources/kube-scheduler-pod-*/kube-scheduler-pod.yaml 

$ sudo chown root /etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/node-kubeconfigs/*.kubeconfig 

$ sudo chown root /var/run/multus/cni/net.d/* 

$ sudo chown root /etc/kubernetes/static-pod-resources/*/*/*/tls.crt 

$ sudo chown root /etc/kubernetes/static-pod-resources/*/*/*/*.key 

$ sudo chown root /var/run/openshift-sdn/cniserver/config.json 

$ sudo chown root /run/ovn-kubernetes/cni/ovn-cni-server.sock 

$ sudo chown root /var/lib/ovn/etc/*.db 

$ sudo chown openvswitch /etc/openvswitch/conf.db 

$ sudo chown openvswitch /etc/openvswitch/.conf.db.~lock~ 

$ sudo chown openvswitch /var/run/openvswitch/ovs-vswitchd.pid 

$ sudo chown openvswitch /etc/openvswitch/system-id.conf 

$ sudo chown openvswitch /run/openvswitch/ovs-vswitchd.pid 

$ sudo chown openvswitch /run/openvswitch/ovsdb-server.pid 

$ sudo chown root /etc/kubernetes/static-pod-resources/kube-scheduler-pod-*/configmaps/scheduler-kubeconfig/kubeconfig 

$ sudo chmod 0600 /etc/cni/net.d/*

$ sudo chmod 0600 /etc/kubernetes/static-pod-resources/kube-controller-manager-pod-*/configmaps/controller-manager-kubeconfig/kubeconfig

$ sudo chmod 0700 /var/lib/etcd

$ sudo chmod 0600 /var/lib/etcd/member/wal/*

$ sudo chmod 0600 /etc/kubernetes/manifests/etcd-pod.yaml

$ sudo chmod 0600 /etc/kubernetes/static-pod-resources/etcd-*/secrets/*/*.crt

$ sudo chmod 0644 /var/lib/cni/networks/openshift-sdn/*

$ sudo chmod 0600 /etc/kubernetes/static-pod-resources/kube-apiserver-pod-*/kube-apiserver-pod.yaml

$ sudo chmod 0600 /etc/kubernetes/static-pod-resources/kube-controller-manager-pod-*/kube-controller-manager-pod.yaml

$ sudo chmod 0600 /etc/kubernetes/static-pod-resources/kube-apiserver-certs/secrets/node-kubeconfigs/*.kubeconfig

$ sudo chmod 0644 /var/run/multus/cni/net.d/*

$ sudo chmod 0600 /etc/kubernetes/static-pod-resources/kube-*/secrets/*/tls.crt

$ sudo chmod 0600 /etc/kubernetes/static-pod-resources/*/*/*/*.key

$ sudo chmod 0600 /run/ovn-kubernetes/cni/ovn-cni-server.sock

$ sudo chmod 0640 /var/lib/ovn/etc/*.db

$ sudo chmod 0640 /etc/openvswitch/conf.db

$ sudo chmod 0600 /etc/openvswitch/.conf.db.~lock~

$ sudo chmod 0644 /var/run/openvswitch/ovs-vswitchd.pid

$ sudo chmod 0644 /etc/openvswitch/system-id.conf

$ sudo chmod 0644 /run/openvswitch/ovs-vswitchd.pid

$ sudo chmod 0644 /run/openvswitch/ovsdb-server.pid

$ sudo chmod 0644 /etc/kubernetes/static-pod-resources/kube-scheduler-pod-*/kube-scheduler-pod.yaml

$ sudo chmod 0600 /etc/kubernetes/static-pod-resources/kube-scheduler-pod-*/configmaps/scheduler-kubeconfig/kubeconfig

$ sudo chmod 0444 /var/run/openshift-sdn/cniserver/config.json

$ sudo chgrp root /etc/kubernetes/kubelet.conf

$ sudo chgrp root /etc/kubernetes/kubelet-ca.crt

$ sudo chgrp root /var/lib/kubelet/kubeconfig

$ sudo chgrp root /etc/systemd/system/kubelet.service

$ sudo chown root /var/lib/kubelet/config.json 

$ sudo chown root /etc/kubernetes/kubelet.conf 

$ sudo chown root /etc/kubernetes/kubelet-ca.crt 

$ sudo chown root /var/lib/kubelet/kubeconfig 

$ sudo chown root /etc/systemd/system/kubelet.service 

$ sudo chmod 0644 /etc/kubernetes/kubelet.conf

$ sudo chmod 0644 /etc/kubernetes/kubelet-ca.crt

$ sudo chmod 0600 /var/lib/kubelet/kubeconfig

$ sudo chmod 0644 /etc/systemd/system/kubelet.service