PCI-DSS v3.2.1 Control Baseline for Red Hat OpenShift Container Platform 4

$ oc create sa service_account_name

$ oc -n openshift-kube-apiserver get configmap config -o json | jq -r '.data."config.yaml"' | jq '.apiServerArguments."enable-admission-plugins"'

$ oc -n openshift-kube-apiserver get configmap config -o json | jq -r '.data."config.yaml"' | jq '.apiServerArguments."enable-admission-plugins"'

$ oc describe clusterrolebindings

$ oc get --as="system:anonymous" --raw='/readyz?verbose'

$ oc get --as="system:anonymous" projects

"apiServerArguments":{
  ...
  "audit-log-maxbackup": [10],
  ...

"apiServerArguments":{
  ...
  "audit-log-maxsize": ["100"],
  ...

"apiServerArguments":{
  ...
  "audit-log-path":"/var/log/kube-apiserver/audit.log",
  ...

"apiServerArguments":{
  ...
  "basic-auth-file":[
    "/path/to/any/file"
  ],
  ...

"apiServerArguments": {
  ...
    "client-ca-file": [
      "/etc/kubernetes/static-pod-certs/configmaps/client-ca/ca-bundle.crt"
    ],
  ...

When you enable etcd encryption, the following OpenShift API server and Kubernetes API server resources are encrypted:

• Secrets
• ConfigMaps
• Routes
• OAuth access tokens
• OAuth authorize tokens

When you enable etcd encryption, encryption keys are created. These keys are rotated on a weekly basis. You must have these keys in order to restore from an etcd backup.

To ensure the correct cipher, set the encryption type aescbc in the apiserver object which configures the API server itself.

spec:
  encryption:
    type: aescbc

For more information, follow the relevant documentation.

"apiServerArguments": {
  ...
    "etcd-cafile": [
        "/etc/kubernetes/static-pod-resources/configmaps/etcd-serving-ca/ca-bundle.crt"
    ],
  ...

...
"etcd-certfile": [
    "/etc/kubernetes/static-pod-resources/secrets/etcd-client/tls.crt"
],
...

...
"etcd-keyfile": [
    "/etc/kubernetes/static-pod-resources/secrets/etcd-client/tls.key"
],
...

"apiServerArguments":{
  ...
  "insecure-bind-address":[
    "127.0.0.1"
  ],
  ...

"apiServerArguments":{
  ...
  "insecure-port":[
    "1234"
  ],
  ...

"apiServerArguments":{
  ...
  "kubelet-certificate-authority":"/etc/kubernetes/static-pod-resources/configmaps/kubelet-serving-ca/ca-bundle.crt",
  ...

"apiServerArguments":{
...
  "kubelet-client-certificate":"/etc/kubernetes/static-pod-resources/secrets/kubelet-client/tls.crt",
...
}

"apiServerArguments":{
...
  "kubelet-client-certificate":"/etc/kubernetes/static-pod-resources/secrets/kubelet-client/tls.crt",
...
}

Note that this particular rule is only valid for OCP releases up to and
including 4.8

"apiServerArguments":{
...
  "kubelet-client-key":"/etc/kubernetes/static-pod-resources/secrets/kubelet-client/tls.key",
...
}

"apiServerArguments":{
...
  "kubelet-client-key":"/etc/kubernetes/static-pod-resources/secrets/kubelet-client/tls.key",
...
}

Note that this particular rule is only valid for OCP releases up to and
including 4.8

"apiServerArguments":{
  ...
  "min-request-timeout":[
    
  ],
  ...

...
"serviceAccountPublicKeyFiles":[
  "/etc/kubernetes/static-pod-resources/configmaps/sa-token-signing-certs"
],
...

"apiServerArguments":{
...
"tls-cert-file": [
  "/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.crt"
],
...
}

"servingInfo":{
  ...
  "cipherSuites": [
    "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384",
    "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
    "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256",
    "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256",
  ],
  ...

"apiServerArguments":{
...
"tls-private-key-file": [
  "/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.key"
],
...
}

"apiServerArguments":{
  ...
  "token-auth-file":[
    "/path/to/any/file"
  ],
  ...

ClusterLogForwarders

For users to interact with OpenShift Container Platform, they must first authenticate to the cluster. The authentication layer identifies the user associated with requests to the OpenShift Container Platform API. The authorization layer then uses information about the requesting user to determine if the request is allowed. Understanding authentication | Authentication | OpenShift Container Platform

The OpenShift Container Platform includes a built-in OAuth server for token-based authentication. Developers and administrators obtain OAuth access tokens to authenticate themselves to the API. It is recommended for an administrator to configure OAuth to specify an identity provider after the cluster is installed. User access to the cluster is managed through the identity provider. Understanding identity provider configuration | Authentication | OpenShift Container Platform

OpenShift includes built-in role based access control (RBAC) to determine whether a user is allowed to perform a given action within the cluster. Roles can have cluster scope or local (i.e. project) scope. Using RBAC to define and apply permissions | Authentication | OpenShift Container Platform

For users to interact with OpenShift Container Platform, they must first authenticate to the cluster. The authentication layer identifies the user associated with requests to the OpenShift Container Platform API. The authorization layer then uses information about the requesting user to determine if the request is allowed. Understanding authentication | Authentication | OpenShift Container Platform

The OpenShift Container Platform includes a built-in OAuth server for token-based authentication. Developers and administrators obtain OAuth access tokens to authenticate themselves to the API. It is recommended for an administrator to configure OAuth to specify an identity provider after the cluster is installed. User access to the cluster is managed through the identity provider. Understanding identity provider configuration | Authentication | OpenShift Container Platform

If the identity provider is LDAP, setting the insecure flag to true would mean that passwords, such as the one used to authenticate the OAuth proxy to the LDAP server would be transmitted in the clear, potentially allowing an attacker to read the password if they captured the network traffic.

"extendedArguments": {
...
  "port": ["0"],
...

"extendedArguments": {
...
...

"extendedArguments": {
...
  "secure-port": ["10257"],
...

"extendedArguments": {
...
  "root-ca-file": [
    "/etc/kubernetes/static-pod-resources/configmaps/serviceaccount-ca/ca-bundle.crt"
  ],
...

"extendedArguments": {
...
  "service-account-private-key-file": [
    "/etc/kubernetes/static-pod-resources/secrets/service-account-private-key/service-account.key"
  ],
...

"extendedArguments": {
...
  "use-service-account-credentials": [
    "true"
  ],
...

$ oc get cm/etcd-pod -n openshift-etcd -o yaml

--cert-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-[a-z]+/etcd-serving-NODE_NAME.crt

[a-z]+

oc get -nopenshift-etcd cm etcd-pod -oyaml | grep "\-\-client-cert-auth="

oc get -nopenshift-etcd cm etcd-pod -oyaml | grep "\-\-key-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-[a-z]+/etcd-serving-NODE_NAME.key"

[a-z]+

$ oc get cm/etcd-pod -n openshift-etcd -o yaml

--peer-cert-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-[a-z]+/etcd-peer-NODE_NAME.crt

[a-z]+

oc get -nopenshift-etcd cm etcd-pod -oyaml | grep "\-\-peer-client-cert-auth="

oc get -nopenshift-etcd cm etcd-pod -oyaml | grep "\-\-peer-key-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-[a-z]+/etcd-peer-NODE_NAME.key"

[a-z]+

tlsCertFile: /path/to/TLS/cert.key

tlsCertFile: /path/to/TLS/cert.key

Note that this particular rule is only valid for OCP releases up to and
including 4.8

tlsPrivateKeyFile: /path/to/TLS/private.key

tlsPrivateKeyFile: /path/to/TLS/private.key

Note that this particular rule is only valid for OCP releases up to and
including 4.8

"apiServerArguments":{
  ...
  "kubelet-read-only-port":[
    "0"
  ],
  ...

OpenShift can audit the details of requests made to the API server through the standard Kubernetes audit capabilities.

In OpenShift, auditing of the API Server is on by default. Audit provides a security-relevant chronological set of records documenting the sequence of activities that have affected system by individual users, administrators, or other components of the system. Audit works at the API server level, logging all requests coming to the server. Each audit log contains two entries:

The request line containing:

• A Unique ID allowing to match the response line (see #2)
• The source IP of the request
• The HTTP method being invoked
• The original user invoking the operation
• The impersonated user for the operation (self meaning himself)
• The impersonated group for the operation (lookup meaning user's group)
• The namespace of the request or none
• The URI as requested

The response line containing:

• The aforementioned unique ID
• The response code

For more information on how to configure the audit profile, please visit the documentation

"apiServerArguments":{
  ...
  "audit-log-maxbackup": [10],
  ...

"apiServerArguments":{
  ...
  "audit-log-maxsize": ["100"],
  ...

"apiServerArguments":{
  ...
  "audit-log-path":"/var/log/openshift-apiserver/audit.log",
  ...

RBAC is a critical feature in terms of security for Kubernetes and OpenShift. It enables administrators to segment the privileges granted to a service account, and thus allows us to limit the access to resources that they get. By defining cluster roles appropriately one is able to codify organizational policy. [1]

[1] https://docs.openshift.com/container-platform/latest/authentication/using-rbac.html

RBAC is a critical feature in terms of security for Kubernetes and OpenShift. It enables administrators to segment the privileges granted to a service account, and thus allows us to limit the access to resources that they get. By defining roles appropriately one is able to codify organizational policy. [1]

[1] https://docs.openshift.com/container-platform/latest/authentication/using-rbac.html

oc patch

/tmp/allowed-registries-patch.yaml

spec:
  registrySources:
    allowedRegistries:
    - my-trusted-registry.internal.example.com

oc patch image.config.openshift.io cluster --patch="$(cat /tmp/allowed-registries-patch.yaml)" --type=merge

oc patch

/tmp/allowed-import-registries-patch.yaml

spec:
  allowedRegistriesForImport:
  - domainName: my-trusted-registry.internal.example.com
    insecure: false

oc patch image.config.openshift.io cluster --patch="$(cat /tmp/allowed-import-registries-patch.yaml)" --type=merge

Containers should not enable more capabilites than needed as this opens the door for malicious use. To enable only the required capabilities, the appropriate Security Context Constraints (SCCs) should set capabilities as a list in allowedCapabilities.

In case an SCC outside the default allow list in the variable var-sccs-with-allowed-capabilities-regex is being flagged, create a TailoredProfile and add the additional SCC to the regular expression in the variable var-sccs-with-allowed-capabilities-regex. An example allowing an SCC named additional follows:

apiVersion: compliance.openshift.io/v1alpha1
kind: TailoredProfile
metadata:
  name: cis-additional-scc
spec:
  description: Allows an additional scc
  setValues:
  - name: upstream-ocp4-var-sccs-with-allowed-capabilities-regex
    rationale: Allow our own custom SCC
    value: ^privileged$|^hostnetwork-v2$|^restricted-v2$|^nonroot-v2$|^additional$
  extends: upstream-ocp4-cis
  title: Modified CIS allowing one more SCC

Finally, reference this TailoredProfile in a ScanSettingBinding For more information on Tailoring the Compliance Operator, please consult the OpenShift documentation: https://docs.openshift.com/container-platform/4.12/security/compliance_operator/compliance-operator-tailor.html

 oc get -nopenshift-sdn ds sdn -ojson | jq -r '.spec.template.spec.containers[] | select(.name == "sdn")'

 oc get -nopenshift-sdn ds sdn -ojson | jq -r '.spec.template.spec.containers[] | select(.name == "sdn")'

{
"configMap": {
  "defaultMode": 420,
  "name": "sdn-config"
  },
"name": "config"
}