CIS Red Hat OpenShift Container Platform 4 Benchmark

$ oc create sa service_account_name
       

$ oc -n openshift-kube-apiserver get configmap config -o json | jq -r '.data."config.yaml"' | jq '.apiServerArguments."enable-admission-plugins"'

$ oc -n openshift-kube-apiserver get configmap config -o json | jq -r '.data."config.yaml"' | jq '.apiServerArguments."enable-admission-plugins"'

$ oc describe clusterrolebindings

$ oc get --as="system:anonymous" --raw='/readyz?verbose'

$ oc get --as="system:anonymous" projects

"apiServerArguments":{
  ...
  "audit-log-maxbackup": [10],
  ...

"apiServerArguments":{
  ...
  "audit-log-maxsize": ["100"],
  ...

"apiServerArguments":{
  ...
  "audit-log-path":"/var/log/kube-apiserver/audit.log",
  ...

"apiServerArguments":{
  ...
  "basic-auth-file":[
    "/path/to/any/file"
  ],
  ...

"apiServerArguments": {
  ...
    "client-ca-file": [
      "/etc/kubernetes/static-pod-certs/configmaps/client-ca/ca-bundle.crt"
    ],
  ...

When you enable etcd encryption, the following OpenShift API server and Kubernetes API server resources are encrypted:

• Secrets
• ConfigMaps
• Routes
• OAuth access tokens
• OAuth authorize tokens

When you enable etcd encryption, encryption keys are created. These keys are rotated on a weekly basis. You must have these keys in order to restore from an etcd backup.

To ensure the correct cipher, set the encryption type to aescbc or aesgcm in the apiserver object which configures the API server itself.

spec:
  encryption:
    type: aescbc

For more information, follow the relevant documentation.

"apiServerArguments": {
  ...
    "etcd-cafile": [
        "/etc/kubernetes/static-pod-resources/configmaps/etcd-serving-ca/ca-bundle.crt"
    ],
  ...

...
"etcd-certfile": [
    "/etc/kubernetes/static-pod-resources/secrets/etcd-client/tls.crt"
],
...

...
"etcd-keyfile": [
    "/etc/kubernetes/static-pod-resources/secrets/etcd-client/tls.key"
],
...

"apiServerArguments":{
  ...
  "insecure-bind-address":[
    "127.0.0.1"
  ],
  ...

"apiServerArguments":{
  ...
  "kubelet-certificate-authority":"/etc/kubernetes/static-pod-resources/configmaps/kubelet-serving-ca/ca-bundle.crt",
  ...

"apiServerArguments":{
...
  "kubelet-client-certificate":"/etc/kubernetes/static-pod-resources/secrets/kubelet-client/tls.crt",
...
}

"apiServerArguments":{
...
  "kubelet-client-certificate":"/etc/kubernetes/static-pod-resources/secrets/kubelet-client/tls.crt",
...
}

Note that this particular rule is only valid for OCP releases up to and
including 4.8

"apiServerArguments":{
...
  "kubelet-client-key":"/etc/kubernetes/static-pod-resources/secrets/kubelet-client/tls.key",
...
}

"apiServerArguments":{
...
  "kubelet-client-key":"/etc/kubernetes/static-pod-resources/secrets/kubelet-client/tls.key",
...
}

Note that this particular rule is only valid for OCP releases up to and
including 4.8

"apiServerArguments":{
  ...
  "min-request-timeout":[
    
  ],
  ...

...
"serviceAccountPublicKeyFiles":[
  "/etc/kubernetes/static-pod-resources/configmaps/sa-token-signing-certs"
],
...

"apiServerArguments":{
...
"tls-cert-file": [
  "/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.crt"
],
...
}

"servingInfo":{
  ...
  "cipherSuites": [
    "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384",
    "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
    "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256",
    "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256",
  ],
  ...

"apiServerArguments":{
...
"tls-private-key-file": [
  "/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.key"
],
...
}

"apiServerArguments":{
  ...
  "token-auth-file":[
    "/path/to/any/file"
  ],
  ...

ClusterLogForwarders

For users to interact with OpenShift Container Platform, they must first authenticate to the cluster. The authentication layer identifies the user associated with requests to the OpenShift Container Platform API. The authorization layer then uses information about the requesting user to determine if the request is allowed. Understanding authentication | Authentication | OpenShift Container Platform

The OpenShift Container Platform includes a built-in OAuth server for token-based authentication. Developers and administrators obtain OAuth access tokens to authenticate themselves to the API. It is recommended for an administrator to configure OAuth to specify an identity provider after the cluster is installed. User access to the cluster is managed through the identity provider. Understanding identity provider configuration | Authentication | OpenShift Container Platform

OpenShift includes built-in role based access control (RBAC) to determine whether a user is allowed to perform a given action within the cluster. Roles can have cluster scope or local (i.e. project) scope. Using RBAC to define and apply permissions | Authentication | OpenShift Container Platform

"extendedArguments": {
...
  "port": ["0"],
...

"extendedArguments": {
...
...

"extendedArguments": {
...
  "secure-port": ["10257"],
...

"extendedArguments": {
...
  "root-ca-file": [
    "/etc/kubernetes/static-pod-resources/configmaps/serviceaccount-ca/ca-bundle.crt"
  ],
...

"extendedArguments": {
...
  "service-account-private-key-file": [
    "/etc/kubernetes/static-pod-resources/secrets/service-account-private-key/service-account.key"
  ],
...

"extendedArguments": {
...
  "use-service-account-credentials": [
    "true"
  ],
...

$ oc get cm/etcd-pod -n openshift-etcd -o yaml

--cert-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-[a-z]+/etcd-serving-NODE_NAME.crt

[a-z]+

oc get -nopenshift-etcd cm etcd-pod -oyaml | grep "\-\-client-cert-auth="

oc get -nopenshift-etcd cm etcd-pod -oyaml | grep "\-\-key-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-[a-z]+/etcd-serving-NODE_NAME.key"

[a-z]+

$ oc get cm/etcd-pod -n openshift-etcd -o yaml

--peer-cert-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-[a-z]+/etcd-peer-NODE_NAME.crt

[a-z]+

oc get -nopenshift-etcd cm etcd-pod -oyaml | grep "\-\-peer-client-cert-auth="

oc get -nopenshift-etcd cm etcd-pod -oyaml | grep "\-\-peer-key-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-[a-z]+/etcd-peer-NODE_NAME.key"

[a-z]+

tlsCertFile: /path/to/TLS/cert.key

tlsPrivateKeyFile: /path/to/TLS/private.key

"apiServerArguments":{
  ...
  "kubelet-read-only-port":[
    "0"
  ],
  ...

OpenShift can audit the details of requests made to the API server through the standard Kubernetes audit capabilities.

In OpenShift, auditing of the API Server is on by default. Audit provides a security-relevant chronological set of records documenting the sequence of activities that have affected system by individual users, administrators, or other components of the system. Audit works at the API server level, logging all requests coming to the server. Each audit log contains two entries:

The request line containing:

• A Unique ID allowing to match the response line (see #2)
• The source IP of the request
• The HTTP method being invoked
• The original user invoking the operation
• The impersonated user for the operation (self meaning himself)
• The impersonated group for the operation (lookup meaning user's group)
• The namespace of the request or none
• The URI as requested

The response line containing:

• The aforementioned unique ID
• The response code

For more information on how to configure the audit profile, please visit the documentation

"apiServerArguments":{
  ...
  "audit-log-maxbackup": [10],
  ...

"apiServerArguments":{
  ...
  "audit-log-maxsize": ["100"],
  ...

"apiServerArguments":{
  ...
  "audit-log-path":"/var/log/openshift-apiserver/audit.log",
  ...

oc patch

/tmp/allowed-registries-patch.yaml

spec:
  registrySources:
    allowedRegistries:
    - my-trusted-registry.internal.example.com

oc patch image.config.openshift.io cluster --patch="$(cat /tmp/allowed-registries-patch.yaml)" --type=merge

oc patch

/tmp/allowed-import-registries-patch.yaml

spec:
  allowedRegistriesForImport:
  - domainName: my-trusted-registry.internal.example.com
    insecure: false

oc patch image.config.openshift.io cluster --patch="$(cat /tmp/allowed-import-registries-patch.yaml)" --type=merge

oc get image.config.openshift.io/cluster -o json | jq '.spec | (.allowedRegistriesForImport[])? | select(.insecure==true)'

oc edit image.config.openshift.io/cluster

oc patch image.config.openshift.io cluster --type=json -p "[{'op': 'remove', 'path': '/spec/registrySources/insecureRegistries'}]"

Containers should not enable more capabilites than needed as this opens the door for malicious use. To enable only the required capabilities, the appropriate Security Context Constraints (SCCs) should set capabilities as a list in allowedCapabilities.

In case an SCC outside the default allow list in the variable var-sccs-with-allowed-capabilities-regex is being flagged, create a TailoredProfile and add the additional SCC to the regular expression in the variable var-sccs-with-allowed-capabilities-regex. An example allowing an SCC named additional follows:

apiVersion: compliance.openshift.io/v1alpha1
kind: TailoredProfile
metadata:
  name: cis-additional-scc
spec:
  description: Allows an additional scc
  setValues:
  - name: ocp4-var-sccs-with-allowed-capabilities-regex
    rationale: Allow our own custom SCC
    value: ^privileged$|^hostnetwork-v2$|^restricted-v2$|^nonroot-v2$|^additional$
  extends: ocp4-cis
  title: Modified CIS allowing one more SCC

Finally, reference this TailoredProfile in a ScanSettingBinding For more information on Tailoring the Compliance Operator, please consult the OpenShift documentation: https://docs.openshift.com/container-platform/latest/security/compliance_operator/co-scans/compliance-operator-tailor.html

 oc get -nopenshift-sdn ds sdn -ojson | jq -r '.spec.template.spec.containers[] | select(.name == "sdn")'

 oc get -nopenshift-sdn ds sdn -ojson | jq -r '.spec.template.spec.containers[] | select(.name == "sdn")'

{
"configMap": {
  "defaultMode": 420,
  "name": "sdn-config"
  },
"name": "config"
}