#OIDC Configuration for Shoot clusters

#OpenID Connect

For older Shoot clusters (Kubernetes version < 1.32), the kube-apiserver of Shoot clusters can be provided with OpenID Connect configuration by providing specific label within the Shoot spec during the Shoot creation. The label in the Shoot manifest must match name of the seed cluster on which your Shoot should reside:

kind: Shoot
apiVersion: core.gardener.cloud/v1beta1
metadata:
  name: <name of the shoot>
  namespace: <namespace name>
  labels: settings.gardener.osc.t-systems.com/realm=<seed name>
spec:
  ...

If the label was not specified in Shoot's YAML manifest during creation time, it will be added automatically by webhook.

Starting with Kubernetes v1.30, Gardener (and Kubernetes) encourages configuration from OIDC-based config to the Structured Authentication approach, where authentication is configured via a ConfigMap in the project namespace and referenced from the Shoot spec via spec.kubernetes.kubeAPIServer.structuredAuthentication.configMapName. For further information please visit Gardener Documentation

#OIDC kubeconfig

OIDC kubeconfig is stored in shoot-name.kubeconfig-oidc secret in the Shoot's Project namespace in the Garden cluster.

The recommended way to use OIDC kubeconfig is the kubectl plugin called kubectl oidc-login.

#Static Token kubeconfig

This kubeconfig contains a static token and provides cluster-admin privileges. It is stored in the shoot-name.kubeconfig secret in the Shoot's Project namespace in the Garden cluster.

Static token kubeconfig is not available for Shoot clusters using Kubernetes version >= 1.27.

apiVersion: core.gardener.cloud/v1beta1
kind: Shoot
...
spec:
  kubernetes:
    enableStaticTokenKubeconfig: true
...

It is not the recommended method to access the shoot cluster with the static kubeconfig since there are security flaws associated with it:

• The static token in the kubeconfig doesn't have any expiration date.
• The static token doesn't have any user identity associated with it. The user associated with that token will always be system:cluster-admin, irrespective of the person accessing the cluster. Hence, it is impossible to audit the events in cluster.

When enableStaticTokenKubeconfig field is not explicitly set in the Shoot spec:

• for Shoot clusters using Kubernetes version < 1.26 the field is defaulted to true.

• for Shoot clusters using Kubernetes version >= 1.26 the field is defaulted to false.

• for Shoot clusters using Kubernetes version >= 1.27 the field is locked to false.

#Example of OIDC configuration in Shoot YAML manifest

spec:
  kubernetes:
    kubeAPIServer:
      enableBasicAuthentication: false
      oidcConfig:
        caBundle: |
          -----BEGIN CERTIFICATE-----
          # This CA certificate resides on the seed cluster
          -----END CERTIFICATE-----
        clientAuthentication:
          extraConfig:
            extra-scopes: >-
              email,offline_access,profile,groups,audience:server:client_id:kubernetes
        clientID: kubernetes
        groupsClaim: groups
        groupsPrefix: 'oidc:'
        issuerURL: https://auth.apps.fsd1-2.ffm.osc.live/dex 
        signingAlgs:
          - RS256
        usernameClaim: preferred_username
        usernamePrefix: 'oidc:'

#How to create custom kubeconfig with limited access to cluster

1. Create custom sa (ServiceAccount)

   kubectl create sa my-service-account -n my-namespace

2. Verify that sa was created successfully

   kubectl get sa -n my-namespace

   example output:

   NAME                 SECRETS   AGE
   default              0         128m
   my-service-account   0         127m

3. Create a token for custom ServiceAccount

TOKEN=$(kubectl create token my-service-account --duration=8760h -n my-namespace)

4. Create a custom role in namespace my-namespace which will allow the ServiceAccount to only access selected resources(e.g., pods,pods logs, shoots, ...) from namespace my-namespace

Example for pods and pods logs:

```yaml
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  namespace: my-namespace
  name: my-role
rules:
- apiGroups: [""]
  resources: ["pods", "pods/log"]
  verbs: ["get", "list"]
```

5. Create RoleBinding to bind created role my-role with ServiceAccount my-service-account

```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: pod-list-sa-binding
  namespace: my-namespace
subjects:
  - kind: ServiceAccount
    name: my-service-account
    namespace: my-namespace
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: my-role
```

6. Generate custom kubeconfig

```bash
kubectl config view --raw --template='
apiVersion: v1
kind: Config
clusters:
- name: my-cluster
  cluster:
    certificate-authority-data: {{ index .clusters 0 "cluster" "certificate-authority-data" }}
    server: {{ index .clusters 0 "cluster" "server" }}
contexts:
- name: my-context
  context:
    cluster: my-cluster
    namespace: "my-namespace"
    user: "my-service-account"
current-context: my-context
users:
- name: "my-service-account"
  user:
    token: "'${TOKEN}'"
' > my-sa-kubeconfig.yaml
```

Custom kubeconfig will be saved into file my-sa-kubeconfig.yaml.

7. Test of newly created kubeconfig

   kubectl get po -n my-namespace --kubeconfig my-sa-kubeconfig.yaml
   NAME            READY   STATUS    RESTARTS   AGE
   hello-busybox   1/1     Running   0          9m47s

Conclusion: As we can see from this short demo, we can define a limited access to a cluster, based on security policy defined within the customer organization.

For more details about RBAC please refer to the official Kubernetes documentation