Authentication with the API server

Kuma exposes API server on ports 5681 and 5682 (protected by TLS).

An authenticated user can be authorized to execute administrative actions such as

  • Managing administrative resources like Kuma Secrets on Universal
  • Generating user token, data plane proxy token, zone ingress token, zone token

User token

A user token is a signed JWT token that contains

  • The name of the user
  • The list of groups that a user belongs to
  • Expiration date of the token

Groups

A user can be a part of many groups. Kuma adds two groups to a user automatically:

  • authenticated users are a part of mesh-system:authenticated.
  • unauthenticated users are part of mesh-system:unauthenticated.

Admin user token

Kuma creates an admin user token on the first start of the control plane. The admin user token is a user token issued for user mesh-system:admin that belongs to mesh-system:admin group. This group is authorized by default to execute all administrative operations.

  1. Access admin user token

    Use kubectl to extract the admin token

    1. kubectl get secret admin-user-token -n {{site.mesh_namespace}} --template={{.data.value}} | base64 -d

  2. Expose Kuma CP to be accessible from your machine

    To access Kuma CP via kumactl, you need to expose Kuma CP outside of a cluster in one of the following ways:

    • Port-forward port 5681
    • Expose port 5681 and protect it by TLS or just expose 5682 with builtin TLS of kuma-control-plane service via a load balancer.
    • Expose port 5681 of kuma-control-plane via Ingress (for example Kong Ingress Controller) and protect it with TLS

  3. Configure kumactl with admin user token

    1. kumactl config control-planes add \
    2.   --name my-control-plane \
    3.   --address https://<CONTROL_PLANE_ADDRESS>:5682 \
    4.   --auth-type=tokens \
    5.   --auth-conf token=<GENERATED_TOKEN> \
    6.   --ca-cert-file=/path/to/ca.crt

    If you are using 5681 port, change the schema to http://.

    If you want to skip CP verification, use --skip-verify instead of --ca-cert-file.

  4. Access admin user token

    Execute the following command on the machine where you deployed the control plane.

    1. curl http://localhost:5681/global-secrets/admin-user-token | jq -r .data | base64 -d

  5. Configure kumactl with admin user token

    1. kumactl config control-planes add \
    2.   --name my-control-plane \
    3.   --address https://<CONTROL_PLANE_ADDRESS>:5682 \
    4.   --auth-type=tokens \
    5.   --auth-conf token=<GENERATED_TOKEN> \
    6.   --ca-cert-file=/path/to/ca.crt

    If you are using 5681 port, change the schema to http://.

    If you want to skip CP verification, use --skip-verify instead of --ca-cert-file.

  6. Disable localhost is admin (optional)

    By default, all requests originated from localhost are authenticated as an mesh-system:admin user. After you retrieve and store the admin token, configure a control plane with KUMA_API_SERVER_AUTHN_LOCALHOST_IS_ADMIN set to false.

Generate user tokens

You can generate user tokens only when you provide the credentials of a user authorized to generate user tokens . kumactl configured with admin user token extracted in the preceding section is authorized to do it.

  1. kumactl generate user-token \
  2.   --name john \
  3.   --group team-a \
  4.   --valid-for 24h

or you can use API

  1. curl localhost:5681/tokens/user \
  2.   -H'authentication: Bearer eyJhbGc...' \
  3.   -H'content-type: application/json' \
  4.   --data '{"name": "john","groups": ["team-a"], "validFor": "24h"}'

Explore an example token

You can decode the tokens to validate the signature or explore details.

For example, run:

  1. kumactl generate user-token \
  2.   --name john \
  3.   --group team-a \
  4.   --valid-for 24h

which returns:

  1. eyJhbGciOiJSUzI1NiIsImtpZCI6IjEiLCJ0eXAiOiJKV1QifQ.eyJOYW1lIjoiam9obiIsIkdyb3VwcyI6WyJ0ZWFtLWEiXSwiZXhwIjoxNjM2ODExNjc0LCJuYmYiOjE2MzY3MjQ5NzQsImlhdCI6MTYzNjcyNTI3NCwianRpIjoiYmYzZDBiMmUtZDg0MC00Y2I2LWJmN2MtYjkwZjU0MzkxNDY4In0.XsaPcQ5wVzRLs4o1FWywf6kw4r2ceyLGxYO8EbyA0fAxU6BPPRsW71ueD8ZlS4JlD4UrVtQQ7LG-z_nIxlDRAYhx4mmHnSjtqWZIsVS13QRrm41zccZ0SKHYxGvWMW4IkGwUbA0UZOJGno8vbpI6jTGfY9bmof5FpJJAj_sf99jCaI1H_n3n5UxtwKVN7dXXD82r6axj700jgQD-2O8gnejzlTjZkBpPF_lGnlBbd39S34VNwT0UlvRJLmCRdfh5EL24dFt0tyzQqDG2gE1RuGvTV9LOT77ZsjfMP9CITICivF6Z7uqvlOYal10jd5gN0A6w6KSI8CCaDLmVgUHvAw

Paste the token into the UI at jwt.io, or use jwt-cli tool

  1. kumactl generate user-token --name=john --group=team-a --valid-for=24h | jwt
  3. To verify on jwt.io:
  5. https://jwt.io/#id_token=eyJhbGciOiJSUzI1NiIsImtpZCI6IjEiLCJ0eXAiOiJKV1QifQ.eyJOYW1lIjoiam9obiIsIkdyb3VwcyI6WyJ0ZWFtLWEiXSwiZXhwIjoxNjM2ODExNjc0LCJuYmYiOjE2MzY3MjQ5NzQsImlhdCI6MTYzNjcyNTI3NCwianRpIjoiYmYzZDBiMmUtZDg0MC00Y2I2LWJmN2MtYjkwZjU0MzkxNDY4In0.XsaPcQ5wVzRLs4o1FWywf6kw4r2ceyLGxYO8EbyA0fAxU6BPPRsW71ueD8ZlS4JlD4UrVtQQ7LG-z_nIxlDRAYhx4mmHnSjtqWZIsVS13QRrm41zccZ0SKHYxGvWMW4IkGwUbA0UZOJGno8vbpI6jTGfY9bmof5FpJJAj_sf99jCaI1H_n3n5UxtwKVN7dXXD82r6axj700jgQD-2O8gnejzlTjZkBpPF_lGnlBbd39S34VNwT0UlvRJLmCRdfh5EL24dFt0tyzQqDG2gE1RuGvTV9LOT77ZsjfMP9CITICivF6Z7uqvlOYal10jd5gN0A6w6KSI8CCaDLmVgUHvAw
  7. ✻ Header
  8. {
  9.   "alg": "RS256",
  10.   "kid": "1",
  11.   "typ": "JWT"
  12. }
  14. ✻ Payload
  15. {
  16.   "Name": "john",
  17.   "Groups": [
  18.     "team-a"
  19.   ],
  20.   "exp": 1636811674,
  21.   "nbf": 1636724974,
  22.   "iat": 1636725274,
  23.   "jti": "bf3d0b2e-d840-4cb6-bf7c-b90f54391468"
  24. }
  25.    Issued At: 1636725274 11/12/2021, 2:54:34 PM
  26.    Not Before: 1636724974 11/12/2021, 2:49:34 PM
  27.    Expiration Time: 1636811674 11/13/2021, 2:54:34 PM
  29. ✻ Signature XsaPcQ5wVzRLs4o1FWywf6kw4r2ceyLGxYO8EbyA0fAxU6BPPRsW71ueD8ZlS4JlD4UrVtQQ7LG-z_nIxlDRAYhx4mmHnSjtqWZIsVS13QRrm41zccZ0SKHYxGvWMW4IkGwUbA0UZOJGno8vbpI6jTGfY9bmof5FpJJAj_sf99jCaI1H_n3n5UxtwKVN7dXXD82r6axj700jgQD-2O8gnejzlTjZkBpPF_lGnlBbd39S34VNwT0UlvRJLmCRdfh5EL24dFt0tyzQqDG2gE1RuGvTV9LOT77ZsjfMP9CITICivF6Z7uqvlOYal10jd5gN0A6w6KSI8CCaDLmVgUHvAw

Token revocation

Kuma doesn’t keep the list of issued tokens. To invalidate the token, you can add it to a revocation list. Every user token has its own ID. As you saw in the previous section, it’s available in payload under jti key. To revoke tokens, specify list of revoked IDs separated by , and store it as GlobalSecret named user-token-revocations

  1. REVOCATIONS=$(echo '0e120ec9-6b42-495d-9758-07b59fe86fb9' | base64) && echo "apiVersion: v1
  2. kind: Secret
  3. metadata:
  4.   name: user-token-revocations
  5.   namespace: kuma-system 
  6. data:
  7.   value: $REVOCATIONS
  8. type: system.kuma.io/global-secret" | kubectl apply -f -
  1. echo "
  2. type: GlobalSecret
  3. name: user-token-revocations
  4. data: " | kumactl apply --var revocations=$(echo '0e120ec9-6b42-495d-9758-07b59fe86fb9' | base64) -f -

Signing key

A user token is signed by a signing key that’s autogenerated on the first start of the control plane. The signing key is a 2048-bit RSA key stored as a GlobalSecret with a name that looks like user-token-signing-key-{serialNumber}.

Signing key rotation

If the signing key is compromised, you must rotate it including all the tokens that were signed by it.

  1. Generate a new signing key

    Make sure to generate the new signing key with a serial number greater than the serial number of the current signing key.

    Check what’s the current highest serial number.

    1.    kubectl get secrets -n kuma-system --field-selector='type=system.kuma.io/global-secret'
    2.    NAME                          TYPE                           DATA   AGE
    3.    user-token-signing-key-1   system.kuma.io/global-secret   1      25m

    In this case, the highest serial number is 1. Generate a new signing key with a serial number of 2

    1.    TOKEN="$(kumactl generate signing-key)" && echo "
    2.    apiVersion: v1
    3.    data:
    4.      value: $TOKEN
    5.    kind: Secret
    6.    metadata:
    7.      name: user-token-signing-key-2
    8.      namespace: kuma-system
    9.    type: system.kuma.io/global-secret
    10.    " | kubectl apply -f -

    Check what’s the current highest serial number.

    1.    kumactl get global-secrets
    2.    NAME                             AGE
    3.    user-token-signing-key-1   36m

    In this case, the highest serial number is 1. Generate a new signing key with a serial number of 2

    1.    echo "
    2.    type: GlobalSecret
    3.    name: user-token-signing-key-2
    4.    data: " | kumactl apply --var key=$(kumactl generate signing-key) -f -

  2. Regenerate user tokens

    Create new user tokens. Tokens are always signed by the signing key with the highest serial number. Starting from now, tokens signed by either new or old signing key are valid.

  3. Remove the old signing key

    1.    kubectl delete secret user-token-signing-key-1 -n kuma-system
    1.    kumactl delete global-secret user-token-signing-key-1

    All new requests to the control plane now require tokens signed with the new signing key.

Disabling bootstrap of the admin user token

You can remove the default admin user token from the storage and prevent it from being recreated. Keep in mind that even if you remove the admin user token, the signing key is still present. A malicious actor that acquires the signing key, can generate an admin token.

  1. Delete admin-user-token Secret

    1. kubectl delete secret admin-user-token -n kuma-namespace

  2. Disable bootstrap of the token Configure a control plane with KUMA_API_SERVER_AUTHN_TOKENS_BOOTSTRAP_ADMIN_TOKEN set to false.

  3. Delete admin-user-token Global Secret

    1. kumactl delete global-secret admin-user-token

  4. Disable bootstrap of the token Configure a control plane with KUMA_API_SERVER_AUTHN_TOKENS_BOOTSTRAP_ADMIN_TOKEN set to false.

Admin client certificates

This section describes the alternative way of authenticating to API Server.

Admin client certificates are deprecated. If you are using it, please migrate to the user token in preceding section.

To use admin client certificates, set KUMA_API_SERVER_AUTHN_TYPE to adminClientCerts.

All users that provide client certificate are authenticated as a user with the name mesh-system:admin that belongs to group mesh-system:admin.

Usage

  1. Generate client certificates by using kumactl

    1. kumactl generate tls-certificate --type=client \
    2.   --cert-file=/tmp/tls.crt \
    3.   --key-file=/tmp/tls.key

  2. Configure the control plane with client certificates

    Create a secret in the namespace in which control plane is installed

    1.    kubectl create secret generic api-server-client-certs -n kuma-system \
    2.      --from-file=client1.pem=/tmp/tls.crt \

    You can provide as many client certificates as you want. Remember to only provide certificates without keys.

    Point to this secret when installing Kuma

    1.    kumactl install control-plane \
    2.      --tls-api-server-client-certs-secret=api-server-client-certs

    Create a secret in the namespace in which control plane is installed

    1.    kubectl create secret generic api-server-client-certs -n kuma-system \
    2.      --from-file=client1.pem=/tmp/tls.crt \

    You can provide as many client certificates as you want. Remember to only provide certificates without keys.

    Set controlPlane.tls.apiServer.clientCertsSecretName to api-server-client-certs via HELM

    Put all the certificates in one directory

    1.    mkdir /opt/client-certs
    2.    cp /tmp/tls.crt /opt/client-certs/client1.pem

    All client certificates must end with .pem extension. Remember to only provide certificates without keys.

    Configure control plane by pointing to this directory

    1.    KUMA_API_SERVER_AUTH_CLIENT_CERTS_DIR=/opt/client-certs \
    2.      kuma-cp run

  3. Configure kumactl with valid client certificate

    1. kumactl config control-planes add \
    2.   --name=<NAME>
    3.   --address=https://<KUMA_CP_DNS_NAME>:5682 \
    4.   --client-cert-file=/tmp/tls.crt \
    5.   --client-key-file=/tmp/tls.key \
    6.   --ca-cert-file=/tmp/ca.crt

    If you want to skip CP verification, use --skip-verify instead of --ca-cert-file.

Multizone

In a multizone setup, users execute a majority of actions on the global control plane. However, some actions like generating dataplane tokens are available on the zone control plane. The global control plane doesn’t propagate authentication credentials to the zone control plane. You can set up consistent user tokens across the whole setup by manually copying signing key from global to zone control planes.