Variables

Defining and using variables in policies from multiple sources.

Variables make policies smarter and reusable by enabling references to data in the policy definition, the admission review request, and external data sources like ConfigMaps, the Kubernetes API Server, OCI image registries, and even external service calls.

Variables are stored as JSON and Kyverno supports using JMESPath (pronounced “James path”) to select and transform JSON data. With JMESPath, values from data sources are referenced in the format of {{key1.key2.key3}}. For example, to reference the name of an new/incoming resource during a kubectl apply action such as a Namespace, you would write this as a variable reference: {{request.object.metadata.name}}. The policy engine will substitute any values with the format {{ <JMESPath> }} with the variable value before processing the rule. For a page dedicated to exploring JMESPath’s use in Kyverno see here. Variables may be used in most places in a Kyverno rule or policy with one exception being in match or exclude statements.

Pre-defined Variables

Kyverno automatically creates a few useful variables and makes them available within rules:

  1. serviceAccountName: the “userName” which is the last part of a ServiceAccount (i.e. without the prefix system:serviceaccount:<namespace>:). For example, when processing a request from system:serviceaccount:nirmata:user1 Kyverno will store the value user1 in the variable serviceAccountName.

  2. serviceAccountNamespace: the “namespace” part of the ServiceAccount. For example, when processing a request from system:serviceaccount:nirmata:user1 Kyverno will store nirmata in the variable serviceAccountNamespace.

  3. request.roles: a list of roles stored in an array the given account may have. For example, ["foo:dave"].

  4. request.clusterRoles: a list of cluster roles stored in an array. For example, ["dave-admin","system:basic-user","system:discovery","system:public-info-viewer"]

  5. images: a map of container image information, if available. See Variables from container images for more information.

Variables from policy definitions

Kyverno policy definitions can refer to other fields in the policy definition as a form of “shortcut”. This can be a useful way to analyze and compare values without having to explicitly define them.

In order for Kyverno to refer to these existing values in a manifest, it uses the notation $(./../key_1/key_2). This may look familiar as it is essentially the same way Linux/Unix systems refer to relative paths. For example, consider the policy manifest snippet below.

 1validationFailureAction: Enforce
 2rules:
 3- name: check-tcpSocket
 4  match:
 5    any:
 6    - resources:
 7        kinds:
 8        - Pod
 9  validate:
10    message: "Port number for the livenessProbe must be less than that of the readinessProbe."
11    pattern:
12      spec:
13        ^(containers):
14        - livenessProbe:
15            tcpSocket:
16              port: "$(./../../../readinessProbe/tcpSocket/port)"
17          readinessProbe:
18            tcpSocket:
19              port: "3000"

In this above example, for any containers found in a Pod spec, the field readinessProbe.tcpSocket.port must be 3000 and the field livenessProbe.tcpSocket.port must be the same value. The lookup expression can be thought of as a cd back three levels and down into the readinessProbe object.

Operators also work on manifest lookup variables as well so the previous snippet could be modified as such.

1- livenessProbe:
2    tcpSocket:
3      port: "$(<./../../../readinessProbe/tcpSocket/port)"
4  readinessProbe:
5    tcpSocket:
6      port: "3000"

In this case, the field livenessProbe.tcpSocket.port must now be less than the value specified in readinessProbe.tcpSocket.port.

For more information on operators see the Operators section.

Escaping Variables

In some cases, you wish to write a rule containing a variable for action on by another program or process flow and not for Kyverno’s use. For example, with the variables in $() notation, these can be escaped with a leading backslash (\) and Kyverno will not attempt to substitute values. Variables written in JMESPath notation can also be escaped using the same syntax, for example \{{ request.object.metadata.name }}.

In the below policy, the value of OTEL_RESOURCE_ATTRIBUTES contains references to other environment variables which will be quoted literally as, for example, $(POD_NAMESPACE).

 1apiVersion: kyverno.io/v1
 2kind: Policy
 3metadata:
 4  name: add-otel-resource-env
 5  namespace: foobar
 6spec:
 7  background: false
 8  rules:
 9  - name: imbue-pod-spec
10    match:
11      any:
12      - resources:
13          kinds:
14          - v1/Pod
15    mutate:
16      patchStrategicMerge:
17        spec:
18          containers:
19          - (name): "?*"
20            env:
21            - name: NODE_NAME
22              value: "mutated_name"
23            - name: POD_IP_ADDRESS
24              valueFrom:
25                fieldRef:
26                  fieldPath: status.podIP
27            - name: POD_NAME
28              valueFrom:
29                fieldRef:
30                  fieldPath: metadata.name
31            - name: POD_NAMESPACE
32              valueFrom:
33                fieldRef:
34                  fieldPath: metadata.namespace
35            - name: POD_SERVICE_ACCOUNT
36              valueFrom:
37                fieldRef:
38                  fieldPath: spec.serviceAccountName
39            - name: OTEL_RESOURCE_ATTRIBUTES
40              value: >-
41                k8s.namespace.name=\$(POD_NAMESPACE),
42                k8s.node.name=\$(NODE_NAME),
43                k8s.pod.name=\$(POD_NAME),
44                k8s.pod.primary_ip_address=\$(POD_IP_ADDRESS),
45                k8s.pod.service_account.name=\$(POD_SERVICE_ACCOUNT),
46                rule_applied=$(./../../../../../../../../name)                

Using a Pod definition as below, this can be tested.

 1apiVersion: v1
 2kind: Pod
 3metadata:
 4  name: test-env-vars
 5spec:
 6  containers:
 7  - name: test-container
 8    image: busybox
 9    command: ["sh", "-c"]
10    args:
11    - while true; do
12      echo -en '\n';
13      printenv OTEL_RESOURCE_ATTRIBUTES;
14      sleep 10;
15      done;
16    env:
17    - name: NODE_NAME
18      value: "node_name"
19    - name: POD_NAME
20      valueFrom:
21        fieldRef:
22          fieldPath: metadata.name
23    - name: POD_NAMESPACE
24      valueFrom:
25        fieldRef:
26          fieldPath: metadata.namespace
27    - name: POD_IP_ADDRESS
28      valueFrom:
29        fieldRef:
30          fieldPath: status.podIP
31  restartPolicy: Never

The result of the mutation of this Pod with respect to the OTEL_RESOURCE_ATTRIBUTES environment variable will be as follows.

1- name: OTEL_RESOURCE_ATTRIBUTES
2      value: k8s.namespace.name=$(POD_NAMESPACE), k8s.node.name=$(NODE_NAME), k8s.pod.name=$(POD_NAME),
3        k8s.pod.primary_ip_address=$(POD_IP_ADDRESS), k8s.pod.service_account.name=$(POD_SERVICE_ACCOUNT),
4        rule_applied=imbue-pod-spec

Variables in Helm

Both Kyverno and Helm use Golang-style variable substitution syntax and, as a result, Kyverno policies containing variables deployed through Helm may need to be “wrapped” to avoid Helm interpreting them as Helm variables.

Because Helm executes its templating routine prior to Kyverno, a Kyverno policy with a variable {{ request.userInfo.username }} must be padded with Helm’s templating so that the resulting value, after the chart is deployed, remains {{ request.userInfo.username }}. Wrap the Kyverno variables in following way shown below:

{{`{{ request.userInfo.username }}`}}

For Kyverno variables which use more complex JMESPath expressions including existence checks, elements such as empty objects or arrays may also need to be escaped even within the overall expression. For example, the value of the below value field

value: "{{ element.securityContext.capabilities.drop[].to_upper(@) || `[]` }}"

would need to become

value: {{ `"{{ element.securityContext.capabilities.drop[].to_upper(@) || `}}`[]`{{` }}"` }}

in order to render properly.

Variables from admission review requests

Kyverno operates as a webhook inside Kubernetes. Whenever a new request is made to the Kubernetes API server, for example to create a Pod, the API server sends this information to the webhooks registered to listen to the creation of Pod resources. This incoming data to a webhook is passed as a AdmissionReview object. There are four commonly used data properties available in any AdmissionReview request:

  • {{request.operation}}: the type of API action being performed (CREATE, UPDATE, DELETE, or CONNECT).
  • {{request.object}}: the object being created or modified. It is null for DELETE requests.
  • {{request.oldObject}}: the object being modified. It is null for CREATE and CONNECT requests.
  • {{request.userInfo}}: contains information on who/what submitted the request which includes the groups and username keys.
  • {{request.namespace}}: the Namespace of the object subject to the operation.

Here are some examples of looking up this data:

  1. Reference a resource name (type string)

{{request.object.metadata.name}}

  1. Reference the metadata (type object)

{{request.object.metadata}}

  1. Reference the name of a new Namespace resource being created

{{request.object.name}}

  1. Reference the name of a user who submitted a request

{{request.userInfo.username}}

Variables from the AdmissionReview can also be combined with user-defined strings to create values for messages and other fields.

  1. Build a name from multiple variables (type string)

"ns-owner-{{request.namespace}}-{{request.userInfo.username}}-binding"

Let’s look at an example of how this AdmissionReview data can be used in Kyverno policies.

In the below ClusterPolicy, we wish to know which account created a given Pod resource. We can use information from the AdmissionReview contents, specifically the username key, to write this information out in the form of an annotation. Apply the following sample.

 1apiVersion: kyverno.io/v1
 2kind: ClusterPolicy
 3metadata:
 4  name: who-created-this
 5spec:
 6  background: false
 7  rules:
 8  - name: who-created-this
 9    match:
10      any:
11      - resources:
12          kinds:
13          - Pod
14    mutate:
15      patchStrategicMerge:
16        metadata:
17          annotations:
18            created-by: "{{request.userInfo.username}}"

This sample will mutate all incoming Pod creation requests with an annotation named created-by and the value of the authenticated user based on their kubeconfig.

Create a simple Pod resource.

1kubectl run busybox --image busybox:1.28

Now get the newly-created busybox Pod.

1kubectl get po busybox -o jsonpath='{.metadata.annotations}'
2{"created-by":"kubernetes-admin"}

In the output, we can clearly see the value of our created-by annotation is kubernetes-admin which, in this case, is the user who created the Pod.

Variables from container images

Kyverno extracts image data from the AdmissionReview request and makes this available as a variable named images of type map in the rule context. The following variables are set under images:

  • registry
  • path
  • name
  • tag
  • digest
  • reference
  • referenceWithTag

Here is an example:

 1{
 2  "containers": {
 3    "nginx": {
 4      "registry": "https://docker.io",
 5      "path": "library/nginx",
 6      "name": "nginx",
 7      "digest": "sha256:5f44022eab9198d75939d9eaa5341bc077eca16fa51d4ef32d33f1bd4c8cbe7d",
 8      "reference": "https://docker.io/library/nginx@sha256:5f44022eab9198d75939d9eaa5341bc077eca16fa51d4ef32d33f1bd4c8cbe7d",
 9      "referenceWithTag": "https://docker.io/library/nginx:"
10    }
11  },
12  "initContainers": {
13    "vault": {
14      "registry": "https://ghcr.io",
15      "path": "vault",
16      "name": "vault",
17      "tag": "v3",
18      "reference":"https://ghcr.io/vault:v3",
19      "referenceWithTag":"https://ghcr.io/vault:v3"
20    }
21  }
22}

Whenever an AdmissionReview request has containers, initContainers, or ephemeralContainers defined, the images variable can be referenced as shown in the examples below. tag and digest are mutually exclusive as an image may only define one.

Reference the image properties of container nginx:

  1. Reference the registry URL

{{images.containers.nginx.registry}}

  1. Reference the path to the image

{{images.containers.nginx.path}}

  1. Reference the image name

{{images.containers.nginx.name}}

  1. Reference the image tag

{{images.containers.nginx.tag}}

  1. Reference the digest

{{images.containers.nginx.digest}}

  1. Reference the readable reference for the image (along with the image digest(preferable) or tag)

{{images.containers.nginx.reference}}

  1. Reference the readable reference for the image (along with the image tag)

{{images.containers.nginx.referenceWithTag}}

For the container nginx, referenceWithTag corresponds to https://docker.io/library/nginx:. Since the image was specified with a digest, reference corresponds to https://docker.io/library/nginx@sha256:5f44022eab9198d75939d9eaa5341bc077eca16fa51d4ef32d33f1bd4c8cbe7d. reference enables referencing the image depending on how it was originally specified: if it was specified with a tag then reference will contain the tag; if specified with digest then reference will contain the digest.

Reference the image properties of initContainer vault:

  1. Reference the registry URL

{{images.initContainers.vault.registry}}

  1. Reference the path to the image

{{images.initContainers.vault.path}}

  1. Reference the image name

{{images.initContainers.vault.name}}

  1. Reference the image tag

{{images.initContainers.vault.tag}}

  1. Reference the digest

{{images.initContainers.vault.digest}}

  1. Reference the readable reference for the image (along with the image digest(preferable) or tag)

{{images.initContainers.vault.reference}}

  1. Reference the readable reference for the image (along with the image tag)

{{images.initContainers.vault.referenceWithTag}}

For initContainer vault, referenceWithTag corresponds to https://ghcr.io/vault:v3 since the image was specified with a tag and not a digest. The reference variable will also hold the same value.

This same pattern and image variable arrangement also works for ephemeral containers.

Kyverno by default sets an empty registry to docker.io and an empty tag to latest. The default registry and whether it should be substituted are configurable options defined in Kyverno’s ConfigMap.

You can also fetch image properties of all containers for further processing. For example, {{ images.containers.*.name }} creates a string list of all image names.

Inline Variables

Variables may be defined in a context for consumption by Kyverno rules. This can be as simple as a static value, another variable, or a nested object. Variables may also be redefined by using the same variable name. The last value that is set is used. The below sets a context variable with a value of foo.

1    context:
2    # A unique name for the for the variable 
3    # if the user redeclares a variable with the same name it should be re-assigned
4    - name: foodata
5      variable:
6        # value defines the value that the variable must have, it may contain jmespath variables or any yaml object that can be represented as a json object.
7        # value, default, and jmespath are optional but either value or jmespath must be defined.
8        value: "foo"

This snippet sets a context variable to the value of request.object.metadata.name. If the value field is not defined, the contents of jmesPath will act on the entire context.

1context:
2- name: objName
3  variable:
4    jmesPath: request.object.metadata.name

And below allows for an inline variable with a nested object as well as a default value for that object if it cannot be resolved. Even if the value is not defined, the default can still be set to global values such as other request.object.* variables from AdmissionReview requests.

 1    context:
 2    - name: nested-metadata
 3      variable:
 4        value:
 5          metadata:
 6            labels: 
 7              name: {{ request.object.metadata.name }}
 8        # the default value a variable may have if after jmespath processing the value ends up being nil
 9        # the default value may also be another variable, for example something from the AdmissionReview
10        default: '{}'
11        # jmespath expression that can be used to modify the `value` before it is assigned to the variable
12        jmesPath: 'to_string(@)'

Variables can reference other variables as well as shown below. Note that context variables are ordered; a variable consumed by another variable must be defined higher in the list of context variables.

1context:
2- name: jpExpression
3  variable:
4    value: name
5- name: objName
6  variable:
7    value:
8      name: "{{ request.object.metadata.name }}"
9    jmesPath: "{{ jpExpression }}"

Variables from external data sources

Some policy decisions require access to cluster resources and data managed by other Kubernetes controllers or external applications. For these types of policies, Kyverno allows HTTP calls to the Kubernetes API server and the use of ConfigMaps.

Data fetched from external sources is stored in a per-rule processing context that is used to evaluate variables by the policy engine. Once the data from external sources is stored in the context, it can be referenced like any other variable data.

Learn more about ConfigMap lookups and API Server calls in the External Data Sources section.

Nested Lookups

It is also possible to nest JMESPath expressions inside one another when mixing data sourced from a ConfigMap and AdmissionReview, for example. By including one JMESPath expression inside the other, Kyverno will first substitute the inner expression before building the outer one as seen in the below example.

 1apiVersion: kyverno.io/v1
 2kind: ClusterPolicy
 3metadata:
 4  name: resource-annotater
 5spec:
 6  background: false
 7  rules:
 8  - name: add-resource-annotations
 9    context:
10    - name: LabelsCM
11      configMap:
12        name: resource-annotater-reference
13        namespace: default
14    match:
15      any:
16      - resources:
17          kinds:
18          - Pod
19    mutate:
20      patchStrategicMerge:
21        metadata:
22          annotations:
23            foo: "{{LabelsCM.data.{{ request.object.metadata.labels.app }}}}"

In this example, AdmissionReview data is first collected in the inner expression in the form of {{request.object.metadata.labels.app}} while the outer expression is built from a ConfigMap context named LabelsCM.

Evaluation Order

Kyverno policies can contain variables in:

  • Rule context
  • Rule preconditions
  • Rule definitions:
    • Validation patterns
    • Validation deny rules
    • Mutate strategic merge patches (patchesStrategicMerge)
    • Generate resource data definitions
    • verifyImages definitions

Variables are not supported in the match and exclude elements, so that rules can be matched quickly without having to load and process data. Variables are also not supported in the patchesJson6902.path key.

Since variables can be nested, it is important to understand the order in which the variables are evaluated. During admission control, here is how the engine processes rules:

  1. The set of matching rules is determined by creating a hash from the request information to retrieve all matching rules based on the rule and resource types.
  2. Each matched rule is further processed to fully evaluate the match and retrieve conditions.
  3. The preconditions are then checked.
  4. The rule body is processed.

This ordering makes it possible to use request data when defining the context, and context variables in preconditions. Within the context itself, each variable is evaluated in the order of definition. Hence, if required, a variable can reference a prior variable but attempts to use a subsequent definition will result in errors. Context variables themselves are resolved when evaluated in the rule context except when the occur in a condition/expression.

JMESPath custom functions

In addition to the list of built-in functions JMESPath offers, Kyverno augments these by adding several others which makes it even easier to craft Kyverno policies.

The special variable {{ @ }} may be used to refer to the current value in a given field, useful for source values.

To find examples of some of these functions in action, see the Kyverno policies library. And for more complete information along with samples for each custom filter, see the JMESPath page here.


Last modified April 08, 2024 at 8:29 AM PST: Refactor links (#1205) (5060f3d)