Customizable Install with Istioctl

Follow this guide to install and configure an Istio mesh for in-depth evaluation or production use. If you are new to Istio, and just want to try it out, follow the quick start instructions instead.

This installation guide uses the istioctl command line tool to provide rich customization of the Istio control plane and of the sidecars for the Istio data plane. It has user input validation to help prevent installation errors and customization options to override any aspect of the configuration.

Using these instructions, you can select any one of Istio’s built-in configuration profiles and then further customize the configuration for your specific needs.

Full customization of the installation can be done through the IstioOperator API.

Prerequisites

Before you begin, check the following prerequisites:

  1. Download the Istio release.
  2. Perform any necessary platform-specific setup.
  3. Check the Requirements for Pods and Services.

Install Istio using the default profile

The simplest option is to install the default Istio configuration profile using the following command:

  1. $ istioctl install

This command installs the default profile on the cluster defined by your Kubernetes configuration. The default profile is a good starting point for establishing a production environment, unlike the larger demo profile that is intended for evaluating a broad set of Istio features.

To enable the Grafana dashboard on top of the default profile, set the addonComponents.grafana.enabled configuration parameter with the following command:

  1. $ istioctl install --set addonComponents.grafana.enabled=true

In general, you can use the --set flag in istioctl as you would with Helm. The only difference is you must prefix the setting paths with values. because this is the path to the Helm pass-through API in the IstioOperator API.

Install from external charts

By default, istioctl uses compiled-in charts to generate the install manifest. These charts are released together with istioctl for auditing and customization purposes and can be found in the release tar in the manifests directory. istioctl can also use external charts rather than the compiled-in ones. To select external charts, set the charts flag to a local file system path:

  1. $ istioctl install --charts=manifests/

If using the istioctl 1.6.0 binary, this command will result in the same installation as istioctl install alone, because it points to the same charts as the compiled-in ones. Other than for experimenting with or testing new features, we recommend using the compiled-in charts rather than external ones to ensure compatibility of the istioctl binary with the charts.

Install a different profile

Other Istio configuration profiles can be installed in a cluster by passing the profile name on the command line. For example, the following command can be used to install the demo profile:

  1. $ istioctl install --set profile=demo

Display the list of available profiles

You can display the names of Istio configuration profiles that are accessible to istioctl by using this command:

  1. $ istioctl profile list
  2. Istio configuration profiles:
  3.     minimal
  4.     preview
  5.     remote
  6.     default
  7.     demo
  8.     empty

Display the configuration of a profile

You can view the configuration settings of a profile. For example, to view the setting for the demo profile run the following command:

  1. $ istioctl profile dump demo
  2. addonComponents:
  3.   grafana:
  4.     enabled: true
  5.   kiali:
  6.     enabled: true
  7.   prometheus:
  8.     enabled: true
  9.   tracing:
  10.     enabled: true
  11. components:
  12.   egressGateways:
  13.   - enabled: true
  14.     k8s:
  15.       resources:
  16.         requests:
  17.           cpu: 10m
  18.           memory: 40Mi
  19.     name: istio-egressgateway
  21. ...

To view a subset of the entire configuration, you can use the --config-path flag, which selects only the portion of the configuration under the given path:

  1. $ istioctl profile dump --config-path components.pilot demo
  2. enabled: true
  3. k8s:
  4.   env:
  5.   - name: POD_NAME
  6.     valueFrom:
  7.       fieldRef:
  8.         apiVersion: v1
  9.         fieldPath: metadata.name
  10.   - name: POD_NAMESPACE
  11.     valueFrom:
  12.       fieldRef:
  13.         apiVersion: v1
  14.         fieldPath: metadata.namespace
  15.   - name: GODEBUG
  16.     value: gctrace=1
  17.   - name: PILOT_TRACE_SAMPLING
  18.     value: "100"
  19.   - name: CONFIG_NAMESPACE
  20.     value: istio-config
  21. ...

Show differences in profiles

The profile diff sub-command can be used to show the differences between profiles, which is useful for checking the effects of customizations before applying changes to a cluster.

You can show differences between the default and demo profiles using these commands:

  1. $ istioctl profile diff default demo
  2.  gateways:
  3.    egressGateways:
  4. -  - enabled: false
  5. +  - enabled: true
  6. ...
  7.      k8s:
  8.         requests:
  9. -          cpu: 100m
  10. -          memory: 128Mi
  11. +          cpu: 10m
  12. +          memory: 40Mi
  13.        strategy:
  14. ...

Generate a manifest before installation

You can generate the manifest before installing Istio using the manifest generate sub-command, instead of istioctl install. For example, use the following command to generate a manifest for the default profile:

  1. $ istioctl manifest generate > $HOME/generated-manifest.yaml

Inspect the manifest as needed, then apply the manifest using this command:

  1. $ kubectl apply -f $HOME/generated-manifest.yaml

This command might show transient errors due to resources not being available in the cluster in the correct order.

Show differences in manifests

You can show the differences in the generated manifests in a YAML style diff between the default profile and a customized install using these commands:

  1. $ istioctl manifest generate > 1.yaml
  2. $ istioctl manifest generate -f samples/operator/pilot-k8s.yaml > 2.yaml
  3. $ istioctl manifest diff 1.yaml 2.yaml
  4. Differences of manifests are:
  6. Object Deployment:istio-system:istio-pilot has diffs:
  8. spec:
  9.   template:
  10.     spec:
  11.       containers:
  12.         '[0]':
  13.           resources:
  14.             requests:
  15.               cpu: 500m -> 1000m
  16.               memory: 2048Mi -> 4096Mi
  17.       nodeSelector: -> map[master:true]
  18.       tolerations: -> [map[effect:NoSchedule key:dedicated operator:Exists] map[key:CriticalAddonsOnly
  19.         operator:Exists]]
  22. Object HorizontalPodAutoscaler:istio-system:istio-pilot has diffs:
  24. spec:
  25.   maxReplicas: 5 -> 10
  26.   minReplicas: 1 -> 2

Verify a successful installation

You can check if the Istio installation succeeded using the verify-install command which compares the installation on your cluster to a manifest you specify.

If you didn’t generate your manifest prior to deployment, run the following command to generate it now:

  1. $ istioctl manifest generate <your original installation options> > $HOME/generated-manifest.yaml

Then run the following verify-install command to see if the installation was successful:

  1. $ istioctl verify-install -f $HOME/generated-manifest.yaml

Customizing the configuration

In addition to installing any of Istio’s built-in configuration profiles, istioctl manifest provides a complete API for customizing the configuration.

The configuration parameters in this API can be set individually using --set options on the command line. For example, to enable the control plane security feature in a default configuration profile, use this command:

  1. $ istioctl install --set values.global.controlPlaneSecurityEnabled=true

Alternatively, the IstioOperator configuration can be specified in a YAML file and passed to istioctl using the -f option:

  1. $ istioctl install -f samples/operator/pilot-k8s.yaml

For backwards compatibility, the previous Helm installation options, with the exception of Kubernetes resource settings, are also fully supported. To set them on the command line, prepend the option name with “values.”. For example, the following command overrides the pilot.traceSampling Helm configuration option:

  1. $ istioctl install --set values.pilot.traceSampling=0.1

Helm values can also be set in an IstioOperator CR (YAML file) as described in Customize Istio settings using the Helm API, below.

If you want to set Kubernetes resource settings, use the IstioOperator API as described in Customize Kubernetes settings.

Identify an Istio component

The IstioOperator API defines components as shown in the table below:

Components
base
pilot
proxy
sidecarInjector
telemetry
policy
citadel
nodeagent
galley
ingressGateways
egressGateways
cni

In addition to the core Istio components, third-party addon components are also available. These can be enabled and configured through the addonComponents spec of the IstioOperator API or using the Helm pass-through API:

  1. apiVersion: install.istio.io/v1alpha1
  2. kind: IstioOperator
  3. spec:
  4.   addonComponents:
  5.     grafana:
  6.       enabled: true
  1. apiVersion: install.istio.io/v1alpha1
  2. kind: IstioOperator
  3. spec:
  4.   values:
  5.     grafana:
  6.       enabled: true

Configure component settings

After you identify the name of the component from the previous table, you can use the API to set the values using the --set flag, or create an overlay file and use the --filename flag. The --set flag works well for customizing a few parameters. Overlay files are designed for more extensive customization, or tracking configuration changes.

The simplest customization is to turn a component on or off from the configuration profile default.

To disable the telemetry component in a default configuration profile, use this command:

  1. $ istioctl install --set components.telemetry.enabled=false

Alternatively, you can disable the telemetry component using a configuration overlay file:

  1. Create this file with the name telemetry_off.yaml and these contents:
  1. apiVersion: install.istio.io/v1alpha1
  2. kind: IstioOperator
  3. spec:
  4.   components:
  5.     telemetry:
  6.       enabled: false
  1. Use the telemetry_off.yaml overlay file with the istioctl install command:
  1. $ istioctl install -f telemetry_off.yaml

Another customization is to select different namespaces for features and components. The following is an example of installation namespace customization:

  1. apiVersion: install.istio.io/v1alpha1
  2. kind: IstioOperator
  3. metadata:
  4.   namespace: istio-system
  5. spec:
  6.   components:
  7.     citadel:
  8.       namespace: istio-citadel

Applying this file will cause the default profile to be applied, with components being installed into the following namespaces:

  • The Citadel component is installed into istio-citadel namespace
  • Remaining Istio components installed into istio-system namespace

Configure gateways

Gateways are a special type of component, since multiple ingress and egress gateways can be defined. In the IstioOperator API, gateways are defined as a list type. The default profile installs one ingress gateway, called istio-ingressgateway. You can inspect the default values for this gateway:

  1. $ istioctl profile dump --config-path components.ingressGateways
  2. $ istioctl profile dump --config-path values.gateways.istio-ingressgateway

These commands show both the IstioOperator and Helm settings for the gateway, which are used together to define the generated gateway resources. The built-in gateways can be customized just like any other component. A new user gateway can be created by adding a new list entry:

  1. apiVersion: install.istio.io/v1alpha1
  2. kind: IstioOperator
  3. spec:
  4.   components:
  5.     ingressGateways:
  6.       - name: istio-ingressgateway
  7.         enabled: true
  8.       - namespace: user-ingressgateway-ns
  9.         name: ilb-gateway
  10.         enabled: true
  11.         k8s:
  12.           resources:
  13.             requests:
  14.               cpu: 200m
  15.           serviceAnnotations:
  16.             cloud.google.com/load-balancer-type: "internal"
  17.           service:
  18.             ports:
  19.             - port: 8060
  20.               targetPort: 8060
  21.               name: tcp-citadel-grpc-tls
  22.             - port: 5353
  23.               name: tcp-dns

Note that Helm values (spec.values.gateways.istio-ingressgateway/egressgateway) are shared by all ingress/egress gateways. If these must be customized per gateway, it is recommended to use a separate IstioOperator CR to generate a manifest for the user gateways, separate from the main Istio installation:

  1. apiVersion: install.istio.io/v1alpha1
  2. kind: IstioOperator
  3. spec:
  4.   profile: empty
  5.   components:
  6.     ingressGateways:
  7.       - name: ilb-gateway
  8.         namespace: user-ingressgateway-ns
  9.         enabled: true
  10.         # Copy settings from istio-ingressgateway as needed.
  11.   values:
  12.     gateways:
  13.       istio-ingressgateway:
  14.         debug: error

Customize Kubernetes settings

The IstioOperator API allows each component’s Kubernetes settings to be customized in a consistent way.

Each component has a KubernetesResourceSpec, which allows the following settings to be changed. Use this list to identify the setting to customize:

  1. Resources
  2. Readiness probes
  3. Replica count
  4. HorizontalPodAutoscaler
  5. PodDisruptionBudget
  6. Pod annotations
  7. Service annotations
  8. ImagePullPolicy
  9. Priority class name
  10. Node selector
  11. Affinity and anti-affinity
  12. Service
  13. Toleration
  14. Strategy
  15. Env

All of these Kubernetes settings use the Kubernetes API definitions, so Kubernetes documentation can be used for reference.

The following example overlay file adjusts the resources and horizontal pod autoscaling settings for Pilot:

  1. apiVersion: install.istio.io/v1alpha1
  2. kind: IstioOperator
  3. spec:
  4.   components:
  5.     pilot:
  6.       k8s:
  7.         resources:
  8.           requests:
  9.             cpu: 1000m # override from default 500m
  10.             memory: 4096Mi # ... default 2048Mi
  11.         hpaSpec:
  12.           maxReplicas: 10 # ... default 5
  13.           minReplicas: 2  # ... default 1
  14.         nodeSelector:
  15.           master: "true"
  16.         tolerations:
  17.         - key: dedicated
  18.           operator: Exists
  19.           effect: NoSchedule
  20.         - key: CriticalAddonsOnly
  21.           operator: Exists

Use istioctl install to apply the modified settings to the cluster:

  1. $ istioctl install -f samples/operator/pilot-k8s.yaml

Customize Istio settings using the Helm API

The IstioOperator API includes a pass-through interface to the Helm API using the values field.

The following YAML file configures global and Pilot settings through the Helm API:

  1. apiVersion: install.istio.io/v1alpha1
  2. kind: IstioOperator
  3. spec:
  4.   values:
  5.     pilot:
  6.       traceSampling: 0.1 # override from 1.0
  7.     global:
  8.       monitoringPort: 15050

Some parameters will temporarily exist in both the Helm and IstioOperator APIs, including Kubernetes resources, namespaces and enablement settings. The Istio community recommends using the IstioOperator API as it is more consistent, is validated, and follows the community graduation process.

Advanced install customization

Customizing external charts and profiles

The istioctl install, manifest generate and profile commands can use any of the following sources for charts and profiles:

  • compiled in charts. This is the default if no --charts option is set. The compiled in charts are the same as those in the manifests/ directory of the Istio release .tgz.
  • charts in the local file system, e.g., istioctl install --charts istio-1.6.0/manifests
  • charts in GitHub, e.g., istioctl install --charts https://github.com/istio/istio/releases/download/1.6.0/istio-1.6.0-linux-arm64.tar.gz

Local file system charts and profiles can be customized by editing the files in manifests/. For extensive changes, we recommend making a copy of the manifests directory and make changes there. Note, however, that the content layout in the manifests directory must be preserved.

Profiles, found under manifests/profiles/, can be edited and new ones added by creating new files with the desired profile name and a .yaml extension. istioctl scans the profiles subdirectory and all profiles found there can be referenced by name in the IstioOperatorSpec profile field. Built-in profiles are overlaid on the default profile YAML before user overlays are applied. For example, you can create a new profile file called custom1.yaml which customizes some settings from the default profile, and then apply a user overlay file on top of that:

  1. $ istioctl generate --charts mycharts/ --set profile=custom1 -f path-to-user-overlay.yaml

In this case, the custom1.yaml and user-overlay.yaml files will be overlaid on the default.yaml file to obtain the final values used as the input for manifest generation.

In general, creating new profiles is not necessary since a similar result can be achieved by passing multiple overlay files. For example, the command above is equivalent to passing two user overlay files:

  1. $ istioctl generate --charts mycharts/ -f manifests/profiles/custom1.yaml -f path-to-user-overlay.yaml

Creating a custom profile is only required if you need to refer to the profile by name through the IstioOperatorSpec.

Patching the output manifest

The IstioOperator CR, input to istioctl, is used to generate the output manifest containing the Kubernetes resources to be applied to the cluster. The output manifest can be further customized to add, modify or delete resources through the IstioOperator overlays API, after it is generated but before it is applied to the cluster.

The following example overlay file (patch.yaml) demonstrates the type of output manifest patching that can be done:

  1. apiVersion: install.istio.io/v1alpha1
  2. kind: IstioOperator
  3. spec:
  4.   profile: empty
  5.   hub: docker.io/istio
  6.   tag: 1.1.6
  7.   components:
  8.     pilot:
  9.       enabled: true
  10.       namespace: istio-control
  11.       k8s:
  12.         overlays:
  13.           - kind: Deployment
  14.             name: istiod
  15.             patches:
  16.               # Select list item by value
  17.               - path: spec.template.spec.containers.[name:discovery].args.[30m]
  18.                 value: "60m" # overridden from 30m
  19.               # Select list item by key:value
  20.               - path: spec.template.spec.containers.[name:discovery].ports.[containerPort:8080].containerPort
  21.                 value: 1234
  22.               # Override with object (note | on value: first line)
  23.               - path: spec.template.spec.containers.[name:discovery].env.[name:POD_NAMESPACE].valueFrom
  24.                 value: |
  25.                   fieldRef:
  26.                     apiVersion: v2
  27.                     fieldPath: metadata.myPath
  28.               # Deletion of list item
  29.               - path: spec.template.spec.containers.[name:discovery].env.[name:REVISION]
  30.               # Deletion of map item
  31.               - path: spec.template.spec.containers.[name:discovery].securityContext
  32.           - kind: Service
  33.             name: istiod
  34.             patches:
  35.               - path: spec.ports.[name:https-dns].port
  36.                 value: 11111 # OVERRIDDEN

Passing the file to istioctl manifest generate -f patch.yaml applies the above patches to the default profile output manifest. The two patched resources will be modified as shown below (some parts of the resources are omitted for brevity):

  1. apiVersion: apps/v1
  2. kind: Deployment
  3. metadata:
  4.   name: istiod
  5. spec:
  6.   template:
  7.     spec:
  8.       containers:
  9.       - args:
  10.         - 60m
  11.         env:
  12.         - name: POD_NAMESPACE
  13.           valueFrom:
  14.             fieldRef:
  15.               apiVersion: v2
  16.               fieldPath: metadata.myPath
  17.         name: discovery
  18.         ports:
  19.         - containerPort: 1234
  20. ---
  21. apiVersion: v1
  22. kind: Service
  23. metadata:
  24.   name: istiod
  25. spec:
  26.   ports:
  27.   - name: https-dns
  28.     port: 11111
  29. ---

Note that the patches are applied in the given order. Each patch is applied over the output from the previous patch. Paths in patches that don’t exist in the output manifest will be created.

List item path selection

Both the istioctl --set flag and the k8s.overlays field in IstioOperator CR support list item selection by [index], [value] or by [key:value]. The –set flag also creates any intermediate nodes in the path that are missing in the resource.

Uninstall Istio

To uninstall Istio, run the following command:

  1. $ istioctl manifest generate <your original installation options> | kubectl delete -f -

The control plane namespace (e.g., istio-system) is not removed by default. If no longer needed, use the following command to remove it:

  1. $ kubectl delete namespace istio-system

See also

Diagnose your Configuration with Istioctl Analyze

Shows you how to use istioctl analyze to identify potential issues with your configuration.

Understand your Mesh with Istioctl Describe

Shows you how to use istioctl describe to verify the configurations of a pod in your mesh.

DNS Certificate Management

Provision and manage DNS certificates in Istio.

Introducing istioctl analyze

Analyze your Istio configuration to detect potential issues and get general insights.

Introducing the Istio Operator

Introduction to Istio’s new operator-based installation and control plane management feature.

Secure Webhook Management

A more secure way to manage Istio webhooks.