SDD 0008 - Platform Configuration Management

Commodore produces a Kapitan target (a target is the part of the Kapitan inventory which defines a unit for which configuration should be rendered) for the target cluster named cluster which includes the classes global.common, global.<kubernetes-distribution>, global.<cloud-provider> and <customer-name>.<cluster-name> . The class hierarchy is defined by the order the classes are included in the target. The current hierarchy is (weakest to strongest):

Components provide two classes: one which defines how the component’s templates should be rendered (symlinked to components.<component-name) and one which defines default parameters for the component (symlinked to defaults.<component-name>). All available default parameters classes are included before any configuration classes. This allows users to include components at any level in the configuration hierarchy without having to worry about a component’s defaults overriding customization which is applied earlier in the hierarchy. Component classes can always access all the configuration, even values that are only defined at a stronger hierarchy level. The defaults.*  notation in the list above is a short-hand notation for the mechanism in Commodore which includes each component’s defaults class explicitly.

The current hierarchy places the Kubernetes distribution before the cloud-provider in the configuration hierarchy. This allows selecting which components are included for each Kubernetes distribution which is supported by Commodore. F.e. this hierarchy allows to selectively include only components which have been tested when bootstrapping the configurations for a new Kubernetes distribution.

Operators are free to include additional classes at each of the global.*  hierarchy levels, f.e. an operator may choose to not have one large global.common  class but rather split up the fleet-wide defaults into smaller classes which are included in global.common , as indicated on the diagram below. The same structure can be applied at the cloud provider level, where the target only includes <cloud-provider>.<region>, but operators are free to keep cloud-wide defaults in one or more classes which are included by each cloud region class.

The customer level of the hierarchy, which is included via class <customer>.<cluster> and stored in a separate repository, can have any structure. The only requirement is that a <cluster>.yml  file exists in the repository for each SYN-managed cluster of the customer. This <cluster>.yml  file can itself include further classes (yaml files) which are defined in the customer repository by including the class as <customer>.<filename>. For example, a customer could have a common.yml  which defines their fleet-wide defaults. Each <cluster>.yml  can then include the customer defaults as <customer>.common .

Speculative

The Commodore job runner is responsible for running Commodore for an output repo if any of the inputs change (input repo, config repo, components). The job runner implementation is open to change, an early proposal was to run the Commodore jobs as CI jobs on the output repo. Commodore does include a Dockerfile which can be used to build a Commodore docker image.

Commodore is opinionated in how it supports Kapitan’s secret references: Only references to secrets in Vault (in the KV v2 storage) are supported, and Commodore uses the path to the Kapitan secret file in the reference to determine the name and key in the corresponding Vault secret. References to secrets must always be made in the configuration and never in Kapitan templates directly. This restriction allows Commodore to scan the configuration and create Kapitan secret files for all the references it finds. Secret files are committed to the output repo together with the catalog.