# Architecture

Commodore’s operation can be separated into three rough stages: dependency fetching, catalog compilation, and secrets management.

Apart from these stages of operation, we also document other architectural choices in this page.

## Dependency fetching

Currently, the first part of Commodore’s operation revolves around fetching all the dependencies required to compile a catalog. Dependency fetching is implemented using Git repositories and `git clone` and calls to the Lieutenant API.

Commodore fetches the following dependencies:

• Cluster facts from Lieutenant API

• Global configuration

• Tenant configuration

• Components as discovered in global and tenant configuration

• Jsonnet libraries as described in the configuration hierarchy

### Component discovery

To discover all required components, Commodore reads the `applications` array which is made available by reclass. If a component should be disabled in a subset of the hierarchy, it can be removed from the `applications` array by adding the component name prefixed with a `~`.

``````applications:
- ~component-to-remove``````

Note that this only works to remove components which have been included previously, and won’t remove components that are included further down in the hierarchy.

Commodore currently has no mechanism to automatically discover components based on their names. Instead all components which are referenced in the hierarchy must be listed in key `components` in the file `commodore.yml` in the root of the global configuration repository. The key `components` holds an array of dictionaries with keys `name` and `url`. Each array entry specifies the repository URL of a single component. A sample `commodore.yml` can be found below. The file `commodore.yml` is also used to define the Commodore class hierarchy.

commodore.yml
``````components:
- name: argocd
url: https://github.com/projectsyn/component-argocd.git
- name: metrics-server
url: https://github.com/projectsyn/component-metrics-server.git``````
 Commodore will attempt to transform HTTP(S) Git URLs to their SSH-based counterparts when configuring the push URL on the local repository. This transformation allows authorized users to push feature branches to component repos without having to first manually adjust the repository’s push URL. User information (`user@` or `user:pass@`) and non-standard ports in HTTP(S) URLs will be removed when transforming the URL to a SSH-based push URL. The transformation assumes that SSH URLs follow the pattern `git@host:path/to/repo.git` (or `ssh://git@host/path/to/repo.git`). This assumption holds for many popular Git hosting services, such as GitHub and GitLab.

Additionally, component repositories can be overridden in the hierarchy in parameter `parameters.component_versions.<component-name>` by setting the key `url`. This can’t be used as a replacement for globally overriding the default location in `commodore.yml`, as the component must be discoverable without having to parse the Kapitan inventory. However, this mechanism can be used to configure a subset of managed clusters to use a fork of a component.

``````parameters:
component_versions:
argocd:
url: https://github.com/projectsyn/component-argocd/``````

### Component versioning

As described in the Commodore concepts, component versions and remote repository locations can be specified in the configuration hierarchy. Component versions are tracked in the inventory in key `version` under parameter `component_versions.<component-name>` and default to the component repository’s default branch.

An example:

``````parameters:
component_versions:
argocd:
version: v1.0.0``````

Reusing the configuration hierarchy for specifying the component versions allows tight integration of component version management with the rest of the configuration.

After cloning component repositories, all components will be checked out on the repository’s default branch. Commodore will then explicitly check out the version specified in the inventory for any components that specify a version. The version specification is parsed from the inventory using Kapitan’s inventory parsing, allowing the version to be overridden at any point in the configuration hierarchy.

### Component instantiation

With SDD0025, we’ve introduced support for instantiating components multiple times. As discussed in the design document, component authors must explicitly declare that their component supports instantiation. Components declare that they support instantiation by setting the field `multi_instance` in `parameters.<component_name>` to `true`. Commodore will exit with an error if a hierarchy tries to instantiate a component which hasn’t declared that it supports instantiation.

Component instance names aren’t namespaced per component, but must be globally unique. Commodore will exit with an error if the hierarchy uses the same instance name twice.

Component instances are declared in the `applications` array using `as` as the instantiation keyword. The current implementation of instances can be seen as a mechanism for introducing aliases for a component. Commodore supports hierarchies which include the same component non-aliased and aliased.

 Non-aliased components are internally transformed into the aliased identity form `component as component`. This enables support for hierarchies which want to include a component only using aliases. A component can be aliased to its own name, regardless of whether the component supports instantiation. Having a component explicitly included both as `component` and `component as component` will result in an error during compilation.

The merged content of `parameters.<component_name>` in the configuration hierarchy is used as the base configuration for each instance. If an instance-aware component is included non-aliased, that "instance" sees the merged content of `parameters.<component_name>` in the hierarchy. For all other instances of a component, the content of `parameters.<instance_name>` is merged into `parameters.<component_name>`. Commodore always sets the meta-parameter `parameters._instance` to the instance name. For non-aliased instances of instance-aware components, `parameters._instance` is set to the component name.

Let’s take the configuration below, which includes component `nfs-client-provisioner` twice, once non-aliased, and once aliased to `nfs-2`, as an example. In this example, we’ll end up with two instances of nfs-client-provisioner, which create volumes on `nfs.example.org:/path/to/share-1` and `nfs.example.org:/path/to/share-2` respectively.

 Commodore will apply the usual rules for the relationship between alias name and alias parameters key. Therefore the parameters key for an aliased component is the alias name, but with all dashes replaced by underscores.
tenant/common.yml
``````applications:
- nfs-client-provisioner
- nfs-client-provisioner as nfs-2
parameters:
nfs_client_provisioner:
server: nfs.example.org
path: /path/to/share-1
nfs_2:
path: /path/to/share-2``````

Similar to Helm charts, the components themselves must make sure to not cause any naming collisions of objects belonging to different instances. This is required both for namespaced and non-namespaced resources. Components can make use of the meta-parameter `_instance` to ensure objects don’t collide, as that parameter is guaranteed to be unique to each instance.

### Component dependencies

Components can specify their dependencies in a `jsonnetfile.json`. Commodore uses jsonnet-bundler to fetch component dependencies.

Components can optionally specify their dependencies in a `jsonnetfile.jsonnet`. In this case, Commodore renders the `jsonnetfile.jsonnet` into `jsonnetfile.json` before running jsonnet-bundler.

Commodore injects the key `parameters.<component_name>.jsonnetfile_parameters` as external variables when rendering the `jsonnetfile.jsonnet`.

 Jsonnet external variables must be string-valued. Therefore it’s not possible to simply pass the full `parameters.component_name` as external variables.

Below a `jsonnetfile.jsonnet` and corresponding `class/defaults.yml` for component `rancher-monitoring` are shown. The `rancher-monitoring` component depends on the `kube-prometheus` Jsonnet library, but requires different versions of the library depending on the target cluster’s Kubernetes version.

jsonnetfile.jsonnet
``````{
version: 1,
dependencies: [
{
source: {
git: {
remote: 'https://github.com/coreos/kube-prometheus',
subdir: 'jsonnet/kube-prometheus',
},
},
version: std.extVar('kube_prometheus_version'),
},
],
legacyImports: true,
}``````
class/defaults.yml
``````parameters:
rancher_monitoring:
kube_prometheus_version:
'1.17': 4e7440f742df31cd6da188f52ddc4e4037b81599
'1.18': f69ff3d63de17f3f52b955c3b7e0d7aff0372873
jsonnetfile_parameters:
# Default to K8s 1.18 if not overridden by cluster version
kube_prometheus_version: ${rancher_monitoring:kube_prometheus_version:1.18}`````` ## Catalog Compilation Commodore uses Kapitan to compile the cluster catalog. Commodore defines a Kapitan target for each component instance. Kapitan is called with a few options enabled. Most importantly, Kapitan is configured to support fetching dependencies of components, such as Helm charts. Further, Kapitan is configured with an extended search path to support component libraries and the builtin `commodore.libjsonnet`. Finally, Kapitan is also configured to search for secret reference files in `catalog/refs` during compilation. See section Secrets Management for more details on the secrets management implemented with Commodore and Kapitan. ### Postprocessing filters After running Kapitan, Commodore applies postprocessing filters to the output of Kapitan. Postprocessing filters allow components to describe transformations that should be applied to the rendered manifests of the component. Commodore supports two types of postprocessing filters: builtin filters and jsonnet filters. Builtin filters are defined by Commodore itself. Commodore currently provides a single builtin filter `helm_namespace` which is intended to be used on files generated by the Kapitan helm plugin. Postprocessing filters are defined in the component class in key `parameters.commodore.postprocess.filters`. This key is expected to hold a list of filter definitions. Each filter definition is an object, which must have keys `type`, `path` and `filter`. The field `type` defines whether the filter definition refers to a builtin or jsonnet filter. The field `path` indicates the directory on which the filter operates. The field `filter` defines which filter to apply. For builtin filters, the `filter` field holds the name of the builtin filter. For jsonnet filters, the `filter` field holds a the path to the jsonnet file defining the filter. The path to the jsonnet filter is relative to the component repository. Filters can be disabled by setting the optional field `enabled` in the filter definition to `false`. If this field isn’t present, filters are treated as enabled. A component can use the `helm_namespace` filter by providing the following filter configuration: component-metrics-server/class/metrics-server.yml ``````parameters: kapitan: ... commodore: postprocess: filters: - path: metrics-server/01_helmchart/metrics-server/templates type: builtin filter: helm_namespace filterargs: namespace:${metrics_server:namespace}
create_namespace: true``````

## Secrets Management

Commodore makes use of Kapitan’s secrets management capabilities, but currently only supports references to secrets in Vault (called "Vaultkv" in the Kapitan documentation).

Commodore takes care of generating secret reference files for any secret references (denoted by `?{vaultkv:…​}`) found in key `parameters` in all the classes included by the Kapitan cluster target. Secret references can use reclass references to define dynamic defaults, as Commodore searches for secret references in the rendered reclass inventory.

Commodore saves the generated reference files are stored in the cluster catalog in directory `refs/`. This directory is configured as the base path in which Kapitan searches for reference files during compilation, allowing references in the inventory to omit the `catalog/refs` prefix which the would have to include otherwise.

Because Commodore manages the secret files, it can guarantee that the secret files and the catalog are always in sync. All secret references MUST be made in the configuration parameters, otherwise Commodore can’t discover them. Additionally, compiled manifests MUST include the secret reference in clear text, for example by setting `stringData` for secret objects, as the secret revealing mechanism can’t find the references if they’re already base64 encoded.

### Secret file generation

Commodore generates the secret files and their contents according to specific rules. A Kapitan secret reference, for example `?{vaultkv:path/to/secret/thekey}`, always refers to a key named `thekey` in a secret named `path/to/secret` in Vault’s KV back-end. The address of the Vault instance and the name of the back-end are configurable:

``````parameters:
secret_management:
vault_addr: https://vault-prod.syn.vshn.net
# Name of the back-end (called mount in Vault)
vault_mount: kv``````

For the secret reference mentioned above, Commodore generates a Kapitan secret file in `catalog/refs/path/to/secret/thekey` with `path/to/secret:thekey` as the reference to the Vault secret.

Kapitan’s `vaultkv` secret engine is configured in the class `global.common` under the dict `secret_management`. The configuration defaults to vault-prod.syn.vshn.net and a back-end with name `clusters/kv`. This can be overridden at any level of the inventory.