GitOps Best Practices for Turtles and CAPI

This section provides best practices for implementing GitOps with Turtles and Cluster API (CAPI), focusing on using a combination of CAPI features and Continuous Delivery for automating initial bootstrapping and application deployment.

These tips are transferable and can be applied even when using a different GitOps solution.

Common Tips

  • Separate CAPI Clusters and ClusterClass Configurations: Keep CAPI cluster configurations (clusters/) distinct from CAPI ClusterClass definitions (templates/). This separation allows multiple clusters to be provisioned from shared templates while reducing the risk of unintended infrastructure changes.

  • Isolate Addon Configurations: Store application manifests in applications/, ensuring they are independent of Clusters and ClusterClasses. This enables selective or grouped provisioning based on user demand or matching selectors, preventing interference with cluster infrastructure. Please be aware that Helm chart dependencies need to be refreshed prior to addon provisioning.

  • Utilize Nested GitRepo Resources: Leverage nested GitRepo resources in Fleet to structure repository content efficiently. This approach allows for modular provisioning, combining ClusterClass definitions, required addons/applications, and cluster configurations while maintaining separation of concerns.

  • Apply Kustomize Overlays for Environment-Specific Customization: Use overlays/ to store Kustomize overlays for different environments. This ensures that configuration changes, such as scaling adjustments, networking modifications, or component upgrades, can be applied dynamically without altering base manifests.

  • Utilize Compare Patches: Fleet allows performing selective ignoring of resource differences if the bundle contains proper rules for diff detection. This would prevent bundles from being stuck in a modified state.

Folder Separation Strategy

A well-structured repository simplifies management and improves clarity. The following folder separation strategy is recommended:

repo-root/
│── fleet/
│   ├── providers/         # CAPIProvider files to use with templates
│   │   ├── core/          # Core provider configuration
│   │   ├── infra-docker/  # Docker provider configuration
│   │   ├── infra-aws/     # AWS provider configuration
│   ├── clusters/          # Cluster-specific configurations
│   │   ├── prod/          # Production clusters
│   │   ├── staging/       # Staging clusters
│   │   ├── dev/           # Development clusters
│   ├── addons/            # Application-specific manifests focused on cluster bootstrapping, like CNI, CCM, CPI configurations.
│   │   ├── cni/           # Group applications based on purpose. Provide fleet.yaml per each sub-directory to maintain bundle separation.
│   │   ├── ccm/
│   ├── templates/         # CAPI ClusterClass templates for cluster provisioning
│   ├── fleet.yaml         # Fleet bundle configuration
│── overlays/              # Kustomize overlays for different environments
│   ├── prod/.             # Kustomize overlays for production environments
│   ├── staging/.          # Kustomize overlays for staging environments
│   ├── dev/.              # Kustomize overlays for development environments

Providers Configuration

CAPI providers define the infrastructure components necessary for cluster provisioning. A simple setup may look like:

repo-root/
│── fleet/
│   ├── providers/
│   │   ├── core/
│   │   |   ├── core.yaml  # Core `CAPIProvider` definition
│   │   |   ├── fleet.yaml # Fleet.yaml definition for the provider setup

where the core provider contains:

apiVersion: turtles-capi.cattle.io/v1alpha1
kind: CAPIProvider
metadata:
  name: cluster-api
spec:
  type: core

Templates Configuration

Specify dependsOn in fleet.yaml for specific CAPIProvider bundles or external GitRepo components. This ensures templates roll out in the correct order and prevents issues where component controllers are missing or not ready to accept ClusterClass definitions.

namespace: capa-clusters
dependsOn:
  - name: core      # Ensure core bundle is installed first
  - name: infra-aws # Wait for AWS provider installation