Feature: environments by piontec · Pull Request #41 · giantswarm/gitops-template

piontec · 2022-06-06T13:34:01Z

This PR introduces the concept of environments with a mono-repo directory-separated layout. I think we need to propose branch separated envs as well.

A few notes about this PR:

In top level cluster files both cluster_id and cluster_name are needed, as some definitions include a base using one var, some the other. We have to unify that.
It is assumed that there are not many clusters, so there's no need to further group them in workload-clusters dir. Should we change that already and make prod/dev/stage subdirs there?

MarcelMue · 2022-06-06T13:42:01Z

bases/environments/deployments/dev/hello_app_cluster/automatic_updates/imageupdate.yaml

@@ -0,0 +1,28 @@
+apiVersion: image.toolkit.fluxcd.io/v1beta1


Why deployments in the directory path? I don't understand what that subfolder is for.

In general, we want more critera than just 'deployment stage', like for example region. That's what I'm preparing here: imagine that there's 1 more folder region there, with east and west subfolders, carrying some infra specific configs. That way you can create a dev cluster in DC east by inheriting from both these labels (but here we have that config merging shit...).

deployments just feels like the wrong name to me then stage could be more descriptive.

I had exactly the same idea, but figured stage/stageing sounds stupid :P Still, I'm open for suggestions. I was also thinking about release.WDYT?

I think stage/staging is fine. Just my current believe that users immediately know what stage means and nearly none will intuitively understand deployment.

Can we change this to stage for now and then continue on here? I still think it will be easier to explain to customers / use.

MarcelMue · 2022-06-06T13:43:28Z

...ement-clusters/MC_NAME/organizations/ORG_NAME/workload-clusters/HELLO_APP_DEV_CLUSTER_1.yaml

+  postBuild:
+    substitute:
+      cluster_domain: "MY_DOMAIN"
+      cluster_id: "HELLO_DEV_1"


The naming here is really inconsistent. HELLO_APP_DEV and HELLO_DEV are used interchangeably.

MarcelMue · 2022-06-06T13:44:41Z

bases/cluster_templates/hello_app_cluster/app_sets/hello-web-app/kustomization.yaml

@@ -0,0 +1,12 @@
+apiVersion: kustomize.config.k8s.io/v1beta1


Why are changes in the app_sets subfolder needed in this PR? It seems like a separate area to me.

This is a change in cluster_templates, not base app_sets. Tunes the app_set for this clsuter_template.

…emplate into feature/environments

…oping them

docs/add_wc_environments.md

Co-authored-by: Lukasz Piatkowski <piontec@gmail.com>

uvegla · 2022-06-20T15:22:19Z

Example kubeconform output (after sed-ing the stdin - from the beginning):

Deployment nginx-deployment is invalid: For field spec: selector is required - For field spec: template is required

This is the example output for kubeval for the same file for reference:

WARN - Set to ignore missing schemas
WARN - stdin contains an invalid Deployment (nginx-deployment) - selector: selector is required
WARN - stdin contains an invalid Deployment (nginx-deployment) - template: template is required

uvegla · 2022-06-20T15:23:31Z

I think it i s nice that kubeconform supports different machine parsable formats like json and common test output formats like tap and junit.

The kubeval tool does not seem to be maintained anymore.

uvegla · 2022-06-20T16:20:55Z

The kubeval replacement with kubeconform was move to a separate PR and got merged to main from there than merged back to this branch.

docs/add_wc_environments.md

Co-authored-by: Bricktop <marcel.mueller1@rwth-aachen.de>

MarcelMue · 2022-06-23T08:21:33Z

docs/add_wc_environments.md

+
+### Stages
+
+We have 3 example clusters under [/bases/environments/stages](/bases/environments/stages):


All other docs follow an instruction format of "this is what you need to do to have environments". Here we now have a mix of "look at these files & this is what you can do". Should there be a clear guide of "this is how you add a stage"?

I think it would make sense and I would love it. I was pondering that myself as well while writing docs.

@MarcelMue Created a ticket for this to discuss async. Also addes topic/documentation as it might make sense for sig/docs as well. See: https://github.com/giantswarm/giantswarm/issues/22611

… template

uvegla · 2022-06-28T13:07:44Z

@piontec @giantswarm/team-honeybadger Updated the docs with the code examples. Please check!

piontec · 2022-06-29T09:05:03Z

Thanks @uvegla!

github-actions · 2022-06-29T09:13:47Z

Rendered manifest diff output log


(file level)
    ---
    apiVersion: v1
    data:
      values: |
        admin_login: hello-admin-in-cluster-template
        db_config:
          db_port: 5433 # has to be in sync with config in default_config_simple_db.yaml
          db_user: hello-web-app # has to be in sync with config in default_config_simple_db.yaml
          host: simple-db.simple-web.svc # has to be in sync with 'namespace:' set in kustomization.yaml
        thread_pool_size: 64
        
    kind: ConfigMap
    metadata:
      name: HELLO_APP_STAGING_1-hello-world-values
      namespace: hello-web
      annotations:
        config.kubernetes.io/origin: |
          configuredIn: ../../../../../../bases/cluster_templates/hello_app_cluster/app_sets/hello-web-app/kustomization.yaml
          configuredBy:
            apiVersion: builtin
            kind: ConfigMapGenerator
          
      labels:
        gitops.giantswarm.io/appSet: hello-web-app
    apiVersion: v1
    data:
      values: |
        port: 5433 # has to be in sync with config in default_config_hello_world.yaml
        user: hello-web-app # has to be in sync with config in default_config_hello_world.yaml
        
    kind: ConfigMap
    metadata:
      annotations:
        config.kubernetes.io/origin: |
          configuredIn: ../../../../../../bases/app_sets/hello-web-app/kustomization.yaml
          configuredBy:
            apiVersion: builtin
            kind: ConfigMapGenerator
          
      labels:
        gitops.giantswarm.io/appSet: hello-web-app
      name: HELLO_APP_STAGING_1-simple-db-values
      namespace: hello-web
    apiVersion: v1
    data:
      values: |
        bastion:
          bootFromVolume: true
          diskSize: 10
          flavor: n1.tiny
          image: aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa
        cloudConfig: cloud-config
        cloudName: openstack
        clusterName: HELLO_APP_STAGING_1
        controlPlane:
          availabilityZones:
          - demo_az_1
          - demo_az_2
          - demo_az_3
          bootFromVolume: true
          diskSize: 50
          flavor: n1.small
          image: bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb
          replicas: 3
        externalNetworkID: cccccccc-cccc-cccc-cccc-cccccccccccc
        kubernetesVersion: v1.20.9
        nodeCIDR: 10.6.0.0/24
        nodeClasses:
        - bootFromVolume: true
          diskSize: 50
          flavor: n1.small
          image: dddddddd-dddd-dddd-dddd-dddddddddddd
          name: default
        nodePools:
        - class: default
          failureDomain: demo_az_1
          name: demo_az_1
          replicas: 1
        - class: default
          failureDomain: demo_az_2
          name: demo_az_2
          replicas: 1
        - class: default
          failureDomain: demo_az_3
          name: demo_az_3
          replicas: 1
        organization: ORG_NAME
        
    kind: ConfigMap
    metadata:
      annotations:
        config.kubernetes.io/origin: |
          configuredIn: ../../../../../../bases/clusters/capo/>=v0.6.0/kustomization.yaml
          configuredBy:
            apiVersion: builtin
            kind: ConfigMapGenerator
          
      labels:
        giantswarm.io/managed-by: flux
      name: HELLO_APP_STAGING_1-config
      namespace: org-ORG_NAME
    apiVersion: v1
    data:
      values: |
        clusterName: HELLO_APP_STAGING_1
        organization: ORG_NAME
        
    kind: ConfigMap
    metadata:
      annotations:
        config.kubernetes.io/origin: |
          configuredIn: ../../../../../../bases/clusters/capo/template/kustomization.yaml
          configuredBy:
            apiVersion: builtin
            kind: ConfigMapGenerator
          
      labels:
        giantswarm.io/managed-by: flux
      name: HELLO_APP_STAGING_1-default-apps-config
      namespace: org-ORG_NAME
    apiVersion: v1
    data:
      values: |
        cloudConfig: cloud-config-giantswarm-2
        cloudName: openstack
        externalNetworkID: aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee
        
    kind: ConfigMap
    metadata:
      annotations:
        config.kubernetes.io/origin: |
          path: ../../../../../../bases/cluster_templates/hello_app_cluster/cluster/cluster_user_config.yaml
          
      labels:
        giantswarm.io/managed-by: flux
      name: HELLO_APP_STAGING_1-user-config
      namespace: org-ORG_NAME
    apiVersion: v1
    data:
      values: |
        thread_pool_size: 64
        
    kind: ConfigMap
    metadata:
      annotations:
        config.kubernetes.io/origin: |
          configuredIn: ../../../../../../bases/environments/stages/staging/hello_app_cluster/kustomization.yaml
          configuredBy:
            apiVersion: builtin
            kind: ConfigMapGenerator
          
      name: HELLO_APP_STAGING_1-hello-world-user-config
    apiVersion: application.giantswarm.io/v1alpha1
    kind: App
    metadata:
      annotations:
        config.kubernetes.io/origin: |
          path: ../../../../../../bases/apps/hello-world/appcr.yaml
          
      labels:
        gitops.giantswarm.io/appSet: hello-web-app
      name: HELLO_APP_STAGING_1-hello-world
      namespace: hello-web
    spec:
      catalog: giantswarm
      config:
        configMap:
          name: HELLO_APP_STAGING_1-hello-world-values
          namespace: hello-world-app
      kubeConfig:
        inCluster: false
      name: hello-world-app
      namespace: hello-world-app
      version: "0.1.8 # {"": "org-ORG_NAME:HELLO_APP_STAGING_1-hello-app"}"
    apiVersion: application.giantswarm.io/v1alpha1
    kind: App
    metadata:
      annotations:
        config.kubernetes.io/origin: |
          path: ../../../../../../bases/apps/simple-db/appcr.yaml
          
      labels:
        gitops.giantswarm.io/appSet: hello-web-app
      name: HELLO_APP_STAGING_1-simple-db
      namespace: hello-web
    spec:
      catalog: giantswarm
      config:
        configMap:
          name: HELLO_APP_STAGING_1-simple-db-values
          namespace: simple-db-app
      kubeConfig:
        inCluster: false
      name: simple-db-app
      namespace: simple-db-app
      version: "0.1.0 # {"": "org-ORG_NAME:HELLO_APP_STAGING_1-simple-db-app"}"
    apiVersion: application.giantswarm.io/v1alpha1
    kind: App
    metadata:
      annotations:
        config.kubernetes.io/origin: |
          path: ../../../../../../bases/clusters/capo/template/cluster.yaml
          
      labels:
        app-operator.giantswarm.io/version: 0.0.0
        giantswarm.io/managed-by: flux
      name: HELLO_APP_STAGING_1
      namespace: org-ORG_NAME
    spec:
      catalog: giantswarm
      config:
        configMap:
          name: HELLO_APP_STAGING_1-config
          namespace: org-ORG_NAME
      kubeConfig:
        inCluster: true
      name: cluster-openstack
      namespace: org-ORG_NAME
      userConfig:
        configMap:
          name: HELLO_APP_STAGING_1-user-config
          namespace: org-ORG_NAME
      version: 0.8.1
    apiVersion: application.giantswarm.io/v1alpha1
    kind: App
    metadata:
      annotations:
        config.kubernetes.io/origin: |
          path: ../../../../../../bases/clusters/capo/template/default_apps.yaml
          
      labels:
        app-operator.giantswarm.io/version: 0.0.0
        giantswarm.io/managed-by: flux
      name: HELLO_APP_STAGING_1-default-apps
      namespace: org-ORG_NAME
    spec:
      catalog: giantswarm
      config:
        configMap:
          name: HELLO_APP_STAGING_1-default-apps-config
          namespace: org-ORG_NAME
      kubeConfig:
        inCluster: true
      name: default-apps-openstack
      namespace: org-ORG_NAME
      version: "${default_apps_release:=0.1.0}"
    apiVersion: application.giantswarm.io/v1alpha1
    kind: Catalog
    metadata:
      annotations:
        config.kubernetes.io/origin: |
          path: ../../../../../../bases/environments/stages/dev/hello_app_cluster/automatic_updates/catalog.yaml
          
      labels:
        application.giantswarm.io/catalog-visibility: internal
        giantswarm.io/managed-by: flux
      name: giantswarm-catalog-oci
      namespace: org-ORG_NAME
    spec:
      description: giantswarm-catalog-oci
      logoURL: "https://avatars.githubusercontent.com/u/7556340?s=60&v=4"
      repositories:
      - URL: "oci://giantswarmpublic.azurecr.io/giantswarm-catalog/"
        type: helm
      storage:
        URL: "oci://giantswarmpublic.azurecr.io/giantswarm-catalog/"
        type: helm
      title: giantswarm-catalog-oci
    apiVersion: image.toolkit.fluxcd.io/v1beta1
    kind: ImagePolicy
    metadata:
      annotations:
        config.kubernetes.io/origin: |
          path: ../../../../../../bases/environments/stages/staging/hello_app_cluster/imagepolicies.yaml
          
      name: HELLO_APP_STAGING_1-hello-app
    spec:
      imageRepositoryRef:
        name: HELLO_APP_STAGING_1-hello-app
      policy:
        semver:
          range: ">=0.1.0 <1.0.0"
    apiVersion: image.toolkit.fluxcd.io/v1beta1
    kind: ImagePolicy
    metadata:
      annotations:
        config.kubernetes.io/origin: |
          path: ../../../../../../bases/environments/stages/staging/hello_app_cluster/imagepolicies.yaml
          
      name: HELLO_APP_STAGING_1-simple-db-app
    spec:
      imageRepositoryRef:
        name: HELLO_APP_STAGING_1-simple-db
      policy:
        semver:
          range: ">=0.1.0 <0.2.0"
    apiVersion: image.toolkit.fluxcd.io/v1beta1
    kind: ImageRepository
    metadata:
      annotations:
        config.kubernetes.io/origin: |
          path: ../../../../../../bases/environments/stages/staging/hello_app_cluster/imagerepositories.yaml
          
      name: HELLO_APP_STAGING_1-hello-app
    spec:
      image: giantswarmpublic.azurecr.io/giantswarm-catalog/hello-world-app
      interval: 10m0s
    apiVersion: image.toolkit.fluxcd.io/v1beta1
    kind: ImageRepository
    metadata:
      annotations:
        config.kubernetes.io/origin: |
          path: ../../../../../../bases/environments/stages/staging/hello_app_cluster/imagerepositories.yaml
          
      name: HELLO_APP_STAGING_1-simple-db-app
    spec:
      image: giantswarmpublic.azurecr.io/giantswarm-catalog/simple-db-app
      interval: 10m0s
    apiVersion: image.toolkit.fluxcd.io/v1beta1
    kind: ImageUpdateAutomation
    metadata:
      annotations:
        config.kubernetes.io/origin: |
          path: ../../../../../../bases/environments/stages/staging/hello_app_cluster/automatic_updates/imageupdate.yaml
          
      labels:
        giantswarm.io/managed-by: flux
      name: HELLO_APP_STAGING_1-image-updates
      namespace: org-ORG_NAME
    spec:
      git:
        checkout:
          ref:
            branch: main
        commit:
          author:
            email: fluxcdbot@users.noreply.github.com
            name: fluxcdbot
          messageTemplate: |
            automated app upgrades:
            {{ range , /usr/bin/envsubst := .Updated.Images -}}
            - {{ .Repository }} to {{ .Identifier }}
            {{ end -}}
            
        push:
          branch: main
      interval: 1m0s
      sourceRef:
        kind: GitRepository
        name: YOUR_GIT_REPO
      update:
        path: ./management-clusters/MC_NAME
        strategy: Setters
    apiVersion: v1
    data:
      values: |
        admin_login: hello-admin-in-cluster-template
        db_config:
          db_port: 5433 # has to be in sync with config in default_config_simple_db.yaml
          db_user: hello-web-app # has to be in sync with config in default_config_simple_db.yaml
          host: simple-db.simple-web.svc # has to be in sync with 'namespace:' set in kustomization.yaml
        thread_pool_size: 64
        
    kind: ConfigMap
    metadata:
      annotations:
        config.kubernetes.io/origin: |
          configuredIn: ../../../../../../bases/cluster_templates/hello_app_cluster/app_sets/hello-web-app/kustomization.yaml
          configuredBy:
            apiVersion: builtin
            kind: ConfigMapGenerator
          
      labels:
        gitops.giantswarm.io/appSet: hello-web-app
      name: HELLO_APP_PROD_1-hello-world-values
      namespace: hello-web
    apiVersion: v1
    data:
      values: |
        port: 5433 # has to be in sync with config in default_config_hello_world.yaml
        user: hello-web-app # has to be in sync with config in default_config_hello_world.yaml
        
    kind: ConfigMap
    metadata:
      annotations:
        config.kubernetes.io/origin: |
          configuredIn: ../../../../../../bases/app_sets/hello-web-app/kustomization.yaml
          configuredBy:
            apiVersion: builtin
            kind: ConfigMapGenerator
          
      labels:
        gitops.giantswarm.io/appSet: hello-web-app
      name: HELLO_APP_PROD_1-simple-db-values
      namespace: hello-web
    apiVersion: v1
    data:
      values: |
        bastion:
          bootFromVolume: true
          diskSize: 10
          flavor: n1.tiny
          image: aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa
        cloudConfig: cloud-config
        cloudName: openstack
        clusterName: HELLO_APP_PROD_1
        controlPlane:
          availabilityZones:
          - demo_az_1
          - demo_az_2
          - demo_az_3
          bootFromVolume: true
          diskSize: 50
          flavor: n1.small
          image: bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb
          replicas: 3
        externalNetworkID: cccccccc-cccc-cccc-cccc-cccccccccccc
        kubernetesVersion: v1.20.9
        nodeCIDR: 10.6.0.0/24
        nodeClasses:
        - bootFromVolume: true
          diskSize: 50
          flavor: n1.small
          image: dddddddd-dddd-dddd-dddd-dddddddddddd
          name: default
        nodePools:
        - class: default
          failureDomain: demo_az_1
          name: demo_az_1
          replicas: 1
        - class: default
          failureDomain: demo_az_2
          name: demo_az_2
          replicas: 1
        - class: default
          failureDomain: demo_az_3
          name: demo_az_3
          replicas: 1
        organization: ORG_NAME
        
    kind: ConfigMap
    metadata:
      annotations:
        config.kubernetes.io/origin: |
          configuredIn: ../../../../../../bases/clusters/capo/>=v0.6.0/kustomization.yaml
          configuredBy:
            apiVersion: builtin
            kind: ConfigMapGenerator
          
      labels:
        giantswarm.io/managed-by: flux
      name: HELLO_APP_PROD_1-config
      namespace: org-ORG_NAME
    apiVersion: v1
    data:
      values: |
        clusterName: HELLO_APP_PROD_1
        organization: ORG_NAME
        
    kind: ConfigMap
    metadata:
      annotations:
        config.kubernetes.io/origin: |
          configuredIn: ../../../../../../bases/clusters/capo/template/kustomization.yaml
          configuredBy:
            apiVersion: builtin
            kind: ConfigMapGenerator
          
      labels:
        giantswarm.io/managed-by: flux
      name: HELLO_APP_PROD_1-default-apps-config
      namespace: org-ORG_NAME
    apiVersion: v1
    data:
      values: |
        values: |
          cloudConfig: cloud-config-giantswarm-2
          cloudName: openstack
          externalNetworkID: prod-bbbb-cccc-dddd-eeeeeeeeeeee
          nodeClasses:
            - bootFromVolume: true
              diskSize: 150
              flavor: n1.large
              image: dddddddd-dddd-dddd-dddd-dddddddddddd
              name: default
        
    kind: ConfigMap
    metadata:
      annotations:
        config.kubernetes.io/origin: |
          configuredIn: ../../../../../../bases/environments/stages/prod/hello_app_cluster/kustomization.yaml
          configuredBy:
            apiVersion: builtin
            kind: ConfigMapGenerator
          
      labels:
        giantswarm.io/managed-by: flux
      name: HELLO_APP_PROD_1-user-config
      namespace: org-ORG_NAME
    apiVersion: v1
    data:
      values: |
        thread_pool_size: 256
        
    kind: ConfigMap
    metadata:
      annotations:
        config.kubernetes.io/origin: |
          configuredIn: ../../../../../../bases/environments/stages/prod/hello_app_cluster/kustomization.yaml
          configuredBy:
            apiVersion: builtin
            kind: ConfigMapGenerator
          
      name: HELLO_APP_PROD_1-hello-world-user-config
    apiVersion: application.giantswarm.io/v1alpha1
    kind: App
    metadata:
      annotations:
        config.kubernetes.io/origin: |
          path: ../../../../../../bases/apps/hello-world/appcr.yaml
          
      labels:
        gitops.giantswarm.io/appSet: hello-web-app
      name: HELLO_APP_PROD_1-hello-world
      namespace: hello-web
    spec:
      catalog: giantswarm
      config:
        configMap:
          name: HELLO_APP_PROD_1-hello-world-values
          namespace: hello-world-app
      kubeConfig:
        inCluster: false
      name: hello-world-app
      namespace: hello-world-app
      version: 0.1.8
    apiVersion: application.giantswarm.io/v1alpha1
    kind: App
    metadata:
      annotations:
        config.kubernetes.io/origin: |
          path: ../../../../../../bases/apps/simple-db/appcr.yaml
          
      labels:
        gitops.giantswarm.io/appSet: hello-web-app
      name: HELLO_APP_PROD_1-simple-db
      namespace: hello-web
    spec:
      catalog: giantswarm
      config:
        configMap:
          name: HELLO_APP_PROD_1-simple-db-values
          namespace: simple-db-app
      kubeConfig:
        inCluster: false
      name: simple-db-app
      namespace: simple-db-app
      version: 0.1.0
    apiVersion: application.giantswarm.io/v1alpha1
    kind: App
    metadata:
      annotations:
        config.kubernetes.io/origin: |
          path: ../../../../../../bases/clusters/capo/template/cluster.yaml
          
      labels:
        app-operator.giantswarm.io/version: 0.0.0
        giantswarm.io/managed-by: flux
      name: HELLO_APP_PROD_1
      namespace: org-ORG_NAME
    spec:
      catalog: giantswarm
      config:
        configMap:
          name: HELLO_APP_PROD_1-config
          namespace: org-ORG_NAME
      kubeConfig:
        inCluster: true
      name: cluster-openstack
      namespace: org-ORG_NAME
      userConfig:
        configMap:
          name: HELLO_APP_PROD_1-user-config
          namespace: org-ORG_NAME
      version: 0.8.1
    apiVersion: application.giantswarm.io/v1alpha1
    kind: App
    metadata:
      annotations:
        config.kubernetes.io/origin: |
          path: ../../../../../../bases/clusters/capo/template/default_apps.yaml
          
      labels:
        app-operator.giantswarm.io/version: 0.0.0
        giantswarm.io/managed-by: flux
      name: HELLO_APP_PROD_1-default-apps
      namespace: org-ORG_NAME
    spec:
      catalog: giantswarm
      config:
        configMap:
          name: HELLO_APP_PROD_1-default-apps-config
          namespace: org-ORG_NAME
      kubeConfig:
        inCluster: true
      name: default-apps-openstack
      namespace: org-ORG_NAME
      version: "${default_apps_release:=0.1.0}"
    apiVersion: v1
    data:
      values: |
        admin_login: hello-admin-in-cluster-template
        db_config:
          db_port: 5433 # has to be in sync with config in default_config_simple_db.yaml
          db_user: hello-web-app # has to be in sync with config in default_config_simple_db.yaml
          host: simple-db.simple-web.svc # has to be in sync with 'namespace:' set in kustomization.yaml
        thread_pool_size: 64
        
    kind: ConfigMap
    metadata:
      annotations:
        config.kubernetes.io/origin: |
          configuredIn: ../../../../../../bases/cluster_templates/hello_app_cluster/app_sets/hello-web-app/kustomization.yaml
          configuredBy:
            apiVersion: builtin
            kind: ConfigMapGenerator
          
      labels:
        gitops.giantswarm.io/appSet: hello-web-app
      name: HELLO_APP_DEV_1-hello-world-values
      namespace: hello-web
    apiVersion: v1
    data:
      values: |
        port: 5433 # has to be in sync with config in default_config_hello_world.yaml
        user: hello-web-app # has to be in sync with config in default_config_hello_world.yaml
        
    kind: ConfigMap
    metadata:
      annotations:
        config.kubernetes.io/origin: |
          configuredIn: ../../../../../../bases/app_sets/hello-web-app/kustomization.yaml
          configuredBy:
            apiVersion: builtin
            kind: ConfigMapGenerator
          
      labels:
        gitops.giantswarm.io/appSet: hello-web-app
      name: HELLO_APP_DEV_1-simple-db-values
      namespace: hello-web
    apiVersion: v1
    data:
      values: |
        bastion:
          bootFromVolume: true
          diskSize: 10
          flavor: n1.tiny
          image: aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa
        cloudConfig: cloud-config
        cloudName: openstack
        clusterName: HELLO_APP_DEV_1
        controlPlane:
          availabilityZones:
          - demo_az_1
          - demo_az_2
          - demo_az_3
          bootFromVolume: true
          diskSize: 50
          flavor: n1.small
          image: bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb
          replicas: 3
        externalNetworkID: cccccccc-cccc-cccc-cccc-cccccccccccc
        kubernetesVersion: v1.20.9
        nodeCIDR: 10.6.0.0/24
        nodeClasses:
        - bootFromVolume: true
          diskSize: 50
          flavor: n1.small
          image: dddddddd-dddd-dddd-dddd-dddddddddddd
          name: default
        nodePools:
        - class: default
          failureDomain: demo_az_1
          name: demo_az_1
          replicas: 1
        - class: default
          failureDomain: demo_az_2
          name: demo_az_2
          replicas: 1
        - class: default
          failureDomain: demo_az_3
          name: demo_az_3
          replicas: 1
        organization: ORG_NAME
        
    kind: ConfigMap
    metadata:
      annotations:
        config.kubernetes.io/origin: |
          configuredIn: ../../../../../../bases/clusters/capo/>=v0.6.0/kustomization.yaml
          configuredBy:
            apiVersion: builtin
            kind: ConfigMapGenerator
          
      labels:
        giantswarm.io/managed-by: flux
      name: HELLO_APP_DEV_1-config
      namespace: org-ORG_NAME
    apiVersion: v1
    data:
      values: |
        clusterName: HELLO_APP_DEV_1
        organization: ORG_NAME
        
    kind: ConfigMap
    metadata:
      annotations:
        config.kubernetes.io/origin: |
          configuredIn: ../../../../../../bases/clusters/capo/template/kustomization.yaml
          configuredBy:
            apiVersion: builtin
            kind: ConfigMapGenerator
          
      labels:
        giantswarm.io/managed-by: flux
      name: HELLO_APP_DEV_1-default-apps-config
      namespace: org-ORG_NAME
    apiVersion: v1
    data:
      values: |
        cloudConfig: cloud-config-giantswarm-2
        cloudName: openstack
        externalNetworkID: aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee
        
    kind: ConfigMap
    metadata:
      annotations:
        config.kubernetes.io/origin: |
          path: ../../../../../../bases/cluster_templates/hello_app_cluster/cluster/cluster_user_config.yaml
          
      labels:
        giantswarm.io/managed-by: flux
      name: HELLO_APP_DEV_1-user-config
      namespace: org-ORG_NAME
    apiVersion: v1
    data:
      values: |
        thread_pool_size: 16
        
    kind: ConfigMap
    metadata:
      annotations:
        config.kubernetes.io/origin: |
          configuredIn: ../../../../../../bases/environments/stages/dev/hello_app_cluster/kustomization.yaml
          configuredBy:
            apiVersion: builtin
            kind: ConfigMapGenerator
          
      name: HELLO_APP_DEV_1-hello-world-user-config
    apiVersion: application.giantswarm.io/v1alpha1
    kind: App
    metadata:
      annotations:
        config.kubernetes.io/origin: |
          path: ../../../../../../bases/apps/hello-world/appcr.yaml
          
      labels:
        gitops.giantswarm.io/appSet: hello-web-app
      name: HELLO_APP_DEV_1-hello-world
      namespace: hello-web
    spec:
      catalog: giantswarm
      config:
        configMap:
          name: HELLO_APP_DEV_1-hello-world-values
          namespace: hello-world-app
      kubeConfig:
        inCluster: false
      name: hello-world-app
      namespace: hello-world-app
      version: "0.1.8 # {"": "org-ORG_NAME:HELLO_APP_DEV_1-hello-app"}"
    apiVersion: application.giantswarm.io/v1alpha1
    kind: App
    metadata:
      annotations:
        config.kubernetes.io/origin: |
          path: ../../../../../../bases/apps/simple-db/appcr.yaml
          
      labels:
        gitops.giantswarm.io/appSet: hello-web-app
      name: HELLO_APP_DEV_1-simple-db
      namespace: hello-web
    spec:
      catalog: giantswarm
      config:
        configMap:
          name: HELLO_APP_DEV_1-simple-db-values
          namespace: simple-db-app
      kubeConfig:
        inCluster: false
      name: simple-db-app
      namespace: simple-db-app
      version: "0.1.0 # {"": "org-ORG_NAME:HELLO_APP_DEV_1-simple-db-app"}"
    apiVersion: application.giantswarm.io/v1alpha1
    kind: App
    metadata:
      annotations:
        config.kubernetes.io/origin: |
          path: ../../../../../../bases/clusters/capo/template/cluster.yaml
          
      labels:
        app-operator.giantswarm.io/version: 0.0.0
        giantswarm.io/managed-by: flux
      name: HELLO_APP_DEV_1
      namespace: org-ORG_NAME
    spec:
      catalog: giantswarm
      config:
        configMap:
          name: HELLO_APP_DEV_1-config
          namespace: org-ORG_NAME
      kubeConfig:
        inCluster: true
      name: cluster-openstack
      namespace: org-ORG_NAME
      userConfig:
        configMap:
          name: HELLO_APP_DEV_1-user-config
          namespace: org-ORG_NAME
      version: 0.8.1
    apiVersion: application.giantswarm.io/v1alpha1
    kind: App
    metadata:
      annotations:
        config.kubernetes.io/origin: |
          path: ../../../../../../bases/clusters/capo/template/default_apps.yaml
          
      labels:
        app-operator.giantswarm.io/version: 0.0.0
        giantswarm.io/managed-by: flux
      name: HELLO_APP_DEV_1-default-apps
      namespace: org-ORG_NAME
    spec:
      catalog: giantswarm
      config:
        configMap:
          name: HELLO_APP_DEV_1-default-apps-config
          namespace: org-ORG_NAME
      kubeConfig:
        inCluster: true
      name: default-apps-openstack
      namespace: org-ORG_NAME
      version: "${default_apps_release:=0.1.0}"
    apiVersion: application.giantswarm.io/v1alpha1
    kind: Catalog
    metadata:
      annotations:
        config.kubernetes.io/origin: |
          path: ../../../../../../bases/environments/stages/dev/hello_app_cluster/automatic_updates/catalog.yaml
          
      labels:
        application.giantswarm.io/catalog-visibility: internal
        giantswarm.io/managed-by: flux
      name: giantswarm-catalog-oci
      namespace: org-ORG_NAME
    spec:
      description: giantswarm-catalog-oci
      logoURL: "https://avatars.githubusercontent.com/u/7556340?s=60&v=4"
      repositories:
      - URL: "oci://giantswarmpublic.azurecr.io/giantswarm-catalog/"
        type: helm
      storage:
        URL: "oci://giantswarmpublic.azurecr.io/giantswarm-catalog/"
        type: helm
      title: giantswarm-catalog-oci
    apiVersion: image.toolkit.fluxcd.io/v1beta1
    kind: ImagePolicy
    metadata:
      annotations:
        config.kubernetes.io/origin: |
          path: ../../../../../../bases/environments/stages/dev/hello_app_cluster/imagepolicies.yaml
          
      name: HELLO_APP_DEV_1-hello-app
    spec:
      filterTags:
        pattern: ".*-dev.*"
      imageRepositoryRef:
        name: HELLO_APP_DEV_1-hello-app
      policy:
        semver:
          range: >=0.1.0
    apiVersion: image.toolkit.fluxcd.io/v1beta1
    kind: ImagePolicy
    metadata:
      annotations:
        config.kubernetes.io/origin: |
          path: ../../../../../../bases/environments/stages/dev/hello_app_cluster/imagepolicies.yaml
          
      name: HELLO_APP_DEV_1-simple-db-app
    spec:
      imageRepositoryRef:
        name: HELLO_APP_DEV_1-simple-db
      policy:
        semver:
          range: ">=0.1.0 <0.2.0"
    apiVersion: image.toolkit.fluxcd.io/v1beta1
    kind: ImageRepository
    metadata:
      annotations:
        config.kubernetes.io/origin: |
          path: ../../../../../../bases/environments/stages/dev/hello_app_cluster/imagerepositories.yaml
          
      name: HELLO_APP_DEV_1-hello-app
    spec:
      image: giantswarmpublic.azurecr.io/giantswarm-catalog/hello-world-app
      interval: 10m0s
    apiVersion: image.toolkit.fluxcd.io/v1beta1
    kind: ImageRepository
    metadata:
      annotations:
        config.kubernetes.io/origin: |
          path: ../../../../../../bases/environments/stages/dev/hello_app_cluster/imagerepositories.yaml
          
      name: HELLO_APP_DEV_1-simple-db-app
    spec:
      image: giantswarmpublic.azurecr.io/giantswarm-catalog/simple-db-app
      interval: 10m0s
    apiVersion: image.toolkit.fluxcd.io/v1beta1
    kind: ImageUpdateAutomation
    metadata:
      annotations:
        config.kubernetes.io/origin: |
          path: ../../../../../../bases/environments/stages/dev/hello_app_cluster/automatic_updates/imageupdate.yaml
          
      labels:
        giantswarm.io/managed-by: flux
      name: HELLO_APP_DEV_1-image-updates
      namespace: org-ORG_NAME
    spec:
      git:
        checkout:
          ref:
            branch: main
        commit:
          author:
            email: fluxcdbot@users.noreply.github.com
            name: fluxcdbot
          messageTemplate: |
            automated app upgrades:
            {{ range , /usr/bin/envsubst := .Updated.Images -}}
            - {{ .Repository }} to {{ .Identifier }}
            {{ end -}}
            
        push:
          branch: main
      interval: 1m0s
      sourceRef:
        kind: GitRepository
        name: YOUR_GIT_REPO
      update:
        path: ./management-clusters/MC_NAME
        strategy: Setters

github-actions · 2022-06-29T09:14:10Z

Validation output log

Testing: MC_NAME ORG_NAME WC_NAME_NO_OUT_OF_BAND apps
yamllint: OK
kubeconform: OK

Testing: MC_NAME ORG_NAME WC_NAME_NO_OUT_OF_BAND automatic-updates
yamllint: OK
kubeconform: OK

Testing: MC_NAME ORG_NAME WC_NAME_NO_OUT_OF_BAND cluster
yamllint: OK
kubeconform: OK

Testing: MC_NAME ORG_NAME WC_NAME_NO_OUT_OF_BAND app_sets
yamllint: OK
kubeconform: OK

Testing: MC_NAME ORG_NAME FROM_TEMPLATE_WC_NAME /
yamllint: OK
kubeconform: OK

Testing: MC_NAME ORG_NAME HELLO_APP_STAGING_CLUSTER_1 /
yamllint: OK
kubeconform: OK

Testing: MC_NAME ORG_NAME HELLO_APP_PROD_CLUSTER_1 /
yamllint: OK
kubeconform: OK

Testing: MC_NAME ORG_NAME WC_NAME_OUT_OF_BAND_NO_FLUX_APP mapi
yamllint: OK
kubeconform: OK

Testing: MC_NAME ORG_NAME WC_NAME_OUT_OF_BAND_NO_FLUX_APP out-of-band
yamllint: OK
kubeconform: OK

Testing: MC_NAME ORG_NAME HELLO_APP_DEV_CLUSTER_1 /
yamllint: OK
kubeconform: OK

Testing: MC_NAME ORG_NAME WC_NAME_OUT_OF_BAND_FLUX_APP mapi
yamllint: OK
kubeconform: OK

Testing: MC_NAME ORG_NAME WC_NAME_OUT_OF_BAND_FLUX_APP out-of-band
yamllint: OK
kubeconform: OK

Testing: MC_NAME ORG_NAME CAPI_WC_NAME apps
yamllint: OK
kubeconform: OK

Testing: MC_NAME ORG_NAME CAPI_WC_NAME cluster
yamllint: OK
kubeconform: OK

piontec added 6 commits June 2, 2022 15:38

add: prod base

7f6b387

wip: prod cluster

ec5d7e4

move prod clsuter template to app specific folder

58712c2

staging cluster

9df3628

add: dev environment

45d9020

fix: unfortunately, both cluster_id and cluster_name are needed

f053324

piontec requested a review from a team June 6, 2022 13:34

piontec self-assigned this Jun 6, 2022

MarcelMue reviewed Jun 6, 2022

View reviewed changes

piontec and others added 14 commits June 7, 2022 15:36

Merge branch 'main' into feature/environments

0f5c4e1

Merge branch 'main' into feature/environments

116e862

Fix typos

3a27e9f

Add base for environment workload cluster docs

39e0ba4

Merge branch 'main' into feature/environments

f8ec529

renamed 'deployments' to 'stages'

b07b866

Add docs on environment cluster bases

342a2fe

Merge branch 'feature/environments' of github.com:giantswarm/gitops-t…

9f01e80

…emplate into feature/environments

Add docs on setting up workload clusters based on environment bases

662d9c3

Add table of content to environment cluster bases and some tips devel…

5f169a5

…oping them

Add tools used by test-all-ffb script to Appendices

5982c30

add note about problems when using multiple bases

07fe15a

set cluster names in YAML to same values as suggested by file names

e60cc87

Format note on merging yamls and fix pre-commit hook warnings

e3c1047

piontec commented Jun 20, 2022

View reviewed changes

uvegla and others added 6 commits June 20, 2022 16:32

Update docs/add_wc_environments.md

71abd1c

Co-authored-by: Lukasz Piatkowski <piontec@gmail.com>

Update docs/add_wc_environments.md

c5d7f88

Co-authored-by: Lukasz Piatkowski <piontec@gmail.com>

Update docs/add_wc_environments.md

f5d58eb

Co-authored-by: Lukasz Piatkowski <piontec@gmail.com>

Update docs/add_wc_environments.md

897e87c

Co-authored-by: Lukasz Piatkowski <piontec@gmail.com>

Update docs/add_wc_environments.md

d66aa9a

Co-authored-by: Lukasz Piatkowski <piontec@gmail.com>

Correct title based on code review and fix pre-commit warnings

8139c20

Replace kubeval with kubeconform

6499cef

The kubeval tool does not seem to be maintained anymore.

uvegla force-pushed the feature/environments branch from d94fc97 to 6499cef Compare June 20, 2022 15:44

uvegla added 3 commits June 20, 2022 17:51

Make kubeconform check output similar to yamllint check output

bc8260b

Format warning about multiple layer merges for environments

a797827

Merge branch 'main' into feature/environments

8688694

piontec marked this pull request as ready for review June 21, 2022 09:32

improve environments docs

5c0a9fd

MarcelMue reviewed Jun 22, 2022

View reviewed changes

docs/add_wc_environments.md Outdated Show resolved Hide resolved

docs/add_wc_environments.md Outdated Show resolved Hide resolved

docs/add_wc_environments.md Outdated Show resolved Hide resolved

piontec and others added 4 commits June 22, 2022 17:16

Update docs/add_wc_environments.md

f91e411

Co-authored-by: Bricktop <marcel.mueller1@rwth-aachen.de>

Update docs/add_wc_environments.md

267a70e

Co-authored-by: Bricktop <marcel.mueller1@rwth-aachen.de>

Update docs/add_wc_environments.md

1348830

Co-authored-by: Bricktop <marcel.mueller1@rwth-aachen.de>

Merge branch 'main' into feature/environments

a0343bc

MarcelMue reviewed Jun 23, 2022

View reviewed changes

uvegla added 4 commits June 28, 2022 13:26

Fix some typos

305feb0

Add example commands to recreate the cluster environment templates

3dd6596

Add example commands to create and instance of an envrionment cluster…

6f773fc

… template

Fix kubeconform validation errors

0fb3745

Merge branch 'main' into feature/environments

89c73a0

uvegla approved these changes Jun 29, 2022

View reviewed changes

added changelog

31fffe9

piontec merged commit 77bfee8 into main Jun 29, 2022

piontec deleted the feature/environments branch June 29, 2022 09:15

gianfranco-l mentioned this pull request Jul 13, 2022

Epic: Promotion of releases between environments for GitOps giantswarm/roadmap#830

Closed

6 tasks


		### Stages

		We have 3 example clusters under [/bases/environments/stages](/bases/environments/stages):

Conversation

piontec commented Jun 6, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

uvegla commented Jun 20, 2022

Uh oh!

uvegla commented Jun 20, 2022

Uh oh!

uvegla commented Jun 20, 2022

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

uvegla Jun 28, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

uvegla commented Jun 28, 2022

Uh oh!

piontec commented Jun 29, 2022

Uh oh!

github-actions bot commented Jun 29, 2022

Uh oh!

github-actions bot commented Jun 29, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

piontec commented Jun 6, 2022 •

edited

Loading

uvegla Jun 28, 2022 •

edited

Loading