Skip to content

Deploy YAML/JSON Kubernetes manifests

ClusterProfile policyRefs Reference

The ClusterProfile spec.policyRefs is a list of Secrets/ConfigMaps. Both Secrets and ConfigMaps data fields can be a list of key-value pairs. Any key is acceptable, and the value can be multiple objects in YAML or JSON format1.

Example: Create a Secret

To create a Kubernetes Secret that contains the Calico YAMLs and make it usable with Sveltos, utilise the below commands.

$ wget https://raw.githubusercontent.com/projectcalico/calico/master/manifests/calico.yaml

$ kubectl create secret generic calico --from-file=calico.yaml --type=addons.projectsveltos.io/cluster-profile

The commands will download the calico.yaml manifest file and afterwards create a Kubernetes secret of type generic by defining the file downloaded in the previous command plus defining the needed type=addons.projectsveltos.io/cluster-profile.

Note

A ClusterProfile can only reference Secrets of type addons.projectsveltos.io/cluster-profile

Example: Create a ConfigMap

The YAML definition below exemplifies a ConfigMap that holds multiple resources2. When a ClusterProfile instance references the ConfigMap, a Namespace and a Deployment instance are automatically deployed in any managed cluster that adheres to the ClusterProfile clusterSelector.

Example - Resources Definition

cat > nginx_cm.yaml <<EOF
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: nginx
  namespace: default
data:
  namespace.yaml: |
    kind: Namespace
    apiVersion: v1
    metadata:
      name: nginx
  deployment.yaml: |
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: nginx-deployment
      namespace: nginx  
    spec:
      replicas: 2 # number of pods to run
      selector:
        matchLabels:
          app: nginx
      template:
        metadata:
          labels:
            app: nginx
        spec:
          containers:
          - name: nginx
            image: nginx:latest # public image from Docker Hub
            ports:
            - containerPort: 80
EOF

Once the required Kubernetes resources are created/deployed, the below example represents a ClusterProfile resource that references the ConfigMap and the Secret created above.

Example - ClusterProfile Definition with Reference

cat > clusterprofile_deploy_nginx.yaml <<EOF
---
apiVersion: config.projectsveltos.io/v1beta1
kind: ClusterProfile
metadata:
  name: deploy-resources
spec:
  clusterSelector:
    matchLabels:      
      env: fv
  policyRefs:
  - name: nginx
    namespace: default
    kind: ConfigMap
  - name: calico
    namespace: default
    kind: Secret
EOF

Note

The namespace definition refers to the namespace where the ConfigMap, and the Secret were created in the management cluster. In our example, both resources created in the default namespace.

When a ClusterProfile references a ConfigMap or a Secret, the kind and name fields are required, while the namespace field is optional. Specifying a namespace uniquely identifies the resource using the tuple namespace, name, and kind, and that resource will be used for all matching clusters.

If you leave the namespace field empty, Sveltos will search for the ConfigMap or the Secret with the provided name within the namespace of each matching cluster.

Example: Understand Namespace Definition

---
apiVersion: config.projectsveltos.io/v1beta1
kind: ClusterProfile
metadata:
  name: deploy-resources
spec:
  clusterSelector:
    matchLabels:
      env: fv
  policyRefs:
  - name: nginx
    kind: ConfigMap

Consider the provided ClusterProfile, when we have two workload clusters matching. One in the foo namespace and another in the bar namespace. Sveltos will search for the ConfigMap nginx in the foo namespace for the Cluster in the foo namespace and for a ConfigMap ngix in the bar namespace for the Cluster in the bar namespace.

More ClusterProfile examples can be found here.

Template-based Referencing for ConfigMaps and Secrets

We can express ConfigMap and Secret names as templates and dynamically generate them using cluster information. This allows for easier management and reduces redundancy.

Available cluster information :

  • cluster namespace: use .Cluster.metadata.namespace
  • cluster name: .Cluster.metadata.name
  • cluster type: .Cluster.kind

Consider two SveltosCluster instances in the civo namespace:

kubectl get sveltoscluster -n civo --show-labels
NAME             READY   VERSION        LABELS
pre-production   true    v1.29.2+k3s1   env=civo,projectsveltos.io/k8s-version=v1.29.2
production       true    v1.28.7+k3s1   env=civo,projectsveltos.io/k8s-version=v1.28.7

Additionally, there are two ConfigMaps named nginx-pre-production and nginx-production (both containing a nginx deployment) within the civo namespace:

kubectl get configmap -n civo                   
NAME                   DATA   AGE
nginx-pre-production   2      4m59s
nginx-production       2      4m41s

The only difference between these ConfigMaps is the replicas setting: 1 for pre-production and 3 for production.

Following ClusterProfile:

  1. Matches both SveltosClusters
  2. Dynamic ConfigMap Selection:
    • For the pre-production cluster, the profile should use the nginx-pre-production ConfigMap.
    • For the production cluster, the profile should use the nginx-production ConfigMap.

apiVersion: config.projectsveltos.io/v1beta1
kind: ClusterProfile
metadata:
  name: deploy-nginx
spec:
  clusterSelector:
    matchLabels:
      env: civo
  policyRefs:
  - name: nginx-{{ .Cluster.metadata.name }}
    kind: ConfigMap
This approach provides a flexible and centralized way to reference ConfigMaps and Secrets based on cluster information.

Template

Define the content for resources in your PolicyRefs using templates. During deployment, Sveltos will automatically populate these templates with relevant information from your cluster and other resources in the management cluster. See the template section template section for details.

Remember to adapt the provided resources to your specific repository structure, cluster configuration, and desired templating logic.

Subresources

Sveltos can be configured to update specific subresources of a resource. This is achieved by leveraging the projectsveltos.io/subresources annotation. When this annotation is present on a resource referenced in the PolicyRefs section, Sveltos will update the designated subresources alongside the main resource.

Subresources are specified as a comma-separated list within the annotation value.

For example, to instruct Sveltos to update the status subresource of a Service, you can create a ConfigMap with the following structure and reference this ConfigMap from a ClusterProfile/Profile:

apiVersion: v1
data:
  service.yaml: |
    apiVersion: v1
    kind: Service
    metadata:
      name: sveltos-subresource
      namespace: default
    spec:
      selector:
        app: foo
      ports:
      - name: my-port
        port: 443
        protocol: TCP
        targetPort: 1032
      type: LoadBalancer
    status:
      loadBalancer:
        ingress:
        - ip: 1.1.1.1
kind: ConfigMap
metadata:
  annotations:
    projectsveltos.io/subresources: status
  name: load-balancer-service
  namespace: default

  1. A ConfigMap is not designed to hold large chunks of data. The data stored in a ConfigMap cannot exceed 1 MiB. If you need to store settings that are larger than this limit, you may want to consider mounting a volume or use a separate database or file service. ↩

  2. Another way to create a Kubernetes ConfigMap resource is with the imperative approach. The below command will create the same ConfigMap resource in the management cluster. 

    $ kubectl create configmap nginx --from-file=namespace.yaml --from-file=deployment.yaml
     ↩