# Kubernetes-Native CI/CD With Tekton and ArgoCD ## Introduction Having learned about the [DigitalOcean Kubernetes Challenge](https://www.digitalocean.com/community/pages/kubernetes-challenge), I decided to take it on to upskill myself on trending DevOps tools for Kubernetes, and to evaluate DigitalOcean's container services. For the challenge, I implemented a Kubernetes-native CI/CD solution using Tekton build and ArgoCD for deployment. In this article, I will share my experience and sample code I've put together, so you too can try it out. ## About the sample code You can find the sample code for the challenge at https://github.com/acwwat/do-k8s-gitops. The folder structure is as follows: * The `app` folder contains the source code for the sample to-do list application that we will build into Docker images and deployed to Kubernetes. The frontend `todo-list-frontend` is a basic Vue.js application and the backend `todo-list-backend` is a basic Java Spring Boot application with an embedded H2 database. * The `config` folder contains the starter Kubernetes YAML file template which we will use to deploy the application to the Kubernetes cluster. * The `tekton` folder contains the Tekton Kubernetes YAML files that defines the various components that form the CI pipeline. Clone or download this GitHub repository as we will need it to set up the CI/CD solution. ## Creating the application and config repositories You need to create two repositories - one for the application to drive CI and the other for the Kubernetes configuration to drive CD. I used GitHub myself, but you can also use other Git repositories so long as they are accessible by Tekton and ArgoCD. For the application repository, check in the content of the `app` folder from the sample code repository. For the configuration repository, check in the content of the `config` folder. Since the Kubernetes YAML file requires a hardcoded container image path from your DigitalOcean Container Registry, you need to copy `todo-list/todo-list.yaml.template` as `todo-list/todo-list.yaml` and replace the URL of your container registry once you have created it. ## Creating access tokens for integration You need to create the following access tokens for integration purposes: * A [GitHub personal access token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token) for the Tekton pipeline to update the Kubernetes YAML files in the configuration repository. The PAT must have all scopes under the **repo** category selected. * A [DigitalOcean personal access token](https://docs.digitalocean.com/reference/api/create-personal-access-token/) for the Tekton pipeline to push the application Docker images to the DigitalOcean Container Registry. The PAT must have the **write** scope selected. Copy the tokens and set them aside for now, as you will later store them as Kubernetes secrets. ## Installing CLI tools To keep things simple, I chose to favor GUIs over CLI tools for this challenge. However you still need to install a few CLI tools as follows: * [kubectl](https://kubernetes.io/docs/tasks/tools/) * [doctl Command Line Interface (CLI)](https://docs.digitalocean.com/reference/doctl/) ## Creating the Kubernetes cluster in DigitalOcean Setting up a Kubernetes cluster can be very involved, but DigitalOcean has made it simple. Once you have your account and logged into the dashboard, [create a project](https://docs.digitalocean.com/products/projects/how-to/create/) and then [create a Kubernetes cluster](https://docs.digitalocean.com/products/kubernetes/how-to/create-clusters/). For the challenge, I used the following settings: ![Create cluster](https://cdn.hashnode.com/res/hashnode/image/upload/v1640893248351/aT8mrO4Nr.png align="left") After the Kubernetes cluster is provisioned after several minutes, install the NGINX Ingress Controller as a 1-Click App on the cluster details page: ![Deploy NGINX Ingress Controller](https://cdn.hashnode.com/res/hashnode/image/upload/v1640893560831/KR4SYpfWC.png align="left") An ingress controller is not a must, however I'd like to evaluate it on DigitalOcean for practical purposes. Allow it a few minutes to deploy. When it is done, run the `doctl` command from the cluster details page to connect `kubectl` to your cluster: ![Connect kubectl to the cluster](https://cdn.hashnode.com/res/hashnode/image/upload/v1640913903308/993J8MHa2.png align="left") ## Creating the Kubernetes Container Registry You need a container registry to store the application container images, so this is our chance to also evaluate the DigitalOcean Container Registry. In the DigitalOcean Dashboard, [create a registry](https://docs.digitalocean.com/products/container-registry/quickstart/) with the **Basic** subscription plan, as we require two repositories (one per Docker image) in the registry. When the registry is created, enable integration to your Kubernetes cluster in the settings. Don't forget to create the Kubernetes YAML file in the configuration Git repository as mentioned earlier! ## Installing Tekton [Tekton](https://tekton.dev/) is an open-source, Kubernetes-native CI/CD framework. Tekton itself is a Kubernetes application and it provides custom resources as building blocks for pipelines. It took me a while to get past the finer granularity vs. GitHub Actions and Azure Pipelines, but I also appreciate its flexibility and tight integration with Kubernetes. In this challenge, I chose to implement CD using ArgoCD, however using Tekton for CD is also perfectly doable. Installing Tekton is relatively simple. Follow the [Getting Started](https://tekton.dev/docs/getting-started/) guide to install the core components and set up the ConfigMaps for persistent volumes. You can also install the Tekton CLI and run the sample task if you like, but we won't use the CLI in this exercise. By default, Tekton uses the `default` service account in the cluster to run pipelines. While it would be a security best practice to create a separate service account for CI/CD, we will just keep the default for now. As well, [install the Tekton Dashboard](https://tekton.dev/docs/dashboard/) which we will use to view pipeline progress. To not expose the Tekton Dashboard to the internet, we can use the port forwarding option to map a local port to the dashboard HTTP port. Open a new command prompt and run the following command: ```plaintext kubectl --namespace tekton-pipelines port-forward svc/tekton-dashboard 9097:9097 ``` Then open `http://localhost:9097` in a web browser and verify that the dashboard is accessible. ![Verify Tekton installation](https://cdn.hashnode.com/res/hashnode/image/upload/v1640897949454/5siku8x9i.png align="left") ## Installing ArgoCD Now let's switch gear to ArgoCD. [ArgoCD](https://argoproj.github.io/cd/) is a declarative, GitOps CD tool for Kubernetes. Its concept is simple - detect changes between current state in the Kubernetes cluster and desired state per configuration in Git, and apply them either on-demand or automatically. With configurations in Git, various DevOps best practices such as version control, code review, and CD are possible. Installing ArgoCD is also very simple - just follow the [Getting Started](https://argo-cd.readthedocs.io/en/stable/getting_started/) guide. We won't need the CLI for the exercise, but feel free to install it when going through the guide. Once installation completes, you can forward the server port to access the ArgoCD UI without exposing it to the internet. Open a new command prompt and run the following command: ```bash kubectl port-forward svc/argocd-server -n argocd 8080:443 ``` Then open `http://localhost:8080` in a web browser and log in as `admin` using the initial password from the secret (or the updated password if changed via the CLI), then verify that the dashboard is accessible. ![Verify ArgoCD installation](https://cdn.hashnode.com/res/hashnode/image/upload/v1640906891360/kcT0QXutl.png align="left") ## Deploying the Tekton CI pipeline Now we are ready to deploy the Tekton CI pipeline for the sample application. Before you start, take a look at the [Concepts](https://tekton.dev/docs/concepts/) page in the Tekton documentation to understand what the building blocks are and how they interact with one another. But in essence, we need to know the following: * A **pipeline** consists of **tasks** to perform the required build operations. * A **task** consists of **steps** to encapsulate a build operation, such as cloning a Git repository or updating Kubernetes YAML files in the configuration Git repository. * Tekton provides a [catalog of shared tasks and pipelines](https://github.com/tektoncd/catalog) that can be used out of the box. * A **step** is an operation run in a container in the Kubernetes cluster. * A **volume-based workspace** is used by tasks and steps to share data across the pipeline Coming from a traditional CI/CD world of Jenkins and such, running a step in a container seems awfully inefficient to me. However it is not the case, since containers spin up relatively quickly and Tekton handles the lifecycle anyway. The CI pipeline for our sample to-do list application consists of the following tasks: 1. Use the [git-clone task](https://github.com/tektoncd/catalog/tree/main/task/git-clone/0.5) from the Tekton catalog to clone the application repository into a (shared) workspace. 2. Use the [kaniko task](https://github.com/tektoncd/catalog/tree/main/task/kaniko/0.5) from the Tekton catalog to build both `todo-list-frontend` and `todo-list-backend` using the provided Dockerfiles, and push the resulting images to the DigitalOcean Container Registry. [Kaniko](https://github.com/GoogleContainerTools/kaniko), by the way, is a tool to build container images inside a Kubernetes cluster. 3. Use a custom task we put together to clone the configuration repository, update the container image version in the Kubernetes YAML file, and commit the changes to Git (for ArgoCD to pick up). Let's now set up the Tekton pipeline for the sample application. ### Setting up the prerequisites Perform the following steps to provision everything the pipeline needs in the Kubernetes cluster. There are many steps, so hang in there! 1. Create a namespace called `todo-list-ci` to host the Tekton pipeline resources with the following command: ```bash kubectl create namespace todo-list-ci ``` 2. Create a secret that the custom task uses to commit the updated Kubernetes YAML file to the configuration repository on GitHub. Since the step uses `git` command with no user interaction, I decided to use the `GIT_ASKPASS` environment variable to provide the PAT after reading [this article](https://coolaj86.com/articles/vanilla-devops-git-credentials-cheatsheet/) as a quick and dirty solution. Create a new YAML file using the following template (see also `tekton/config-git-askpass.yaml.template` in the sample code repository) and your GitHub PAT previously created: ```yaml apiVersion: v1 kind: Secret metadata: name: config-git-askpass type: Opaque stringData: .git-askpass: | echo "### Your GitHub PAT goes here ###" ``` Then create the secret with the following command: ```bash kubectl apply -f config-git-askpass.yaml -n todo-list-ci ``` 3. Create a secret that provides the `kaniko` task a Docker config to push the application container images to our DigitalOcean Container Registry. Note that the `kaniko` task expects the file name to be `config.json`, which is different from the default file names that Kubernetes expects. In any case, create a new YAML file using the following template (see also `tekton/do-docker-config.yaml.template` in the sample code repository) and your DigitalOcean PAT previously created: ```yaml apiVersion: v1 kind: Secret metadata: name: do-docker-config type: kubernetes.io/dockercfgjson data: config.json: | ### Your DigitalOcean PAT goes here ### ``` Then create the secret with the following command: ```bash kubectl apply -f do-docker-config.yaml -n todo-list-ci ``` 4. Install the `git-clone` and `kaniko` tasks from the [Tekton catalog](https://github.com/tektoncd/catalog) with the following commands: ```bash kubectl apply -f https://raw.githubusercontent.com/tektoncd/catalog/main/task/git-clone/0.5/git-clone.yaml -n todo-list-ci kubectl apply -f https://raw.githubusercontent.com/tektoncd/catalog/main/task/kaniko/0.5/kaniko.yaml -n todo-list-ci ``` 5. The pipeline requires a shared workspace to store the source code and configuration from Git. In the `PipelineRun` configuration, we need to provide a persistent volume claim (PVC) to provision the workspace. The YAML file is as follows (see also `tekton/todo-list-ci-pvc.yaml` in the sample code repository): ```yaml apiVersion: v1 kind: PersistentVolumeClaim metadata: name: todo-list-ci-pvc spec: accessModes: - ReadWriteOnce resources: requests: storage: 5Gi ``` To create the PVC, run the following command: ```bash kubectl apply -f todo-list-ci-pvc.yaml -n todo-list-ci ``` ### Installing the custom task resource Since there is no readily available task to update the Kubernetes YAML file with new image versions, I had to create a custom task. Drawing inspiration from [Sebastian Daschner's example](https://github.com/sdaschner/tekton-argocd-example/blob/main/pipeline/task-deploy.yaml), the resulting task resource is defined as follows (see also `tekton/update-config-task.yaml` in the sample code repository): ```yaml apiVersion: tekton.dev/v1beta1 kind: Task metadata: name: update-config spec: params: - name: buildVersion - name: gitUrl - name: gitBranch - name: k8sYaml - name: gitUserName - name: gitUserEmail workspaces: - name: git-source - name: git-askpass-secret steps: - name: git-checkout image: alpine/git:v2.26.2 workingDir: "$(workspaces.git-source.path)" script: | #!/usr/bin/env sh set -e cp $(workspaces.git-askpass-secret.path)/.git-askpass ~/ chmod +x ~/.git-askpass export GIT_ASKPASS=~/.git-askpass rm -rf config git clone -b $(inputs.params.gitBranch) $(inputs.params.gitUrl) config - name: update-yaml image: alpine/git:v2.26.2 workingDir: "$(workspaces.git-source.path)" script: | #!/usr/bin/env sh set -e cd config sed -i "s#/todo-list-frontend:[a-zA-Z0-9.]\\+#/todo-list-frontend:$(inputs.params.buildVersion)#" $(inputs.params.k8sYaml) sed -i "s#/todo-list-backend:[a-zA-Z0-9.]\\+#/todo-list-backend:$(inputs.params.buildVersion)#" $(inputs.params.k8sYaml) cat $(inputs.params.k8sYaml) - name: commit-push-changes-gitops image: alpine/git:v2.26.2 workingDir: "$(workspaces.git-source.path)" script: | #!/usr/bin/env sh set -e cd config cp $(workspaces.git-askpass-secret.path)/.git-askpass ~/ chmod +x ~/.git-askpass export GIT_ASKPASS=~/.git-askpass git config --global user.email "$(inputs.params.gitUserEmail)" git config --global user.name "$(inputs.params.gitUserName)" git add . git commit --allow-empty -m "[tekton] Set deployment to version $(inputs.params.buildVersion)" git push origin $(inputs.params.gitBranch) ``` As you can see, there is a common workspace used by all three steps, each of which runs a different container and script. The task is also using the secret that contains the GitHub PAT as a file that the `GIT_ASKPASS` environment variable refers to. As for the logic, the task first clones the configuration repository on GitHub, then uses `sed` to update the container image version in `todo-list.yaml`, and finally commits/pushes the change into Git. Now that you understand how this custom task works, create it in your Kubernetes cluster with the following command: ```bash kubectl apply -f update-config-task.yaml -n todo-list-ci ``` ### Installing the pipeline resource Lastly, we need to install the pipeline resource. Refer to the definition below (see also `tekton/todo-list-ci-pipeline.yaml` in the sample code repository): ```yaml apiVersion: tekton.dev/v1beta1 kind: Pipeline metadata: name: todo-list-ci-pipeline spec: workspaces: - name: git-source - name: do-docker-config-secret - name: config-git-askpass-secret params: - name: buildVersion - name: appGitUrl - name: appGitBranch default: main - name: frontendPathToContext - name: frontendImageUrl - name: backendPathToContext - name: backendImageUrl - name: configGitUrl - name: configGitBranch default: main - name: k8sYaml - name: gitUserName - name: gitUserEmail tasks: - name: clone-repo taskRef: name: git-clone workspaces: - name: output workspace: git-source params: - name: url value: "$(params.appGitUrl)" - name: revision value: "$(params.appGitBranch)" - name: subdirectory value: "app" - name: deleteExisting value: "true" - name: frontend-build-image taskRef: name: kaniko runAfter: - clone-repo workspaces: - name: source workspace: git-source - name: dockerconfig workspace: docker-config-secret params: - name: CONTEXT value: app/$(params.frontendPathToContext) - name: IMAGE value: $(params.frontendImageUrl):$(params.buildVersion) - name: backend-build-image taskRef: name: kaniko runAfter: - clone-repo workspaces: - name: source workspace: git-source - name: dockerconfig workspace: do-docker-config-secret params: - name: CONTEXT value: app/$(params.backendPathToContext) - name: IMAGE value: $(params.backendImageUrl):$(params.buildVersion) - name: update-config taskRef: name: update-config runAfter: - frontend-build-image - backend-build-image workspaces: - name: git-source workspace: git-source - name: git-askpass-secret workspace: config-git-askpass-secret params: - name: buildVersion value: "$(params.buildVersion)" - name: gitUrl value: "$(params.configGitUrl)" - name: gitBranch value: "$(params.configGitBranch)" - name: k8sYaml value: "$(params.k8sYaml)" - name: gitUserName value: "$(params.gitUserName)" - name: gitUserEmail value: "$(params.gitUserEmail)" ``` The definition is a tad long, but it performs the tasks as explained earlier. Let's create it with the following command: ```bash kubectl apply -f todo-list-ci-pipeline.yaml -n todo-list-ci ``` Now that our Tekton CI pipeline is ready to go, feel free to take a well-deserved break and digest all the information we've gone through so far. ## Create an application in ArgoCD At last, we need to create the CD pipeline to complete our solution. Luckily this is very simple to do in the ArgoCD UI. You could use a [declarative setup](https://argo-cd.readthedocs.io/en/stable/operator-manual/declarative-setup/) to see full benefits, but let's take it easy for now. Follow the steps below to create the application: 1. Log in to the ArgoCD UI. 2. Click the **New App** button. ![Create new app](https://cdn.hashnode.com/res/hashnode/image/upload/v1640907062710/rhCf772F2.png align="left") 3. In the **General** section, enter an application name and select the `default` project. ![Provide general information for app creation](https://cdn.hashnode.com/res/hashnode/image/upload/v1640907286260/RgVmkKl4q.png align="left") 4. Scroll down to the **Source** section and enter your configuration repository URL and `todo-list` for the path. ![Provide source information for app creation](https://cdn.hashnode.com/res/hashnode/image/upload/v1640907869263/9NpeXlgEU.png align="left") 5. Scroll down to the **Destination** section and select `https://kubernetes.default.svc` as the cluster URL (this is the value for deploying to the same cluster in which ArgoCD is running) and enter `todo-list` as the namespace. Click the **Create** button to create the app. ![Provide destination information for app creation](https://cdn.hashnode.com/res/hashnode/image/upload/v1640907786130/hQ3i8RmYB.png align="left") The todo-list app is now created with the **missing** and **OutOfSync** status: ![New app status](https://cdn.hashnode.com/res/hashnode/image/upload/v1640908097553/xNaSNsxhF.png align="left") Don't sync it just yet because we haven't deployed the container images to the container registry! We will get the container images ready by running the CI pipeline. ## Running the Tekton CI pipeline There are two ways to run a pipeline in Tekton: 1. Creating a [PipelineRun](https://tekton.dev/docs/pipelines/pipelineruns/) Kubernetes custom resource 2. Configuring a [Tekton Trigger](https://tekton.dev/docs/triggers/) to automatically start pipeline runs Triggers can be overwhelming for beginners, so we will just manually create a PipelineRun resource to start the pipeline for this exercise. Create a new YAML file using the following template (see also `tekton/todo-list-ci-pipelinerun.yaml.template` in the sample code repository) and replace the values with ones specific to your environment: ```yaml apiVersion: tekton.dev/v1beta1 kind: PipelineRun metadata: generateName: todo-list-ci- spec: pipelineRef: name: todo-list-ci-pipeline params: - name: buildVersion value: "0.0.1" - name: appGitUrl value: ### Your app Git repo URL goes here ### - name: appGitBranch value: ### Your app Git branch name goes here (typically main or master) ### - name: frontendPathToContext value: "todo-list-frontend" - name: frontendImageUrl value: "### Your DigitalOcean Container Registry URL goes here ###/todo-list-frontend" - name: backendPathToContext value: "todo-list-backend" - name: backendImageUrl value: ""### Your DigitalOcean Container Registry URL goes here ###/todo-list-backend" - name: configGitUrl value: ### Your config Git repo URL goes here ### - name: configGitBranch value: ### Your config Git branch name goes here (typically main or master) ### - name: k8sYaml value: "todo-list/todo-list.yaml" serviceAccountName: default workspaces: - name: git-source persistentVolumeClaim: claimName: todo-list-ci-pvc - name: do-docker-config-secret secret: secretName: do-docker-config - name: config-git-askpass-secret secret: secretName: config-git-askpass ``` To start a new pipeline run, run the following command: ```bash kubectl create -f todo-list-ci-pipelinerun.yaml -n todo-list-ci ``` The command should complete with the PipelineRun ID similar to the following: ```plaintext pipelinerun.tekton.dev/todo-list-ci-r2f94 created ``` Now, head over to the Tekton Dashboard in a web browser. Select **PipelineRuns** from the left menu and you will see that our pipeline is running! ![Pipeline run status](https://cdn.hashnode.com/res/hashnode/image/upload/v1640908907482/LQGSxcRgz.png align="left") Click the PipelineRun name and you will see the progress. In the screenshot below, I opened the `build-and-push` step under the `frontend-build-image` task, so we can see the progress of Kaniko building the image for the frontend application. ![Pipeline run details](https://cdn.hashnode.com/res/hashnode/image/upload/v1640909047859/hGnyZbQ7n.png align="left") When this is done, verify in your configuration GitHub repository that a commit has been made to update the container image versions to 0.0.1. Next we can verify the deployment in ArgoCD! ## Synchronizing the application in ArgoCD for the first time Head over to the ArgoCD UI in the web browser. Then follow these steps to manually synchronize the application manifests, which will trigger a full deployment. Click the **Sync** button in the todo-list application tile to open the synchronize dialog, then click the **Synchronize** button to start the process. ![Synchronize application manifests](https://cdn.hashnode.com/res/hashnode/image/upload/v1640909545665/R_QLCRglt.png align="left") Wait for the status to show the green **Healthy** and **Synced** statuses, then click the tile to see the deployment details. In the application details page, you will see a visualization of the Kubernetes resources for the application and other details. To open the application, click the 3rd button inside the `todo-list` ingress box as highlighted in the screenshot below: ![Application resource details and visualization](https://cdn.hashnode.com/res/hashnode/image/upload/v1640910013947/_0DUsboAY.png align="left") Finally we get to see the fruit of our labor! Feel free to play around with the to-do list application to make sure that it is working. ![Verify to-do list application](https://cdn.hashnode.com/res/hashnode/image/upload/v1640910156759/tndIT6N5E.png align="left") ## Enabling auto-sync in ArgoCD to complete the CI/CD solution As we have finally verified a deployment, we need to make one final change to complete our CI/CD solution. To truly enable GitOps and CD, we need to turn on auto-sync for the application in ArgoCD. Back on the app details page in ArgoCD UI, click the **App Details** button to open the details dialog. In the **Summary** tab, scroll down to the **Sync Policy** section and click the **Enable Auto-Sync** button. ![Enable auto-sync](https://cdn.hashnode.com/res/hashnode/image/upload/v1640910564858/-xFE-ztOo6.png align="left") Then click the **OK** button when the prompt appears: ![Confirm auto-sync enablement](https://cdn.hashnode.com/res/hashnode/image/upload/v1640910568975/ybQxzSNp-c.png align="left") Lastly, we will trigger another run for the Tekton CI pipeline for the final verification. We could make some functional changes to the todo-list application, but to test auto-sync we can just update the image version to simulate an "application update". Edit the PipelineRun YAML file created earlier and change the `buildVersion` parameter value to, say, **0.0.2**. Then run the following command to start a new pipeline run: ```bash kubectl create -f todo-list-ci-pipelinerun.yaml -n todo-list-ci ``` Check the pipeline progress in Tekton Dashboard. When the run completes, go back to ArgoCD UI and verify that it has automatically synchronize after on the Git changes. You may need to refresh the status in the UI to reflect the latest status. In the application details page, the current sync status now shows that version **0.0.2** is deployed: ![Verify app version](https://cdn.hashnode.com/res/hashnode/image/upload/v1640911182837/qB8fYNl9j.png align="left") Congratulations! You have successfully created a basic Kubernetes-native CI/CD solution using Tekton and ArgoCD on DigitalOcean! After you have admired your creation and played with the environment more, don't forget to [destroy the Kubernetes cluster in DigitalOcean](https://docs.digitalocean.com/products/kubernetes/how-to/destroy-clusters/) and any leftover resources (volumes and load balancers for the ingress) to avoid unnecessary cost! ## What's next? In this challenge, I have only scratched the surface of Tekton and ArgoCD. With my remaining DigitalOcean credits available for another month, I would like to build upon the current solution with the following: 1. Implement trigger to truly complete the CI pipeline 2. Extend the current solution to multiple environments (dev, QA, prod) and evaluate various high availability features 3. Research on best practices and improve the current solution Check back later for new posts as I explore further! ## Resources During the challenge, I found the following resources that helped me understand how to use the tools and solve unexpected issues. Credit goes to the authors and I hope this curated list helps you as well. * [Kubernetes-Native Build & Release Pipelines with Tekton and ArgoCD](https://medium.com/dzerolabs/using-tekton-and-argocd-to-set-up-a-kubernetes-native-build-release-pipeline-cf4f4d9972b0) by Adri Villela - A comprehensive series on end-to-end Kubernetes-native CI/CD, as recommended on the challenge page itself. * [Tekton with ArgoCD](https://github.com/sdaschner/tekton-argocd-example) by Sebastian Daschner - A great blueprint for CI/CD with Tekton and ArgoCD. It saved me time scripting the Git update part for this challenge. * [ArgoCD Tutorial for Beginners | GitOps CD for Kubernetes](https://youtu.be/MeU5_k9ssrs) by TechWorld with Nana - A well-explained video tutorial for ArgoCD on YouTube. She has a knack for explaining complex concepts and I am a big fan of her videos. * [How to fix in Kubernetes – Deleting a PVC stuck in status "Terminating"](https://veducate.co.uk/kubernetes-pvc-terminating/) - This article helped me fix a problem where I could not delete a PVC and the associated volume. * [The Vanilla DevOps Git Credentials & Private Packages Cheatsheet](https://coolaj86.com/articles/vanilla-devops-git-credentials-cheatsheet/) by AJ ONeal - This cheatsheet helped me choose the best Git credentials management strategy for this challenge.