We have all heard off writers looking at a blank page for hours trying to get over writers block. For Kubernetes admins, setting up a brand new Kubernetes cluster can be just as daunting. 

Especially when it involves configuring multiple elements like ArgoCD, a PostgreSQL database, monitoring tooling, and custom web apps.

However, what if there was an easier way? Our new GitOps template is designed to make this process straightforward and hassle-free. It offers an easy to set up solution that requires almost no manual steps for a basic yet highly extendable GitOps setup. With this template, you can have your cluster up and running quickly and efficiently, allowing you to focus on what matters most, building and scaling your applications.

If you’re reading this, you likely already appreciate the benefits of GitOps and have a solid understanding of Kubernetes. However, until recently, there hasn’t been an elegant, standardized process for managing the packages installed in your Kubernetes clusters. That’s why we built one.

Let’s put it to the test. With this template, ArgoCD, CloudNativePG and a simple bookmarking web app will be installed using the Glasskube Package Manager and they’ll receive upgrades through a Renovate integration via pull requests. This future-proof setup is easy to maintain and ensures your clusters stay up-to-date effortlessly.

The next logical extension for infrastructure IaC

There are clear frameworks, templates, and standards for provisioning infrastructure across almost all cloud providers, and even templates for provisioning Kubernetes clusters themselves. However, this is often where the smooth, paved Infrastructure as Code (IaC) road turns to gravel. Kubernetes admins are frequently left to their own devices to install, configure, and manage the packages within the Kubernetes cluster due to a lack of standardized and simple package management tooling.

Converting package configurations to code and managing them through GitOps, as you would with internal applications, has proven difficult. We believe the Glasskube package is a significant step toward the much-needed standardization in this area.

What is a Glasskube package?

A Glasskube package is a standardized unit for managing software deployments within Kubernetes clusters using the Glasskube Package Manager. It is defined by a PackageManifest, which contains all necessary information for identifying, configuring and installing a package. This manifest can include a Helm resource or a link to a manifest. Dependencies between packages can be declared, ensuring all required components are present before installation.

Use ArgoCD to deploy Glasskube packages

Helm charts or plain Kubernetes manifests lack the comprehensive framework that Glasskube packages offer for deploying resources declaratively in a reliable, consistent, and easily maintained way. In this demo, we’ll use the GitOps template to bootstrap a fresh Kubernetes cluster with an instance of ArgoCD, which will then be used to deploy all subsequent packages.

Here’s what the cluster will look like.

Template diagram

Template structure

🔗 Link to template

The repository contains:

• a bootstrap directory containing the initial/parent Argo Application, and the necessary Glasskube manifests.
• a packages directory containing the ArgoCD ,cloudnative-pg and kube-prometheus-stack cluster packages.
• an apps folder containing a simple shiori bookmarking web app. 
• the renovate configuration file.

Glasskube custom resources will only be picked up by ArgoCD after placing the package definition files inside the packages directory.

⚠️ Please do not delete/uninstall the argo-cd package, as this will also remove everything else!

Note that the parent application used to bootstrap (bootstrap/glasskube-application.yaml) will not be synced after the initial setup. If you want to change something about it, you will have to change the application via argo directly.

Cluster setup

Prerequisites

You will have to have access to an empty Kubernetes cluster.

The easiest would be creating a new Minikube cluster with:

minikube start -p gitops

⚠️ Glasskube should not yet be bootstrapped in the cluster.

Install the Glasskube CLI

Make sure to have at least Glasskube version 0.16.0 installed locally. If you don’t, you can simply run:

brew install glasskube/tap/glasskube

How to use the template

Use this repository as your GitOps template

Create a public GitHub repository based on this starter template by clicking “Use this template“. You can move it and/or make it private afterward.

Replace the placeholder repoURL in your local GitOps repository

Replace the default value of repoURL to your repository url:

• Line 12 in: bootstrap/glasskube-application.yaml
• Lines 11, 16 and 26 in: bootstrap/glasskube/applicationset.yaml

Bootstrap ArgoCD and Glasskube for your Kubernetes cluster:

glasskube bootstrap git --url <your-repo>

The result

As a result, your cluster will be powered with GitOps capabilities by ArgoCD, as well as package management features by Glasskube. Argo manages itself, the Glasskube installation, as well as Glasskube packages — all of which you can now manage GitOps-style with this repo.

Run glasskube serve to open the Glasskube UI and either open the ArgoCD UI there, or with the command glasskube open argo-cd – but of course you can also use the Argo CLI. Follow the ArgoCD docs to get and reset the password to log in.

💡 Note that it might take a couple of minutes to start up ArgoCD, and for the initial GitOps sync to happen.

In this template, for demonstration purposes we’ve also installed the cloudnative-pg and kube-prometheus-stackclusterpackages and a bookmarking application (shiori), which is making use of cloudnative-pg.

Managing your cluster

Both CLI and UI offer features to manage your cluster following GitOps best practices.

Using the CLI

The relevant CLI commands offer the flags --dry-run and -o yaml which output the yaml object code which should then be pushed to your repository to be deployed.

Through the UI

The UI, when installed with the above glasskube bootstrap git command, will also output the yaml objects which you can copy to use in your git repo, instead of applying your changes directly.

Installing packages

To install a ClusterPackage, e.g. cert-manager, use the install command like this:

glasskube install cert-manager --dry-run -o yaml --yes > cert-manager.yaml

Instead of directly installing the ClusterPackage, this will write the ClusterPackage custom resource to the cert-manager.yaml file, which can now be put into a new directory packages/cert-manager/ in the git repository. Once pushed to the repo, ArgoCD will pick up the changes after at most 5 minutes, create the ArgoCD Application wrapping the Glasskube ClusterPackage. After that, the Glasskube operator will pick up the ClusterPackage and finally install it in the cluster.

Similarly, when using the Glasskube UI, one can generate the ClusterPackage resource by using the “Show YAML” button on the page of the ClusterPackage.

Updating packages

There are two options handling package version updates:

Manually

Using the glasskube update --dry-run -o yaml command, or the corresponding button on the Glasskube UI. And then pushing to your repo as seen before. 

The downside of doing it that way, is that someone has to manually execute the command, even though checking for updates and preparing the updates to the git repository as an automatable task.

Automatically with Renovate

Once Renovate is integrated to track your GitOps repo, it will look for Glasskube packages and compare their versions to the official package repositories. When new versions are available, it will automatically open a PR. Once merged, you’ll be running the latest versions of your packages.

Uninstalling packages

To uninstall a package or a ClusterPackage, simply remove the custom resource from the git repository.

Updating Glasskube

When a new Glasskube version is available, the manifests have to be updated. To update the Glasskube manifests in your git repo, run the following command:

glasskube bootstrap --dry-run -o yaml --force > bootstrap/glasskube/glasskube.yaml

After reviewing and merging those changes the update will be picked up by ArgoCD. The --force flag is necessary for the command to continue manifest validation, even though failures occur.

Working with Apps

This template also contains a demo application: a bookmark manager called shiori.
Its manifests are defined in apps/shiori, which is a pattern you can follow for your own custom applications.

In a minikube environment, two manual steps are required to access the application (for more information consult the
minikube docs):

• Run minikube addons enable ingress -p gitops.
• Run minikube ip -p gitops and add the line <your-IP> my-shiori.example to your /etc/hosts file.

After that you will be able to access the application via http://my-shiori.example in your browser.
The default login credentials are shiori / gopher – for more information check the shiori docs.

In general, you can use the apps directory to deploy such custom applications into your cluster. Any subdirectory will be
picked up by ArgoCD and grouped as an Application.

Monitoring with kube-prometheus-stack

This template also installs the kube-prometheus-stack clusterpackage, which is an easy way to get started with monitoring your cluster. You can open Grafana with glasskube open kube-prometheus-stack. It does not come preconfigured in this example, but you can easily add a nice postgres dashboard and observe the metrics of the database while you are working with the bookmarking application.

Setting up a postgres dashboard

We are going to make use of the cloudnativepg dashboard.
Import it by opening the dashboard-import page, pasting
https://grafana.com/grafana/dashboards/20417-cloudnativepg/
into the first textfield, and pressing "Load". Use the "Prometheus" data source on the following screen and finish the import process.

Of course monitoring your experimental minikube cluster is a bit of an overkill, but this is simply to demonstrate how these kind of cluster administration tasks can be integrated into this gitops stack.

Template setup walkthrough

Upcoming Features

Private Repo Support

We are aware that GitOps repositories should not be public, but for simplicity we omitted this feature in the first version of the new GitOps-bootstrap command. Supporting private repos with authentication of course has high priority for the upcoming releases. We will also replace the repoURL automatically, such that you don’t need to this step manually when setting up the repo.

Improved Renovate Integration

As described above, the renovate integration currently is regex-based, and it does not consider dependencies yet. However, we don’t see these shortcomings as a blocker and recommend to try out the renovate integration in the Glasskube/Argo Gitops setup.

Improved Dependency Resolution

Installing packages with dependencies is not 100% GitOps-compatible yet, as the dependencies will be created by the operator. Consider this: To install a ClusterPackage <P> that has a dependency on D, one would do glasskube install <P> --dry-run -o yaml, which would output the ClusterPackage custom resource for <P>. However, the dependency D will only be resolved at reconciliation time by the package operator, and will therefore not be represented in the git repository at all.

A temporary workaround would be to have a closer look at the output of the install command, which also shows the dependencies which will be installed and in which version. One could then manually add the required packages custom resources to the git repo as well. However, this will be tackled in a future version to make the user experience better, see this issue.

Summary

With this template repository and guide we show how Glasskube can easily be set up in a ArgoCD powered Gitops environment, and how efficient package management is possible with this stack. Additionally we install a web application to show how custom applications can make use of the Gitops setup and Glasskube packages.

This is a first concept with some minor shortcomings, but we will continue to improve GitOps support.

Feedback

We love feedback! Whether you are just starting out or you are a seasoned professional, we'd like to hear your thoughts, inputs and questions regarding this starter template and corresponding guide here, in the glasskube/glasskube repo or on our Discord. Thanks!