Sergio Ocón-CárdenasMonokle, Kustomize & Quality Kubernetes Deployments — Kubeshop
From the beginning, Kubernetes development has been complex. And as developers learned new tricks, split one manifest into many, tacked on configuration management or templating, and launched dozens or hundreds of new pods with a single run of kubectl, Kubernetes development got *really *complex.
In time, it was too much for a single person to understand, maintain, and optimize every resource and relationship required to deploy an application, whether between two locally-stored manifest files or two pods in a production cluster. New versions of existing Kubernetes API, like networking, were created and allowed new use cases, in many cases at the expense of additional complexity.
As a stopgap solution to this ballooning complexity, developers started “forking” manifests from vendors, using them as templates. Instead of writing a manifest for a reverse Nginx proxy again and again, why not download a manifest directly from the Nginx developers, add a few tweaks, and shortcut your time to deployment?
This model addressed one fundamental problem: that of deploying an application and the additional configuration elements required for it to work. Unfortunately, it didn’t take into account the full lifecycle of the application. When the vendor released a new version of their manifest (the configuration file) with added features or bug fixes, they had to re-fork and re-apply their customizations. This meticulous and manual workflow to stay synchronized with vendor releases was rigid and created more risk, slowing development.
There are many solutions in the Kubernetes space to address this problem, but one of them is built into kubectl: Kustomize—native configuration management designed for simplifying YAML without templates or forking.
How developers use Kustomize
Kustomize is an open-source project that “lets you customize raw, template-free YAML files for multiple purposes, leaving the original YAML untouched and usable as is.” It’s now the most popular tool for customizing Kubernetes manifests reasonably, and it’s even built directly into the Kubernetes CLI since K8s v1.14.
Kustomize lets developers preserve the base settings of their applications, Kubernetes resources, or components, and then apply “patches” that override said defaults. It allows you to add, remove and update configuration options, something that is useful in a whole variety of situations, like:
Defining environmental variables, like LOG_LEVEL, to help you debug issues
Deploy different versions of the database for development and production
Using Kubernetes secrets without committing them to Git by using a generator
Adding the same labels, namespaces and annotations to all Kubernetes resources
Editing the number of replicas created by a Kubernetes deployment, or the appropriate CPU and memory
Make all of the above dependent on the environment where you are deploying the application (Development, QA, Testing, Production)
Unfortunately, you will still need to create and use YAML files. Kubernetes simply won’t allow you to get rid of them because it uses manifests as the base that gets updated through customizations. So, Kustomize rescues us by allowing us to select an existing YAML manifest to customize before applying it in the server.
Let’s say that your application requires three configuration files (manifests) — a ConfigMap, a Deployment, and a Service — which are your “base” files, as Kustomize defines them. Your working directory would look like this:
your-app/
├── configmap.yaml
├── deployment.yaml
└── service.yaml
The easiest way to use Kustomize is to modify all files by adding a label kustomization.yaml file:
your-app/
├── configmap.yaml
├── deployment.yaml
├── service.yaml
└── kustomization.yaml
Once you have added Kustomization to the mix, create a single kustomization.yaml file into your working directory, and refer it to the previous files:
<table>
<tr>
<td><code>commonLabels:</code>
<p>
<code> app: hello</code>
<p>
<code>resources: \
- configmap.yaml \
- deployment.yaml</code>
<p>
<code>- service.yaml</code>
</td>
</tr>
<tr>
<td>
</td>
</tr>
</table>
For more complex customizations, like those dependent on the deployment environment, Kustomize supports the concept of overlay and a base. The base declares things that the variants share in common (both resources and a common customization of those resources), and the overlays declare the differences.
You then create an overlay (or patch; patch.yaml) that contains the configurations you want to override, and then build your edited manifests, and deploy your Kubernetes cluster, with kubectl apply -k.
In the end, your source tree looks something like this:
your-app/
├── base/
│ ├── configmap.yaml
│ ├── deployment.yaml
│ ├── service.yaml
│ └── kustomize.yaml
└── overlays/
├── production/
├── deployment.yaml
│ └── kustomize.yaml
└── testing/
├── deployment.yaml
└── kustomize.yaml
Kustomize also has some advanced capabilities, like generators for ConfigMaps and Secrets, that allows you to create them from files or literals.
You can find more details about Kustomize in the Kubernetes documentation and the kustomize repo:
kubernetes-sigs/kustomize: Customization of kubernetes YAML configurations (github.com)
Kustomize | SIG CLI (kubernetes.io)
The challenges of using Kustomize
While Kustomize simplifies development workflows around creating patches for commonly-reused manifests and Kubernetes resources, it’s not without some of its own headaches.
Primarily, Kustomize necessitates even more YAML files to create, edit, and maintain. You have not only your base files, which you might have developed yourself or “borrowed” from an open-source vendor/repository, but also at least one kustomization.yaml file and more than one `
` file. In the example above, you’ve gone from 3 YAML files that define everything to at least 5 YAML files, which are now intricately linked. You can’t fully understand your Kubernetes cluster unless you examine both the bases and the patches.
The complexity behind those relationships matters a lot when you’re debugging and validating the output of your manifests. If you try deploying and get an error in response, even a common Kubernetes error, it won’t be immediately clear whether said error comes from your base or a patch. You’ll have to spend time, and probably a few tweak-deploy-tweak-deploy cycles to get things right.
Furthermore, the Kustomization files need their own lifecycle. You need to store and manage them separately, as the underlying manifest files evolve with time, and you will need to create new versions to adapt your customization to new structures and values.
How does Monokle’s Manifest IDE help?
First off, let’s get clear on what Monokle and a Manifest IDE are. Monokle is an open source cross-platform desktop app for managing Kubernetes configuration files (like manifests) simplifying your everyday development tasks to help you deploy faster and safer. Monokle gives you a high-level view of your manifests, their resources, and their relationships, while abstracting away a ton of YAML’s inherent complexity and syntax issues.
But, when it comes to Kustomize, there’s even more.
Visualize the scope and relationships of your Kustomizations
The moment you add or select a folder containing one or more kustomization.yaml files, Monokle automatically highlights one of the files, visualizes all its “downstream” Kustomizations and resources in the Navigator, and shows the YAML of the Kustomization in the Editor.
With this three-panel view of your Kustomize files, you can quickly explore the scope of a Kustomization by viewing incoming/outgoing links, quickly jumping between interconnected resources, and editing YAML files directly based on your findings.
No more exploring through nests of patch/overlay folders and bringing up many files side-by-side (or in a grid of four boxes!) to understand how one tweak affects the rest of your deployment.
Preview and debug Kustomize resources
Monokle can perform a quick dry-run of your bases and patches to preview the generated resources, which helps you understand what would actually get installed on your Kubernetes cluster.
Hover over a Kustomization file to reveal the Preview button, which runs kustomize apply -k on that specific file to perform a dry-run and generate the possible output.
You can now explore all the generated resources and resource links, which streamlines the debugging process. If you edit the content of a previewed kustomization.yaml file, click the Reload button to recreate the preview for immediate visibility into how your latest tweak will get installed on your destination cluster.
No more applying your manifests with kubectl repeatedly trying to pin down the origin of a bug created by the obscured network of manifests that Kustomize generates.
Example: Kustomizing a vendor-supplied manifest
Let’s run through an example on how you might use Monokle’s manifest features at scale.
Imagine you want to deploy an off-the-shelf Kubernetes manifest from a trusted vendor, which will become your base file. You want to apply some modifications to it but you want to avoid forking too much from the original file and its subsequent versions. To get the result you want, you can put that file into a base/ folder, and create an overlay/ folder to hold the Kustomize files tailored for your target environments. Some simple kustomization.yaml files to define the base and the overlays, and the required patch.yaml files to override specific configurations of the vendor’s manifests. This is a lot of steps.
Monokle greatly simplifies the process and reduces the chance of errors helping you manage all the manifests you edit. By previewing them in real-time to debug your Kustomizations, you can rest assured that everything works before you try deploying to production.
If at a later stage you hear that your vendor has just released a new version of their manifest with some new features you’ve been eagerly waiting for, you can use Monokle to quickly update and otherwise breaking changes in your customizations to make them work again.
To roll out a new version of your existing manifest files, put Monokle into Preview mode to generate the resources and source code. This will reveal exactly where the breaking change impacts your Kustomization. Monokle’s editor will make it easy for you to edit your patch and use the preview to make sure that you have a successful dry run without needing to test it on your Kubernetes cluster.
You can even investigate a step further by running a Diff, which shows you exactly what changes will get pushed to your cluster’s deployments for one last validation before you hit Deploy.
Monokle simplifies the management of configuration files, including manifests, customizations and overlays, so you can focus on development and reduce the impact of changes on your dependencies.
Start Kustomizing with Monokle
To get started, hop on over to our downloads page to get Monokle for macOS, Windows, or Linux. In order to run the preview functionality, you’ll also need Kustomize installed locally (follow the instructions to do so in your operating system)
Join us on Discord! We’re happy to answer any questions about managing your Kustomizations through Monokle. We’re building our community, learning about your workflows, and imagining new features to make Kubernetes development simple again.
Let us know your feedback and ideas so we can make Monokle even more awesome and cover more of your use cases.
