KubeshopThis post was originally published on the New Stack.
When companies talk about modernizing their tech stack or removing legacy infrastructure, it often includes everything except their testing toolchain — they are just happy when it isn’t broken.
While almost everything from simple web apps to stateful databases have moved toward more cloud native architectures, the way we test our applications is frequently still firmly stuck in another era.
It’s not without reason: Testing is usually tightly integrated with development tooling and related workflows because running them as part of builds allows us to easily check that the built artifacts are working as they should. However, this also creates a tightly coupled stack that is brittle to change and not resilient in the face of failure.
Cloud native is all about making things more modular, reusable and loosely coupled, yet a tightly coupled testing toolchain is the antithesis of these ideals. Being tied to certain tools and related workflows limits testing modularity across single tests, entire workflows and disparate infrastructures.
When only a small bug fix or functionality is added, integrated CI/CD workflows often force you to rerun your entire test suite, even when only a single test run may be needed. Once you have a workflow set up, what you are allowed to do as a tester is often limited by your CI/CD tooling and the access you’re given. And it’s often up to you to cobble together a coherent test-result dashboard based on the tools used to build and test your applications. Finally, once you have built your testing workflow for one cluster or platform, it is probably easier to start fresh again rather than try to port it elsewhere and even keeping it relevant for your current infrastructure can be a challenge.
Making Testing Cloud Native
Just putting testing on a cloud native platform like Kubernetes won’t help that much either. Provisioning test environments is also somewhat easier, especially with a GitOps approach, but grappling with different environments, cluster networking, remote access and so forth is still a challenge. And unfortunately, many testing tools aren’t “made for k8s,” which can result in clunky approaches when using them to test applications running on Kubernetes.
To help overcome these challenges and bring testing into the cloud native era, a few things are necessary. First, making your pipelines more reusable and modular to be used in any CI/CD or testing framework allows you to quickly adapt to changes in your solution without the need to adapt or create a completely different pipeline that is tightly coupled to the CI/CD or testing framework. Secondly, tests should be aware of the state of your infrastructure and integrated into it. By making testing cloud native, we can all bring code to production faster and more confidently.
Testing on Kubernetes for the Cloud Native Era
Building off the insights above, we’ve created TestKube, an open source lightweight testing framework for Kubernetes that takes a somewhat opinionated approach to testing.
1. Tests are included in the state of your cluster.
Tests are defined and executed using Kubernetes constructs and services in the cluster itself, ensuring that your applications and services are always “testable,” instead of having to rely on correctly configured, externally available tools and artifacts at any time. Test execution is performed from within the cluster itself, removing the need to configure/open your Kubernetes network for the sake of test execution, and no external dependencies are needed to validate that things are working as they should.
Test execution can be triggered by both internal and external triggers — kubectl commands, Kubernetes resource updates, or by CI/CD systems interacting directly with the TestKube API server. Need to rerun a single test to validate a ConfigMap update? No problem! Or run the entire test suite after your application has been deployed, triggered by ArgoCD? TestKube has you covered.
2. Test workflows and results are modular.
Test scripts, orchestrations and executors follow a generic format that is possible to use/implement for any type of test — functional, performance, security, conformance, etc. Test results are aggregated in a common format and exposed to external tooling via APIs or as standard metrics that can be consumed by tools like Prometheus, Grafana and such. This modularity allows teams to repurpose and reuse tests as needed and moves away from being tied to the abilities of a single CI/CD tool for test-related activities on Kubernetes.
3. Testing is environment aware and integrated.
The underlying architecture is open and modular. Since native Kubernetes constructs, like custom resources and ConfigMaps, are used to configure and orchestrate tests, GitOps workflows and tooling can ensure that tests in your cluster are always up to date. QA engineers can focus on building the right tests for your applications without having to worry about how or when to deploy them into your cluster or environment for execution, while they still have the power to both run tests and analyze test results as needed.
Where We Are and Where We Are Going
Given the ambitious goals set out above, Testkube is definitely a work in progress but already in a state to deliver on several of them. We’ve started with supporting the execution of Postman Collections for APIs and Cypress tests for UIs, and we plan to support more types of scripts, including Selenium, K6, JMeter, SoapUI and Cucumber. Check out the latest 0.8.0 release to see how you can orchestrate multiple scripts into complex integration tests and get integrated results via TestKube’s test result dashboard.
