Author: Sander Rodenhuis, Co-founder and CEO of Red Kubes (Gold Sponsor, KCD Chennai 2022)

Introduction

Kubernetes is still a relatively new technology but is being adopted at high speed. But be aware that Kubernetes is only a general-purpose cluster operating system kernel and requires additional applications and configuration to safely run and manage your containerized business applications.

Also, Kubernetes is not the holy grail for everything. If you're just running a couple of containers, then Kubernetes might not be the ideal go-to technology because of the risk of underutilization. This makes sharing a K8s cluster with multiple teams an interesting option.

Allowing multiple tenants on a shared cluster comes with challenges. For instance, how do you make sure the tenants can work independently of each other without interfering with one another?

Now, this would not be that hard to implement but it becomes a little more complicated if you would like these tenants to also be able to share generic platform applications and allow one tenant more control compared to another. Delegating control within the whole stack requires a lot of integration and custom engineering.

This article will explain how delegation of control can be easily implemented using the Otomi open-source project.

What is delegation of control?

From a management perspective, the best definition of delegation is when an administrator gives a user or group of users the responsibility and authority to complete specific tasks. In the IT world, delegated administration or delegation of control is about the decentralisation of role-based access control.

In Kubernetes, this model scales poorly because access control can only be done on the API level using Kubernetes RBAC. Allowing one tenant more access than another tenant can result in a very complex configuration moreover RBAC on the application level is implemented per application.

As a result, operation teams become burdened with lots of tasks. These - mainly not automated - tasks can incur high latency times or result in poor security practices.

Suppose you would like to:

Enforce specific resource quota to team A and offer team B the ability to adjust the configured resource quota
Allow team B to be able to configure network policies while the network policies of team A can only be implemented by the admin
Allow access to a team (tenant) on the platform based on an LDAP group membership and allow team B to change the group mapping, while the group mapping of team A can only be changed by the admin
Allow access to shared Kubernetes apps like HashiCorp Vault and/or Harbor per team/tenant

How much time would you think it takes to support this?

What is Otomi?

Otomi is a self-hosted PaaS for Kubernetes and can be installed in one run on any Kubernetes cluster. Otomi consists of a suite of pre-configured and integrated Kubernetes open-source apps, combined with self-service and automation.

The teams feature in Otomi offers an advanced multi-tenancy setup, where teams (tenants) get access to a single web UI with self-service tasks and shared applications. Administrators can create teams and delegate control to the teams and the group members.

How does Otomi support delegation?

Otomi can run in multi-tenant mode, allowing the creation of tenants (called Teams in Otomi). The foundation of a team is a Kubernetes namespace, combined with a default RBAC policy. Access to a team is controlled based on group membership.

You can use Keycloak as an IdP or configure Keycloak to act as an identity broker using an external IdP (like Azure AD). A user who is a member of the group mapped to the team will automatically get access to the applications running in the team-namespace.

Teams have access to self-service tasks to add

Pre-deployed K8s services
Knative services to the service mesh (based on Istio)
Configure public exposure
Create K8s Jobs and Cronjobs
Configure ingress/egress network policies

An admin can delegate control based on the following self-service flags:
Services

Configure ingress: allow the use of the self-service feature to expose services publicly
NetworkPolicy: allow the use of the self-service feature to configure ingress/egress network policies

Team

Alerts: grant the team permission to configure Alerts for the team
OIDC: grant the team permission to configure OIDC for the team
Resource Quota: grant the team permission to configure Resource Quota for the team
DownloadKubeConfig: grant the team permission to download the KubeConfig file
Network policy: grant the team permission to enable/disable network policies for the team

Delegation in action

All the above-mentioned configurations and self-service tasks can be enabled through the Otomi console without needing to write any YAML configurations.

Curious to try it out? We have created a demo environment for you folks to log in and explore the Otomi console.

username: demo
password: demo

Wrapping up

If you would like to have more control over what users of your Kubernetes clusters can and can not do, while at the same time offering a standardised workflow coupled with automation, self-service, and role-based access to shared applications in an advanced multi-tenant setup, then check out Otomi.

As open-source enthusiasts, we are always looking for contributors. Here are the links to our open-source projects.

redkubes/otomi-core (Do star us and show your support)
redkubes/workshops
redkubes/quickstart
Docs

Delegating control with an advanced multi-tenant setup in Kubernetes using Otomi