Whenever you’re implementing a monitoring and observability solution for your environment, a few questions occur like what you’re going to monitor and the specific production environments that need to be monitored.

Another large and important question is how you’re going to monitor the environment.

With Kubernetes and monitoring/observability platforms that go with it, there are a ton of options. It can definitely get a bit overwhelming.

With Kube-Prometheus, the feeling of being overwhelmed should hopefully subside.

In this blog post, you’ll learn all about what Kube-Prometheus is and how to get it up and running.

What Is Kube-Prometheus?

It’s the “easy button”.

The Prometheus Operator itself gives you the ability to install all Prometheus and Grafana features in a declarative way. For example, below is a Kubernetes Manifest to install Alertmanager, which can be installed in a declarative fashion because the Prometheus Operator exists.

apiVersion: monitoring.coreos.com/v1
kind: Alertmanager
metadata:
  labels:
    app.kubernetes.io/component: alert-router
    app.kubernetes.io/instance: main
    app.kubernetes.io/name: alertmanager
    app.kubernetes.io/part-of: kube-prometheus
    app.kubernetes.io/version: 0.24.0
  name: main
  namespace: monitoring
spec:
  image: quay.io/prometheus/alertmanager:v0.24.0
  nodeSelector:
    kubernetes.io/os: linux
  podMetadata:
    labels:
      app.kubernetes.io/component: alert-router
      app.kubernetes.io/instance: main
      app.kubernetes.io/name: alertmanager
      app.kubernetes.io/part-of: kube-prometheus
      app.kubernetes.io/version: 0.24.0
  replicas: 3
  resources:
    limits:
      cpu: 100m
      memory: 100Mi
    requests:
      cpu: 4m
      memory: 100Mi
  securityContext:
    fsGroup: 2000
    runAsNonRoot: true
    runAsUser: 1000
  serviceAccountName:

Kube-Prometheus on the other hand installs Alertmanager for you.

Essentially, Kube-Prometheus contains a bunch of Kubernetes Manifests for installing components like Grafana, Alertmanager, Node Exporter, Kube State Metrics, and several other pieces including all of the RBAC permissions and Service Accounts needed to get it up and running.

Instead of installing all of the components manually, you can just use Kube-Prometheus.

Here’s a list of what’s installed: https://github.com/prometheus-operator/kube-prometheus/tree/main/manifests

Methods Of Installation

Now that you know what Kube-Prometheus is, which is essentially a collection of Kubernetes Manifests to get up and running fast with the Prometheus/Grafana stack, let’s learn a few different installation options.

The first is Helm and the second is a traditional Kubernetes Manifest deployment.

Please note that you don’t have to run both of these. You can run one or the other, or run one, delete one, and then run the next. That way there are no conflicts and you see both processes.

Helm

First, add the prometheus-community repo.

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts

Next, ensure that it’s up to date.

helm repo update

When installing Kube-Prometheus, you’ll use the prometheus-community Helm Chart. However, the Kube-Prometheus stack isn’t the only option available when using the prometheus-community Chart. Because of that, ensure that you specify the kube-prometheus-stack option.

helm install kube-prometheus prometheus-community/kube-prometheus-stack

You can find the Helm Chart here: https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack

Kubernetes Manifests

First, clone the prometheus-operator Git repository.

git clone https://github.com/prometheus-operator/kube-prometheus.git

Next, run the configuration for the Kubernetes Manifests.

The following command is used with sleep to avoid any race conditions, which ensures that the proper Service Accounts and RBAC permissions are set up prior to any of the Manifests getting deployed.

kubectl create -f manifests/setup

until kubectl get servicemonitors --all-namespaces ; do date; sleep 1; echo ""; done

kubectl create -f manifests/

Retrieve all Namespaces and you should see one called monitoring.

kubectl get namespaces

After a minute or two, ensure that all of the resources in the monitoring Namespace are up and running.

kubectl get all -n monitoring

Viewing Grafana

Now that you’ve installed the Kube-Prometheus stack, let’s log into Grafana.

First, retrieve the Grafana service. For the Helm installation, the service name will be kube-prometheus-grafana. For the Manifest installation, the service name will be grafana.

Next, forward the port so you can access the Grafana UI.

For Helm installation:

kubectl port-forward svc/kube-prometheus-grafana :80

For Manifest installation:

kubectl port-forward svc/grafana -n monitoring :3000

The default username/password for Grafana with the Helm Installation is:

• admin
• prom-operator.

The default username/password for Grafana with the Manifest installation is:

• admin
• admin

Click the Browse button under dashboards.

Under general (or default), you’ll see a bunch of pre-populated dashboards for you.

For example, click on the Kubernetes / Computer Resources / Cluster option. You’ll see a Grafana dashboard with information around your cluster.

What makes Kube-Prometheus so great is that all of the dashboards are populated for you. You don’t have to do much of anything to get the environment up and running out of the box.

You now have a great place to start with Prometheus and Grafana for monitoring/observability of your environment.