Hello there !
We saw in the part 2 the differents architectures that could be applied between both cloud providers for monitoring, logging and DevOps. We also talked about network traffic cost.
In this part 3, we will see how to build a DevOps platform in Google Cloud with GitLab and Kubernetes.
Let's start by provisioning the Google Cloud infrastructure.
Prerequisites
- Download and install Terraform.
- Download, install, and configure the Google Cloud SDK.
- Download and install Vault.
- Download and install ArgoCD.
- Download, install, and configure the Scaleway CLI
Plan
- Enabling the required services on devops project.
- Creating Virtual Private Network.
- Creating a Cloud NAT.
- Creating a KMS key for encryption.
- Creating the GKE cluster with the configured service account attached.
- Creating a public IP for Vault.
- Generating certificates for Vault.
- Deploying Vault in Kubernetes. [1]
- Registering Gitlab runners
- Configuring ArgoCD
We start by creating a private devops
repository in Gitlab and copying the terraform files into infra/plan
.
Terraform
Enabling Services
infra/plan/main.tf
resource "google_project_service" "service" {
count = length(var.project_services)
project = var.project_id
service = element(var.project_services, count.index)
disable_on_destroy = false
}
data "google_project" "project" {
}
Virtual Private Network
The following terraform file creates:
- A custom VPC
- A subnet with secondary ranges
- A Cloud NAT with two NAT IPs
infra/plan/vpc.tf
resource "google_compute_network" "vpc" {
name = "vpc"
auto_create_subnetworks = false
project = var.project_id
depends_on = [google_project_service.service]
}
resource "google_compute_subnetwork" "subnet-vpc" {
name = "subnet"
ip_cidr_range = var.subnet_ip_range_primary
region = var.region
network = google_compute_network.vpc.id
project = var.project_id
secondary_ip_range = [
{
range_name = "secondary-ip-ranges-devops-services"
ip_cidr_range = var.subnet_secondary_ip_range_services
},
{
range_name = "secondary-ip-ranges-devops-pods"
ip_cidr_range = var.subnet_secondary_ip_range_pods
}
]
private_ip_google_access = false
}
resource "google_compute_address" "nat" {
count = 2
name = "nat-external-${count.index}"
project = var.project_id
region = var.region
depends_on = [google_project_service.service]
}
resource "google_compute_router" "router" {
name = "router"
project = var.project_id
region = var.region
network = google_compute_network.vpc.self_link
bgp {
asn = 64514
}
}
resource "google_compute_router_nat" "nat" {
name = "nat-1"
project = var.project_id
router = google_compute_router.router.name
region = var.region
nat_ip_allocate_option = "MANUAL_ONLY"
nat_ips = google_compute_address.nat.*.self_link
source_subnetwork_ip_ranges_to_nat = "LIST_OF_SUBNETWORKS"
subnetwork {
name = google_compute_subnetwork.subnet-vpc.self_link
source_ip_ranges_to_nat = ["PRIMARY_IP_RANGE", "LIST_OF_SECONDARY_IP_RANGES"]
secondary_ip_range_names = [
google_compute_subnetwork.subnet-vpc.secondary_ip_range[0].range_name,
google_compute_subnetwork.subnet-vpc.secondary_ip_range[1].range_name,
]
}
}
GKE Cluster
The following terraform file creates:
- A GKE Cluster
- Default nodepool
- DevOps nodepool to run Gitlab runners
- Vault nodepool to run Vault
infra/plan/gke.tf
resource "google_container_cluster" "gke-devops-cluster" {
provider = google-beta
name = "gke-cluster-devops"
location = var.gke_devops_cluster_location
network = google_compute_network.vpc.id
subnetwork = google_compute_subnetwork.subnet-vpc.id
private_cluster_config {
enable_private_endpoint = false
enable_private_nodes = true
master_ipv4_cidr_block = var.master_ipv4_cidr_block
}
project = var.project_id
remove_default_node_pool = true
initial_node_count = 1
maintenance_policy {
recurring_window {
start_time = "2020-10-01T09:00:00-04:00"
end_time = "2050-10-01T17:00:00-04:00"
recurrence = "FREQ=WEEKLY"
}
}
enable_shielded_nodes = true
ip_allocation_policy {
cluster_secondary_range_name = "secondary-ip-ranges-devops-pods"
services_secondary_range_name = "secondary-ip-ranges-devops-services"
}
networking_mode = "VPC_NATIVE"
logging_service = "logging.googleapis.com/kubernetes"
monitoring_service = "monitoring.googleapis.com/kubernetes"
master_authorized_networks_config {
cidr_blocks {
cidr_block = var.gitlab_public_ip_ranges
display_name = "GITLAB PUBLIC IP RANGES"
}
cidr_blocks {
cidr_block = var.authorized_source_ranges
display_name = "Authorized IPs"
}
cidr_blocks {
cidr_block = "${google_compute_address.nat[0].address}/32"
display_name = "NAT IP 1"
}
cidr_blocks {
cidr_block = "${google_compute_address.nat[1].address}/32"
display_name = "NAT IP 2"
}
}
addons_config {
horizontal_pod_autoscaling {
disabled = false
}
http_load_balancing {
disabled = false
}
network_policy_config {
disabled = false
}
}
network_policy {
provider = "CALICO"
enabled = true
}
pod_security_policy_config {
enabled = false
}
release_channel {
channel = "STABLE"
}
workload_identity_config {
identity_namespace = "${var.project_id}.svc.id.goog"
}
database_encryption {
state = "ENCRYPTED"
key_name = google_kms_crypto_key.kubernetes-secrets.self_link
}
authentication.
master_auth {
username = ""
password = ""
client_certificate_config {
issue_client_certificate = false
}
}
depends_on = [
google_project_service.service,
google_project_iam_member.service-account,
google_compute_router_nat.nat
]
}
resource "google_container_node_pool" "gke-nodepools-default" {
project = var.project_id
name = "gke-nodepools-default"
location = var.gke_devops_cluster_location
cluster = google_container_cluster.gke-devops-cluster.name
initial_node_count = 1
node_config {
machine_type = var.node_pools_machine_type
metadata = {
disable-legacy-endpoints = "true"
}
oauth_scopes = [
"https://www.googleapis.com/auth/logging.write",
"https://www.googleapis.com/auth/monitoring",
"https://www.googleapis.com/auth/compute",
"https://www.googleapis.com/auth/devstorage.read_only"
]
tags = [
"gke-devops-nodes"]
}
}
resource "google_container_node_pool" "gke-nodepools-devops" {
project = var.project_id
name = "gke-nodepools-devops"
location = var.gke_devops_cluster_location
cluster = google_container_cluster.gke-devops-cluster.name
autoscaling {
max_node_count = 3
min_node_count = 0
}
node_config {
machine_type = var.node_pools_machine_type
preemptible = true
metadata = {
disable-legacy-endpoints = "true"
}
oauth_scopes = [
"https://www.googleapis.com/auth/logging.write",
"https://www.googleapis.com/auth/monitoring",
"https://www.googleapis.com/auth/compute",
"https://www.googleapis.com/auth/devstorage.read_only"
]
labels = {
"nodepool" = "devops"
}
taint {
key = "devops-reserved-pool"
value = "true"
effect = "NO_SCHEDULE"
}
tags = [
"gke-devops-nodes"]
}
}
resource "google_container_node_pool" "gke-nodepools-vault" {
project = var.project_id
name = "gke-nodepools-vault"
location = var.gke_devops_cluster_location
cluster = google_container_cluster.gke-devops-cluster.name
initial_node_count = 1
autoscaling {
max_node_count = 3
min_node_count = 1
}
node_config {
machine_type = var.node_pools_machine_type
service_account = google_service_account.vault-server.email
metadata = {
disable-legacy-endpoints = "true"
google-compute-enable-virtio-rng = "true"
}
oauth_scopes = [
"https://www.googleapis.com/auth/logging.write",
"https://www.googleapis.com/auth/monitoring",
"https://www.googleapis.com/auth/compute",
"https://www.googleapis.com/auth/devstorage.read_only",
"https://www.googleapis.com/auth/cloud-platform"
]
labels = {
nodepool = "vault"
service = "vault"
}
workload_metadata_config {
node_metadata = "SECURE"
}
taint {
key = "vault-reserved-pool"
value = "true"
effect = "NO_SCHEDULE"
}
tags = [
"gke-devops-nodes", "vault"]
}
}
resource "google_compute_address" "vault" {
name = "vault-lb"
region = var.region
project = var.project_id
depends_on = [google_project_service.service]
}
output "address" {
value = google_compute_address.vault.address
}
Vault
The following terraform file:
- Creates the vault service account
- Adds the service account to the project
- Adds user-specified roles
- Creates the storage bucket
- Generates a random suffix for the KMS keyring
- Creates the KMS key ring
- Creates the crypto key for encrypting init keys
- Creates the crypto key for encrypting Kubernetes secrets
- Grants GKE access to the key
infra/plan/vault.tf
resource "google_service_account" "vault-server" {
account_id = "vault-server"
display_name = "Vault Server"
project = var.project_id
}
resource "google_project_iam_member" "service-account" {
count = length(var.vault_service_account_iam_roles)
project = var.project_id
role = element(var.vault_service_account_iam_roles, count.index)
member = "serviceAccount:${google_service_account.vault-server.email}"
}
resource "google_project_iam_member" "service-account-custom" {
count = length(var.service_account_custom_iam_roles)
project = var.project_id
role = element(var.service_account_custom_iam_roles, count.index)
member = "serviceAccount:${google_service_account.vault-server.email}"
}
resource "google_storage_bucket" "vault" {
name = "${var.project_id}-vault-storage"
project = var.project_id
force_destroy = true
location = var.region
storage_class = "REGIONAL"
versioning {
enabled = true
}
lifecycle_rule {
action {
type = "Delete"
}
condition {
num_newer_versions = 1
}
}
depends_on = [google_project_service.service]
}
# Generate a random suffix for the KMS keyring. Like projects, key rings names
# must be globally unique within the project. A key ring also cannot be
# destroyed, so deleting and re-creating a key ring will fail.
#
# This uses a random_id to prevent that from happening.
resource "random_id" "kms_random" {
prefix = var.kms_key_ring_prefix
byte_length = "8"
}
# Obtain the key ring ID or use a randomly generated on.
locals {
kms_key_ring = var.kms_key_ring != "" ? var.kms_key_ring : random_id.kms_random.hex
}
resource "google_kms_key_ring" "vault" {
name = local.kms_key_ring
location = var.region
project = var.project_id
depends_on = [google_project_service.service]
}
resource "google_kms_crypto_key" "vault-init" {
name = var.kms_crypto_key
key_ring = google_kms_key_ring.vault.id
rotation_period = "604800s"
}
resource "google_kms_crypto_key" "kubernetes-secrets" {
name = var.kubernetes_secrets_crypto_key
key_ring = google_kms_key_ring.vault.id
rotation_period = "604800s"
}
resource "google_project_iam_member" "kubernetes-secrets-gke" {
project = var.project_id
role = "roles/cloudkms.cryptoKeyEncrypterDecrypter"
member = "serviceAccount:service-${data.google_project.project.number}@container-engine-robot.iam.gserviceaccount.com"
}
TLS
The following terraform file:
- Generates self-signed TLS certificates
- Creates the Vault server certificates
- Creates the request to sign the cert with our CA
- Signs the cert
infra/plan/tls.tf
resource "tls_private_key" "vault-ca" {
algorithm = "RSA"
rsa_bits = "2048"
}
resource "tls_self_signed_cert" "vault-ca" {
key_algorithm = tls_private_key.vault-ca.algorithm
private_key_pem = tls_private_key.vault-ca.private_key_pem
subject {
common_name = "vault-ca.local"
organization = "HashiCorp Vault"
}
validity_period_hours = 8760
is_ca_certificate = true
allowed_uses = [
"cert_signing",
"digital_signature",
"key_encipherment",
]
provisioner "local-exec" {
command = "echo '${self.cert_pem}' > ../tls/ca.pem && chmod 0600 ../tls/ca.pem"
}
}
resource "tls_private_key" "vault" {
algorithm = "RSA"
rsa_bits = "2048"
}
resource "tls_cert_request" "vault" {
key_algorithm = tls_private_key.vault.algorithm
private_key_pem = tls_private_key.vault.private_key_pem
dns_names = [
"vault",
"vault.local",
"vault.${var.public_dns_name}",
"vault.default.svc.cluster.local",
]
ip_addresses = [
google_compute_address.vault.address,
]
subject {
common_name = "vault.local"
organization = "HashiCorp Vault"
}
}
resource "tls_locally_signed_cert" "vault" {
cert_request_pem = tls_cert_request.vault.cert_request_pem
ca_key_algorithm = tls_private_key.vault-ca.algorithm
ca_private_key_pem = tls_private_key.vault-ca.private_key_pem
ca_cert_pem = tls_self_signed_cert.vault-ca.cert_pem
validity_period_hours = 8760
allowed_uses = [
"cert_signing",
"client_auth",
"digital_signature",
"key_encipherment",
"server_auth",
]
provisioner "local-exec" {
command = "echo '${self.cert_pem}' > ../tls/vault.pem && echo '${tls_self_signed_cert.vault-ca.cert_pem}' >> ../tls/vault.pem && chmod 0600 ../tls/vault.pem"
}
}
Kubernetes
The following terraform file:
- Queries the client configuration for our current service account
- Writes the vault TLS secret
- Creates vault resources in Kubernetes
infra/plan/k8s.tf
data "google_client_config" "current" {}
provider "kubernetes" {
load_config_file = false
host = google_container_cluster.gke-devops-cluster.endpoint
cluster_ca_certificate = base64decode(
google_container_cluster.gke-devops-cluster.master_auth[0].cluster_ca_certificate,
)
token = data.google_client_config.current.access_token
}
resource "kubernetes_secret" "vault-tls" {
metadata {
name = "vault-tls"
}
data = {
"vault.crt" = "${tls_locally_signed_cert.vault.cert_pem}\n${tls_self_signed_cert.vault-ca.cert_pem}"
"vault.key" = tls_private_key.vault.private_key_pem
"ca.crt" = tls_self_signed_cert.vault-ca.cert_pem
}
}
resource "kubernetes_service" "vault-lb" {
metadata {
name = "vault"
labels = {
app = "vault"
}
}
spec {
type = "LoadBalancer"
load_balancer_ip = google_compute_address.vault.address
load_balancer_source_ranges = [var.subnet_secondary_ip_range_pods, var.authorized_source_ranges]
external_traffic_policy = "Local"
selector = {
app = "vault"
}
port {
name = "vault-port"
port = 443
target_port = 8200
protocol = "TCP"
}
}
depends_on = [google_container_cluster.gke-devops-cluster]
}
resource "kubernetes_stateful_set" "vault" {
metadata {
name = "vault"
labels = {
app = "vault"
}
}
spec {
service_name = "vault"
replicas = var.num_vault_pods
selector {
match_labels = {
app = "vault"
}
}
template {
metadata {
labels = {
app = "vault"
}
}
spec {
termination_grace_period_seconds = 10
affinity {
pod_anti_affinity {
preferred_during_scheduling_ignored_during_execution {
weight = 50
pod_affinity_term {
topology_key = "kubernetes.io/hostname"
label_selector {
match_expressions {
key = "app"
operator = "In"
values = ["vault"]
}
}
}
}
}
}
node_selector = {
nodepool = "vault"
}
toleration {
key = "vault-reserved-pool"
operator = "Equal"
effect = "NoSchedule"
value = "true"
}
container {
name = "vault-init"
image = var.vault_init_container
image_pull_policy = "IfNotPresent"
resources {
requests {
cpu = "100m"
memory = "64Mi"
}
}
env {
name = "GCS_BUCKET_NAME"
value = google_storage_bucket.vault.name
}
env {
name = "KMS_KEY_ID"
value = google_kms_crypto_key.vault-init.self_link
}
env {
name = "VAULT_ADDR"
value = "http://127.0.0.1:8200"
}
env {
name = "VAULT_SECRET_SHARES"
value = var.vault_recovery_shares
}
env {
name = "VAULT_SECRET_THRESHOLD"
value = var.vault_recovery_threshold
}
}
container {
name = "vault"
image = var.vault_container
image_pull_policy = "IfNotPresent"
args = ["server"]
security_context {
capabilities {
add = ["IPC_LOCK"]
}
}
port {
name = "vault-port"
container_port = 8200
protocol = "TCP"
}
port {
name = "cluster-port"
container_port = 8201
protocol = "TCP"
}
resources {
requests {
cpu = "500m"
memory = "256Mi"
}
}
volume_mount {
name = "vault-tls"
mount_path = "/etc/vault/tls"
}
env {
name = "VAULT_ADDR"
value = "http://127.0.0.1:8200"
}
env {
name = "POD_IP_ADDR"
value_from {
field_ref {
field_path = "status.podIP"
}
}
}
env {
name = "VAULT_LOCAL_CONFIG"
value = <<EOF
api_addr = "https://vault.${var.public_dns_name}"
cluster_addr = "https://$(POD_IP_ADDR):8201"
log_level = "warn"
ui = true
seal "gcpckms" {
project = "${google_kms_key_ring.vault.project}"
region = "${google_kms_key_ring.vault.location}"
key_ring = "${google_kms_key_ring.vault.name}"
crypto_key = "${google_kms_crypto_key.vault-init.name}"
}
storage "gcs" {
bucket = "${google_storage_bucket.vault.name}"
ha_enabled = "true"
}
listener "tcp" {
address = "127.0.0.1:8200"
tls_disable = "true"
}
listener "tcp" {
address = "$(POD_IP_ADDR):8200"
tls_cert_file = "/etc/vault/tls/vault.crt"
tls_key_file = "/etc/vault/tls/vault.key"
tls_disable_client_certs = true
}
EOF
}
readiness_probe {
initial_delay_seconds = 5
period_seconds = 5
http_get {
path = "/v1/sys/health?standbyok=true"
port = 8200
scheme = "HTTPS"
}
}
}
volume {
name = "vault-tls"
secret {
secret_name = "vault-tls"
}
}
}
}
}
}
output "root_token_decrypt_command" {
value = "gsutil cat gs://${google_storage_bucket.vault.name}/root-token.enc | base64 --decode | gcloud kms decrypt --key ${google_kms_crypto_key.vault-init.self_link} --ciphertext-file - --plaintext-file -"
}
Other
infra/plan/variables.tf
variable "gke_devops_cluster_location" {
type = string
default = "europe-west1"
}
variable "region" {
type = string
}
variable "node_pools_machine_type" {
type = string
default = "e2-standard-2"
}
variable "master_ipv4_cidr_block" {
type = string
}
variable "subnet_ip_range_primary" {
type = string
default = "10.10.10.0/24"
}
variable "subnet_secondary_ip_range_services" {
type = string
default = "10.10.11.0/24"
}
variable "subnet_secondary_ip_range_pods" {
type = string
default = "10.1.0.0/20"
}
variable "public_dns_name" {
type = string
}
// deployment project id
variable "project_id" {
type = string
}
variable "gitlab_public_ip_ranges" {
type = string
description = "GITLAB PUBLIC IP RANGES"
}
variable "vault_service_account_iam_roles" {
type = list(string)
default = [
"roles/logging.logWriter",
"roles/monitoring.metricWriter",
"roles/monitoring.viewer",
"roles/cloudkms.cryptoKeyEncrypterDecrypter",
"roles/storage.objectAdmin"
]
description = "List of IAM roles to assign to the service account of vault."
}
variable "service_account_custom_iam_roles" {
type = list(string)
default = []
description = "List of arbitrary additional IAM roles to attach to the service account on the Vault nodes."
}
variable "project_services" {
type = list(string)
default = [
"secretmanager.googleapis.com",
"cloudkms.googleapis.com",
"cloudresourcemanager.googleapis.com",
"container.googleapis.com",
"compute.googleapis.com",
"iam.googleapis.com",
"logging.googleapis.com",
"monitoring.googleapis.com",
"cloudbuild.googleapis.com"
]
description = "List of services to enable on the project."
}
# This is an option used by the kubernetes provider, but is part of the Vault
# security posture.
variable "authorized_source_ranges" {
type = string
description = "Addresses or CIDR blocks which are allowed to connect to the Vault IP address. The default behavior is to allow anyone (0.0.0.0/0) access. You should restrict access to external IPs that need to access the Vault cluster."
}
#
# KMS options
# ------------------------------
variable "kms_key_ring_prefix" {
type = string
default = "vault"
description = "String value to prefix the generated key ring with."
}
variable "kms_key_ring" {
type = string
default = ""
description = "String value to use for the name of the KMS key ring. This exists for backwards-compatability for users of the existing configurations. Please use kms_key_ring_prefix instead."
}
variable "kms_crypto_key" {
type = string
default = "vault-init"
description = "String value to use for the name of the KMS crypto key."
}
variable "num_vault_pods" {
type = number
default = 3
description = "Number of Vault pods to run. Anti-affinity rules spread pods across available nodes. Please use an odd number for better availability."
}
#
# Kubernetes options
# ------------------------------
variable "kubernetes_secrets_crypto_key" {
type = string
default = "kubernetes-secrets"
description = "Name of the KMS key to use for encrypting the Kubernetes database."
}
variable "vault_container" {
type = string
default = "vault:1.2.1"
description = "Name of the Vault container image to deploy. This can be specified like \"container:version\" or as a full container URL."
}
variable "vault_init_container" {
type = string
default = "sethvargo/vault-init:1.0.0"
description = "Name of the Vault init container image to deploy. This can be specified like \"container:version\" or as a full container URL."
}
variable "vault_recovery_shares" {
type = string
default = "1"
description = "Number of recovery keys to generate."
}
variable "vault_recovery_threshold" {
type = string
default = "1"
description = "Number of recovery keys required for quorum. This must be less than or equal to \"vault_recovery_keys\"."
}
infra/plan/terraform.tfvars
region = "<GCP_REGION>"
gke_devops_cluster_location = "<GCP_GKE_CLUSTER_ZONE>"
master_ipv4_cidr_block = "172.23.0.0/28"
project_id = "<GCP_PROJECT_ID>"
gitlab_public_ip_ranges = "34.74.90.64/28"
authorized_source_ranges = "<LOCAL_IP_RANGES>"
public_dns_name = "<PUBLIC_DNS_NAME>"
infra/plan/backend.tf
terraform {
backend "gcs" {
}
}
infra/plan/version.tf
terraform {
required_version = ">= 0.12"
required_providers {
google = "~> 3.0"
}
}
Deploy GCP resources
Before deploying our DevOps platform, we need to export some global variables:
export GCP_PROJECT_ID=<GCP_PROJECT_ID>
export SW_PROJECT_NAME=<SW_PROJECT_NAME>
export GIT_REPOSITORY_URL=<MY_REPO>/demo-env.git
export GCP_REGION_DEFAULT=europe-west1
export GCP_GKE_CLUSTER_ZONE=europe-west1-b
export GCP_KUBE_CONTEXT_NAME="gke_${GCP_PROJECT_ID}_${GCP_GKE_CLUSTER_ZONE}_gke-cluster-devops"
export PUBLIC_DNS_NAME=
export PUBLIC_DNS_ZONE_NAME=
export TERRAFORM_BUCKET_NAME=bucket-${GCP_PROJECT_ID}-sw-gcp-terraform-backend
Create a Google Cloud Storage bucket for terraform states:
gcloud config set project ${GCP_PROJECT_ID}
gsutil mb -c standard -l ${GCP_REGION_DEFAULT} gs://${TERRAFORM_BUCKET_NAME}
gsutil versioning set onย gs://${TERRAFORM_BUCKET_NAME}
Initialize the terraform by completing the backend config. The states will be saved in GCP.
cd infra/plan
terraform init \
-backend-config="bucket=${TERRAFORM_BUCKET_NAME}" \
-backend-config="prefix=googlecloud/terraform/state"
Now we can complete the variables in the file infra/plan/terraform.tfvars
and deploy our DevOps Platform in Google Cloud !
sed -i "s/<LOCAL_IP_RANGES>/$(curl -s http://checkip.amazonaws.com/)\/32/g;s/<PUBLIC_DNS_NAME>/${PUBLIC_DNS_NAME}/g;s/<GCP_PROJECT_ID>/${GCP_PROJECT_ID}/g;s/<GCP_REGION>/${GCP_REGION_DEFAULT}/g;s/<GCP_GKE_CLUSTER_ZONE>/${GCP_GKE_CLUSTER_ZONE}/g" terraform.tfvars
terraform apply
Once the terraform is finished, we can access to the GKE Cluster using this command:
gcloud container clusters get-credentials gke-cluster-devops --zone ${GCP_GKE_CLUSTER_ZONE} --project ${GCP_PROJECT_ID}
Configuring Vault
To save Scaleway credentials in Vault, we need to set some environment variable for Vault.
Set Vault's address, the CA to use for validation, and the initial root token.
# cd infra/plan
export VAULT_ADDR="https://$(terraform output address)"
export VAULT_TOKEN="$(eval `terraform output root_token_decrypt_command`)"
export VAULT_CAPATH="$(cd ../ && pwd)/tls/ca.pem"
Save scaleway credentials:
vault secrets enable -path=scaleway/project/${SW_PROJECT_NAME} -version=2 kv
vault kv put scaleway/project/${SW_PROJECT_NAME}/credentials/access key="<SCW_ACCESS_KEY>"
vault kv put scaleway/project/${SW_PROJECT_NAME}/credentials/secret key="<SCW_SECRET_KEY>"
vault kv put scaleway/project/${SW_PROJECT_NAME}/config id="<SW_PROJECT_ID>"
To read the terraform states, reading secrets, building docker images within a Gitlab pipeline, we need to create a Google Service Account (GSA) with the necessary permissions:
gcloud iam service-accounts create gsa-dev-deployer
gcloud projects add-iam-policy-binding ${GCP_PROJECT_ID} \
--role roles/container.developer \
--member "serviceAccount:gsa-dev-deployer@${GCP_PROJECT_ID}.iam.gserviceaccount.com"
gcloud projects add-iam-policy-binding ${GCP_PROJECT_ID} \
--role roles/secretmanager.secretAccessor \
--member "serviceAccount:gsa-dev-deployer@${GCP_PROJECT_ID}.iam.gserviceaccount.com"
gcloud projects add-iam-policy-binding ${GCP_PROJECT_ID} \
--role roles/cloudbuild.builds.builder \
--member "serviceAccount:gsa-dev-deployer@${GCP_PROJECT_ID}.iam.gserviceaccount.com"
gcloud projects add-iam-policy-binding ${GCP_PROJECT_ID} \
--role roles/iam.serviceAccountAdmin \
--member "serviceAccount:gsa-dev-deployer@${GCP_PROJECT_ID}.iam.gserviceaccount.com"
Create a namespace to run Gitlab runner jobs:
kubectl create namespace sw-dev
To allow the Gitlab runner job to access GCP resources, we need to bind the GSA created above to a Kubernetes Service Account (KSA) [2]:
kubectl create serviceaccount -n sw-dev ksa-sw-dev-deployer
gcloud iam service-accounts add-iam-policy-binding \
--role roles/iam.workloadIdentityUser \
--member "serviceAccount:${GCP_PROJECT_ID}.svc.id.goog[sw-dev/ksa-sw-dev-deployer]" \
gsa-dev-deployer@${GCP_PROJECT_ID}.iam.gserviceaccount.com
kubectl annotate serviceaccount \
-n sw-dev \
ksa-sw-dev-deployer \
iam.gke.io/gcp-service-account=gsa-dev-deployer@${GCP_PROJECT_ID}.iam.gserviceaccount.com
Let's now create the policy policy-sw-dev-deployer
to allow our Gitlab runner to read contents from Vault:
vault policy write policy-sw-dev-deployer - <<EOF
# Read-only permissions
path "scaleway/project/${SW_PROJECT_NAME}/*" {
capabilities = [ "read" ]
}
EOF
Create a token and add the policy-sw-dev-deployer
policy.
GITLAB_RUNNER_VAULT_TOKEN=$(vault token create -policy=policy-sw-dev-deployer | grep "token" | awk 'NR==1{print $2}')
If you prefer a temporary access token, configure auth methods to automatically assign a set of policies to tokens:
vault auth enable approle
vault write auth/approle/role/sw-dev-deployer \
secret_id_ttl=10m \
token_num_uses=10 \
token_ttl=20m \
token_max_ttl=30m \
secret_id_num_uses=40 \
token_policies=policy-sw-dev-deployer
ROLE_ID=$(vault read -field=role_id auth/approle/role/sw-dev-deployer/role-id)
SECRET_ID=$(vault write -f -field=secret_id auth/approle/role/sw-dev-deployer/secret-id)
GITLAB_RUNNER_VAULT_TOKEN=$(vault write auth/approle/login role_id="$ROLE_ID" secret_id="$SECRET_ID" | grep "token" | awk 'NR==1{print $2}')
Vault is actually accessible from the public IP created previously. However, if you have a your own domain name registered in Google Cloud DNS, you can create an alias record to access Vault from a subdomain:
gcloud dns record-sets transaction start --zone=$PUBLIC_DNS_ZONE_NAME
gcloud dns record-sets transaction add "$(gcloud compute addresses list --filter=name=vault-lb --format="value(ADDRESS)")" --name=vault.$PUBLIC_DNS_NAME. --ttl=300 --type=A --zone=$PUBLIC_DNS_ZONE_NAME
gcloud dns record-sets transaction execute --zone=$PUBLIC_DNS_ZONE_NAME
VAULT_ADDR="https://vault.${PUBLIC_DNS_NAME}"
We complete the configuration by saving the vault token in Google Secret Manager.
gcloud beta secrets create vault-token --locations $GCP_REGION_DEFAULT --replication-policy user-managed
echo -n "${GITLAB_RUNNER_VAULT_TOKEN}" | gcloud beta secrets versions add vault-token --data-file=-
Configuring GitLab
Gitlab CI
Create a Gitlab SSH Key to allow the runner to push on git repositories and save the private key in Secret Manager:
gcloud beta secrets create gitlab-ssh-key --locations $GCP_REGION_DEFAULT --replication-policy user-managed
cd ~/.ssh
ssh-keygen -t rsa -b 4096
gcloud beta secrets versions add gitlab-ssh-key --data-file=./id_rsa
Save the private key in an environment variable:
GITLAB_SSH_KEY=$(gcloud secrets versions access latest --secret=gitlab-ssh-key)
Our Gitlab runner will run in a container. To make it quick, create a Docker image with all the necessary tools: Vault, ArgoCD, Terraform, gcloud sdk, sw cli.
docker/Dockerfile
FROM gcr.io/google.com/cloudsdktool/cloud-sdk:alpine
RUN gcloud components install kustomize kpt kubectl alpha beta
# install argocd
RUN curl -sSL -o /usr/local/bin/argocd https://github.com/argoproj/argo-cd/releases/download/$(curl --silent "https://api.github.com/repos/argoproj/argo-cd/releases/latest" | grep '"tag_name"' | sed -E 's/.*"([^"]+)".*/\1/')/argocd-linux-amd64
RUN chmod +x /usr/local/bin/argocd
# install vault
ENV VAULT_VERSION=1.6.0
RUN curl -sS "https://releases.hashicorp.com/vault/${VAULT_VERSION}/vault_${VAULT_VERSION}_linux_amd64.zip" > vault.zip && \
unzip vault.zip -d /usr/bin && \
rm vault.zip
# install terraform
ENV TERRAFORM_VERSION=0.12.24
RUN curl -sS "https://releases.hashicorp.com/terraform/${TERRAFORM_VERSION}/terraform_${TERRAFORM_VERSION}_linux_amd64.zip" > terraform.zip && \
unzip terraform.zip -d /usr/bin && \
rm terraform.zip
# Install sw cli
ENV SCW_VERSION=2.2.3
RUN curl -o /usr/local/bin/scw -L "https://github.com/scaleway/scaleway-cli/releases/download/v${SCW_VERSION}/scw-${SCW_VERSION}-linux-x86_64"
RUN chmod +x /usr/local/bin/scw
RUN vault -v
RUN terraform -v
RUN argocd
RUN gcloud -v
RUN scw version
ARG GITLAB_SSH_KEY
ARG VAULT_ADDR
ARG VAULT_CA
ARG VAULT_TOKEN
RUN echo -n $VAULT_CA > /home/ca.pem
RUN sed -i 's/\\n/\n/g' /home/ca.pem
ENV GITLAB_SSH_KEY=$GITLAB_SSH_KEY
ENV VAULT_ADDR=$VAULT_ADDR
ENV VAULT_TOKEN=$VAULT_TOKEN
ENV VAULT_CAPATH="/home/ca.pem"
docker/cloudbuild.yaml
steps:
- name: 'gcr.io/cloud-builders/docker'
args: [ 'build',
'--build-arg',
'VAULT_CA=${_VAULT_CA}',
'--build-arg',
'GITLAB_SSH_KEY=${_GITLAB_SSH_KEY}',
'-t', 'eu.gcr.io/${_PROJECT_ID}/tools:$_VERSION',
'.' ]
images:
- 'eu.gcr.io/${_PROJECT_ID}/tools:$_VERSION'
We use Google Cloud Build to build an publish our Image in Google Container Registry (GCR)
cd docker
gcloud builds submit --config cloudbuild.yaml --substitutions \
_VAULT_CA="$(awk 'NF {sub(/\r/, ""); printf "%s\\n",$0;}' ../infra/tls/ca.pem)",_VERSION="latest",_GITLAB_SSH_KEY="$GITLAB_SSH_KEY",_PROJECT_ID="$GCP_PROJECT_ID"
Now we can configure our Gitlab runner in Kubernetes. Start by installing Helm 3:
curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3
chmod 700 get_helm.sh
./get_helm.sh
Create kubernetes roles to the Gitlab runner.
gitlab/dev/rbac-gitlab-demo-dev.yml
apiVersion: v1
kind: ServiceAccount
metadata:
name: ksa-sw-devops-gitlab-deployer
namespace: sw-dev
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: role-ksa-sw-devops-gitlab-deployer
namespace: sw-dev
rules:
- apiGroups: [""] # "" indicates the sw API group
resources: ["pods", "pods/exec", "secrets"]
verbs: ["get", "list", "watch", "create", "patch", "delete"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: rolebinding-ksa-sw-devops-gitlab-deployer
namespace: sw-dev
subjects:
- kind: ServiceAccount
name: ksa-sw-devops-gitlab-deployer # Name is case sensitive
apiGroup: ""
roleRef:
kind: Role #this must be Role or ClusterRole
name: role-ksa-sw-devops-gitlab-deployer # this must match the name of the Role or ClusterRole you wish to bind to
apiGroup: rbac.authorization.k8s.io
kubectl apply -f gitlab/dev/rbac-gitlab-demo-dev.yml
Add Gitlab Helm package:
helm repo add gitlab https://charts.gitlab.io
Save the runner registration token in a secret:
kubectl create secret generic secret-sw-devops-gitlab-runner-tokens --from-literal=runner-token='' --from-literal=runner-registration-token='<DEMO_INFRA_REPO_RUNNER_TOKEN>' -n sw-dev
Deploy the Gitlab Runner in Kubernetes:
gitlab/dev/values.yaml
## Specify a imagePullPolicy
## 'Always' if imageTag is 'latest', else set to 'IfNotPresent'
## ref: http://kubernetes.io/docs/user-guide/images/#pre-pulling-images
##
imagePullPolicy: IfNotPresent
## The GitLab Server URL (with protocol) that want to register the runner against
## ref: https://docs.gitlab.com/runner/commands/README.html#gitlab-runner-register
##
gitlabUrl: https://gitlab.com/
## The registration token for adding new Runners to the GitLab server. This must
## be retrieved from your GitLab instance.
## ref: https://docs.gitlab.com/ee/ci/runners/
##
# runnerRegistrationToken: "<>"
## Unregister all runners before termination
##
## Updating the runner's chart version or configuration will cause the runner container
## to be terminated and created again. This may cause your Gitlab instance to reference
## non-existant runners. Un-registering the runner before termination mitigates this issue.
## ref: https://docs.gitlab.com/runner/commands/README.html#gitlab-runner-unregister
##
unregisterRunners: true
## When stopping the runner, give it time to wait for its jobs to terminate.
##
## Updating the runner's chart version or configuration will cause the runner container
## to be terminated with a graceful stop request. terminationGracePeriodSeconds
## instructs Kubernetes to wait long enough for the runner pod to terminate gracefully.
## ref: https://docs.gitlab.com/runner/commands/#signals
terminationGracePeriodSeconds: 3600
## Set the certsSecretName in order to pass custom certificates for GitLab Runner to use
## Provide resource name for a Kubernetes Secret Object in the same namespace,
## this is used to populate the /etc/gitlab-runner/certs directory
## ref: https://docs.gitlab.com/runner/configuration/tls-self-signed.html#supported-options-for-self-signed-certificates
##
#certsSecretName:
## Configure the maximum number of concurrent jobs
## ref: https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-global-section
##
concurrent: 10
## Defines in seconds how often to check GitLab for a new builds
## ref: https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-global-section
##
checkInterval: 30
## For RBAC support:
rbac:
create: false
## Run the gitlab-bastion container with the ability to deploy/manage containers of jobs
## cluster-wide or only within namespace
clusterWideAccess: false
## If RBAC is disabled in this Helm chart, use the following Kubernetes Service Account name.
##
serviceAccountName: ksa-sw-devops-gitlab-deployer
## Configure integrated Prometheus metrics exporter
## ref: https://docs.gitlab.com/runner/monitoring/#configuration-of-the-metrics-http-server
##
metrics:
enabled: true
## Configuration for the Pods that the runner launches for each new job
##
runners:
# config: |
# [[runners]]
# [runners.kubernetes]
# image = "ubuntu:16.04"
## Default container image to use for builds when none is specified
##
image: ubuntu:18.04
## Specify whether the runner should be locked to a specific project: true, false. Defaults to true.
##
locked: false
## The amount of time, in seconds, that needs to pass before the runner will
## timeout attempting to connect to the container it has just created.
## ref: https://docs.gitlab.com/runner/executors/kubernetes.html
##
pollTimeout: 360
## Specify whether the runner should only run protected branches.
## Defaults to False.
##
## ref: https://docs.gitlab.com/ee/ci/runners/#protected-runners
##
protected: true
## Service Account to be used for runners
##
serviceAccountName: ksa-sw-dev-deployer
## Run all containers with the privileged flag enabled
## This will allow the docker:stable-dind image to run if you need to run Docker
## commands. Please read the docs before turning this on:
## ref: https://docs.gitlab.com/runner/executors/kubernetes.html#using-docker-dind
##
privileged: false
## The name of the secret containing runner-token and runner-registration-token
secret: secret-sw-devops-gitlab-runner-tokens
## Namespace to run Kubernetes jobs in (defaults to 'default')
##
namespace: sw-dev
## Build Container specific configuration
##
builds:
# cpuLimit: 200m
# memoryLimit: 256Mi
cpuRequests: 100m
memoryRequests: 128Mi
## Service Container specific configuration
##
services:
# cpuLimit: 200m
# memoryLimit: 256Mi
cpuRequests: 100m
memoryRequests: 128Mi
## Helper Container specific configuration
##
helpers:
# cpuLimit: 200m
# memoryLimit: 256Mi
cpuRequests: 100m
memoryRequests: 128Mi
## Specify the tags associated with the runner. Comma-separated list of tags.
##
## ref: https://docs.gitlab.com/ce/ci/runners/#using-tags
##
tags: "k8s-dev-runner"
## Node labels for pod assignment
##
nodeSelector:
nodepool: devops
## Specify node tolerations for CI job pods assignment
## ref: https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/
##
nodeTolerations:
- key: "devops-reserved-pool"
operator: "Equal"
value: "true"
effect: "NoSchedule"
## Configure environment variables that will be injected to the pods that are created while
## the build is running. These variables are passed as parameters, i.e. `--env "NAME=VALUE"`,
## to `gitlab-runner register` command.
##
## Note that `envVars` (see below) are only present in the runner pod, not the pods that are
## created for each build.
##
## ref: https://docs.gitlab.com/runner/commands/#gitlab-runner-register
##
# env:
## Distributed runners caching
## ref: https://gitlab.com/gitlab-org/gitlab-runner/blob/master/docs/configuration/autoscale.md#distributed-runners-caching
##
## If you want to use gcs based distributing caching:
## First of all you need to uncomment General settings and GCS settings sections.
# cache:
## General settings
# cacheType: gcs
# cachePath: "k8s_platform_sw_devops_runner"
# cacheShared: false
## GCS settings
# gcsBucketName:
## Use this line for access using access-id and private-key
# secretName: gcsaccess
## Use this line for access using google-application-credentials file
# secretName: google-application-credentials
## Helper container security context configuration
## Refer to https://docs.gitlab.com/runner/executors/kubernetes.html#using-security-context
# pod_security_context:
# run_as_non_root: true
# run_as_user: 100
# run_as_group: 100
# fs_group: 65533
# supplemental_groups: [101, 102]
helm install -n sw-dev sw-dev -f gitlab/dev/values.yaml gitlab/gitlab-runner
Let's test if the Gitlab runner has the appropriate authorizations:
kubectl run -it \
--image eu.gcr.io/${GCP_PROJECT_ID}/tools \
--serviceaccount ksa-sw-dev-deployer \
--namespace sw-dev \
gitlab-runner-auth-test
Inside the pod container, run gcloud auth list
. You can delete the pod afterwards kubectl delete pod gitlab-runner-auth-test -n sw-dev
.
Gitlab Projects
- Create repositories
demo-app
,demo-env
anddemo-infra
. - Add protected tags
v*
on both repositories-app
and-env
. Go toSettings > Repository > Protected Tags
. - Enable Gitlab runners on
-infra
,-env
and-app
. Go toSettings > CI / CD > Runners > Specific Runners > Enable for this project
. - Lock gitlab runners for current projects (icon "edit" of the activated runner).
Configuring ArgoCD
Argo CD is a declarative, GitOps continuous delivery tool for Kubernetes. Let's configure it in our Kubernetes cluster:
Install ArgoCD in Kubernetes:
kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml
kubectl create clusterrolebinding cluster-admin-binding --clusterrole=cluster-admin --user="$(gcloud config get-value account)"
To access the Argo CD API Server, we need to patch the service:
kubectl patch svc argocd-server -n argocd -p '{"spec": {"type": "LoadBalancer"}}'
In addition to the default configuration we need to :
- Register the Gitlab env repository.
- Create a policy for the ArgoCD users.
k8s/argocd-configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: argocd-cm
namespace: argocd
labels:
app.kubernetes.io/name: argocd-cm
app.kubernetes.io/part-of: argocd
data:
repositories: |
- url: <GIT_REPOSITORY_URL>
passwordSecret:
name: demo
key: password
usernameSecret:
name: demo
key: username
# add an additional local user with apiKey and login capabilities
# apiKey - allows generating API keys
# login - allows to login using UI
admin.enabled: "true"
accounts.demo.enabled: "true"
accounts.demo: login
accounts.gitlab.enabled: "true"
accounts.gitlab: apiKey
k8s/argocd-rbac-configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: argocd-rbac-cm
namespace: argocd
labels:
app.kubernetes.io/name: argocd-rbac-cm
app.kubernetes.io/part-of: argocd
data:
# policies
policy.default: role:readonly
policy.csv: |
p, role:demo-admins, applications, create, *, allow
p, role:demo-admins, applications, get, *, allow
p, role:demo-admins, applications, list, *, allow
p, role:demo-admins, clusters, create, *, allow
p, role:demo-admins, clusters, get, *, allow
p, role:demo-admins, clusters, list, *, allow
p, role:demo-admins, projects, create, *, allow
p, role:demo-admins, projects, get, *, allow
p, role:demo-admins, projects, list, *, allow
p, role:demo-admins, repositories, create, *, allow
p, role:demo-admins, repositories, get, *, allow
p, role:demo-admins, repositories, list, *, allow
g, gitlab, role:demo-admins
If we have a domain name, we can access ArgoCD from a subdomain.
- Create a DNS for ArgoCD
- Create SSL certificates
k8s/argocd-server-ingress.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: argocd-server-https-ingress
namespace: argocd
annotations:
ingress.kubernetes.io/ssl-redirect: "true"
kubernetes.io/ingress.global-static-ip-name: argocd
networking.gke.io/managed-certificates: argocd
spec:
backend:
serviceName: argocd-server
servicePort: http
rules:
- host: argocd.<PUBLIC_DNS_NAME>
http:
paths:
- path: /
backend:
serviceName: argocd-server
servicePort: http
k8s/argocd-server-managed-certificate.yaml
apiVersion: networking.gke.io/v1beta1
kind: ManagedCertificate
metadata:
name: argocd
namespace: argocd
spec:
domains:
- argocd.<PUBLIC_DNS_NAME>
k8s/argocd-server.patch.yaml
spec:
template:
spec:
containers:
- command:
- argocd-server
- --staticassets
- /shared/app
- --insecure
name: argocd-server
We will apply those manifests later.
Configure the app-demo-dev
application in ArgoCD:
cd k8s
export GITLAB_USERNAME_SECRET=<GITLAB_USERNAME_SECRET>
export GITLAB_CI_PUSH_TOKEN=<GITLAB_CI_PUSH_TOKEN>
kubectl create secret generic demo -n argocd \
--from-literal=username=$GITLAB_USERNAME_SECRET \
--from-literal=password=$GITLAB_CI_PUSH_TOKEN
As we saw, if you have a cloud DNS available, we can create an ingress for ArgoCD and attaching an external static IP + a managed SSL certificate:
gcloud compute addresses create argocd --global
gcloud dns record-sets transaction start --zone=$PUBLIC_DNS_ZONE_NAME
gcloud dns record-sets transaction add $(gcloud compute addresses list --filter=name=argocd --format="value(ADDRESS)") --name=argocd.$PUBLIC_DNS_NAME. --ttl=300 --type=A --zone=$PUBLIC_DNS_ZONE_NAME
gcloud dns record-sets transaction execute --zone=$PUBLIC_DNS_ZONE_NAME
To reach ArgoCD services, we need to allow GCP Load balancers IP ranges:
gcloud compute firewall-rules create fw-allow-health-checks \
--network=vpc \
--action=ALLOW \
--direction=INGRESS \
--source-ranges=35.191.0.0/16,130.211.0.0/22 \
--rules=tcp
We can now enable ingress resource for ArgoCD:
kubectl patch svc argocd-server -n argocd -p '{"spec": {"type": "NodePort"}}'
kubectl annotate svc argocd-server -n argocd \
cloud.google.com/neg='{"ingress": true}'
sed -i "s/<PUBLIC_DNS_NAME>/${PUBLIC_DNS_NAME}/g" argocd-server-managed-certificate.yaml
kubectl apply -f argocd-server-managed-certificate.yaml
sed -i "s/<PUBLIC_DNS_NAME>/${PUBLIC_DNS_NAME}/g" argocd-server-ingress.yaml
kubectl apply -f argocd-server-ingress.yaml
kubectl patch deployment argocd-server -n argocd -p "$(cat argocd-server.patch.yaml)"
Once ArgoCD server ingress is created, login using the CLI:
kubectl wait ingress argocd-server-https-ingress --for=condition=available --timeout=600s -n argocd
ARGOCD_ADDR="argocd.${PUBLIC_DNS_NAME}"
# get default password
ARGOCD_DEFAULT_PASSWORD=$(kubectl get pods -n argocd -l app.kubernetes.io/name=argocd-server -o name | cut -d'/' -f 2)
argocd login $ARGOCD_ADDR --grpc-web
# change password
argocd account update-password
# for any issue, reset the password, edit the argocd-secret secret and update the admin.password field with a new bcrypt hash. You can use a site like https://www.browserling.com/tools/bcrypt to generate a new hash.
kubectl -n argocd patch secret argocd-secret \
-p '{"stringData": {
"admin.password": "<BCRYPT_HASH>",
"admin.passwordMtime": "'$(date +%FT%T%Z)'"
}}'
Create repositories, users and rbacs:
sed -i "s,<GIT_REPOSITORY_URL>,$GIT_REPOSITORY_URL,g" argocd-configmap.yaml
kubectl apply -n argocd -f argocd-configmap.yaml
kubectl apply -n argocd -f argocd-rbac-configmap.yaml
Change demo user password:
argocd account update-password --account demo --current-password "${ARGOCD_DEFAULT_PASSWORD}" --new-password "<NEW_PASSWORD>"
Generate an access token for ArgoCD:
AROGOCD_TOKEN=$(argocd account generate-token --account gitlab)
Save ArgoCD token in Secret Manager:
gcloud beta secrets create argocd-token --locations $GCP_REGION_DEFAULT --replication-policy user-managed
echo -n "${AROGOCD_TOKEN}" | gcloud beta secrets versions add argocd-token --data-file=-
Scaleway
scw init
Create a service account to allow Kapsule to access Google Container Registry:
SW_SA_EMAIL=$(gcloud iam service-accounts --format='value(email)' create sw-gcr-auth-ro)
gcloud projects add-iam-policy-binding ${GCP_PROJECT_ID} --member serviceAccount:$SW_SA_EMAIL --role roles/storage.objectViewer
Conclusion
Our DevOps platform is now ready to deploy resources on Scaleway. In the next part we will see how to deploy a Kapsule Cluster in Scaleway from Gitlab runners registered in Google Cloud.
Documentation
[1] https://github.com/sethvargo/vault-on-gke
[2] https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity