Crossplane vs Terraform

Spacelift team - Oct 18 - - Dev Community

In the infrastructure as code (IaC) world, Terraform and Crossplane are both prominent tools. Terraform’s popularity has risen in the last few years, and Crossplane is on the verge of a big breakthrough. Both tools are used for infrastructure provisioning, but their features and design principles are different.

Terraform is a BSL tool designed to provision cloud infrastructure using HashiCorp Configuration Language (HCL), making it easy to define, manage, and version infrastructure as code (IaC), while Crossplane provides a control plane to manage your IaC directly from your K8s cluster.

In this post, we will review both of them and help define which tool best fits your needs.

What is Crossplane?

Crossplane is an open-source IaC tool that is deployed inside a Kubernetes (k8s) cluster to enable the management of cloud infrastructure via the k8s API and kubectl.

Initially, Crossplane was developed by Upbound, but later they donated the project to the Cloud Native Computing Foundation (CNCF). Three years after the donation, Crossplane is now at the maturity level of an incubating project. You can learn more about CNCF's project maturity levels here.

Crossplane encapsulates guardrails such as policies and permissions behind a custom API to enable customers to self-service without requiring them to become infrastructure experts.

Key features

Key features of Crossplane:

  • Extensible platform - Users can create new custom resources to represent any cloud service.
  • Integrated with Kubernetes - It offers seamless integration with K8s, allowing infrastructure deployments to be composed in the same way as application deployments.
  • Multi-cloud capabilities - With Crossplane, you can provision and manage resources across multiple cloud providers.
  • Event-driven architecture - Crossplane's architecture allows it to respond immediately to changes, continuously reconciling the desired state of resources.
  • Open-source and community-driven - Crossplane is continuously evolving due to its open-source nature.

What is Terraform?

Terraform is an IaC tool, developed by HashiCorp, which allows users to manage their infrastructure using a declarative configuration language called HashiCorp Configuration Language (HCL).

In recent years, Terraform has become an industry standard for IaC, due to the large community that has formed around it.

Terraform was initially developed under an open-source license, but on August 10, 2023, it switched to BUSL.

Key features

Key features of Terraform:

  • State management - Your infrastructure configuration is saved to a state file to track resources and configuration.
  • Declarative code - HCL is really easy to use: Users describe the desired state of their infrastructure, and Terraform takes care of it.
  • Large ecosystem - It supports over 3k providers, from major cloud platforms to smaller services.
  • Modularity and reusability - Its declarative language means you can break your infrastructure into multiple reusable modules. 

Differences between Terraform and Crossplane

Terraform and Crossplane are two different tools designed to achieve the same objective: infrastructure provisioning and management.

1. Framework and setup

Terraform is a standalone tool that can be easily installed in any operating system, while Crossplane can only be installed as an operator inside your Kubernetes cluster.

2. Configuration language

The configuration language used by Terraform is HCL, which is a declarative language that supports loops and conditional statements as well. The default configuration language for Crossplane is yaml, making it familiar to those who are already using Kubernetes and Ansible. Both tools support JSON as well, but you won't see many configurations defined like this.

3. Workflow

Terraform workflow is easy to understand and is based on imperative runs. The tool is easy to install on any operating system, and Terraform's registry offers comprehensive documentation for modules and providers. Crossplane's workflow can be a little more complicated, especially because you first need to have a K8s cluster installed, know how K8s works, and then learn how you can leverage Crossplane.

4. State handling

Related to state handling, Terraform has a central state management system with optional (but this is a best practice) remote backends that support state locking to ensure that no jobs run in parallel that may compromise the state.

Crossplane has controller-based continuous reconciliation making it capable of constantly monitoring and modifying resources to ensure that the actual state of infrastructure matches the desired state defined in the configuration.

5. Community support and licensing

Terraform has great community support, but it feels like it is declining due to its licensing change to BSL (luckily, OpenTofu is here). Crossplane has a much smaller community, but it seems to be rising, especially because it is under the CNCF umbrella with an Apache License 2.0.

Terraform vs. Crossplane table comparison

Terraform and Crossplane are two different tools designed to achieve the same objective: infrastructure provisioning and management.

The table below summarizes the main differences between them:

crossplane vs terraform table comparison

When to use Crossplane and when Terraform?

Usually, Crossplane is used when you are heavily invested in Kubernetes and want a unified approach for managing both your infrastructure and applications. For major cloud providers, you could also use a k8s operator as an alternative.

Crossplane performs real-time management and reconciliation of your cloud resources based on the configuration you have defined.

Terraform has a mature ecosystem that supports a wide range of providers. It is also pretty easy to set up, use, and learn. If you are looking for a tool with strong community support and extensive resources and documentation, Terraform is the best choice.

In addition, Terraform offers fine-grained control over your infrastructure creation and management, and having the plan mechanism in place before running an apply to your infrastructure makes things less error-prone.

Terraform does a good job on its own, but to complete your Terraform workflow, you can extend it with TACOs such as Spacelift, and integrate policies, drift-detection, and third-party tools like TFLint, tfsec, and Infracost.

Using Terraform and Crossplane together

To use Crossplane, you first need to deploy your Kubernetes cluster.

The best way to deploy your k8s cluster is by --- you've guessed it --- using Terraform. So even though you will use Crossplane for infrastructure management, the best practice for the underlying management of the cluster is via Terraform.

You can also use Terraform and Crossplane together in another scenario. Basically, you could use Terraform for provisioning the foundational infrastructure components such as VPC, IAM, and EKS, and use Crossplane solely to manage the infrastructure resources your microservices need.

💡 You might also like:

What is Upbound?

Upbound is the company behind Crossplane. They offer two enterprise-grade distributions of Crossplane, Upbound and Universal Crossplane (UXP), which provide additional features such as governance, security, and the ability to scale better. UXP is open-source, too, and is a dropdown replacement for Crossplane, aiming to add tools and enterprise features beyond Crossplane's CNCF scope.

UXP can run both on-premise or any cloud solutions and with their Upbound official providers, you have 100% coverage of each provider API, having resource parity with Terraform.

Upbound, or Upbound Cloud, is a SaaS version of Crossplane meant to simplify the operation and management of Crossplane.

Building a similar infrastructure for AWS using Crossplane and Terraform

In this section, we will build a simple AWS infrastructure containing a VPC and a Subnet using both Crossplane and Terraform.

Installing and configuring Crossplane

To install Crossplane, you need a Kubernetes Cluster. For this example, I am going to use an EKS cluster that already exists. I will follow Crossplane's documentation for the AWS Quickstart for installing the tool.

The first step is to enable the Crossplane Helm Chart:

helm repo add\
crossplane-stable https://charts.crossplane.io/stable
helm repo update
Enter fullscreen mode Exit fullscreen mode

After that, I will install the chart:

helm install crossplane\
crossplane-stable/crossplane\
--namespace crossplane-system\
--create-namespace
Enter fullscreen mode Exit fullscreen mode

Check if Crossplane is installed successfully by running:

kubectl get pods -n crossplane-system
NAME                                       READY   STATUS    RESTARTS   AGE
crossplane-8697f8cff4-cfgcg                1/1     Running   0          26s
crossplane-rbac-manager-6f8dbd9ffd-bdth8   1/1     Running   0          26s
Enter fullscreen mode Exit fullscreen mode

Next, let's install the EC2 AWS provider in order to create VPCs inside the cluster:

cat <<EOF | kubectl apply -f -
apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
  name: provider-aws-ec2
spec:
  package: xpkg.upbound.io/upbound/provider-aws-ec2:v0.40.0
EOF
Enter fullscreen mode Exit fullscreen mode

You can see if the provider has been installed correctly by running:

kubectl get providers
NAME                          INSTALLED   HEALTHY   PACKAGE                                               AGE
provider-aws-ec2              True        True      xpkg.upbound.io/upbound/provider-aws-ec2:v0.40.0      20s
upbound-provider-family-aws   True        True      xpkg.upbound.io/upbound/provider-family-aws:v0.40.0   55m
Enter fullscreen mode Exit fullscreen mode

The next step in the documentation is to create a secret based on your AWS credentials, but we will not do that --- static credentials should be avoided at all costs.

We will use IAM Roles for Service Accounts (IRSA) to avoid unnecessary static credentials. First, install eksctl by following this guide. Then create an OpenID Connect (OIDC) identity provider for your EKS cluster:

eksctl utils associate-iam-oidc-provider --cluster=<cluster_name> --region=<region> ---approve
Enter fullscreen mode Exit fullscreen mode

Then, let's create a role and associate it to the Crossplane service account:

eksctl create iamserviceaccount --cluster=<clusterName> --name=<serviceAccountName> --namespace=<serviceAccountNamespace> --attach-policy-arn=<policyARN> --region=<region> --override-existing-serviceaccounts --approve
Enter fullscreen mode Exit fullscreen mode

By default, the serviceAccountName is crossplane and the namespace is crossplane-system.

To simplify things, I will attach the admin policy to this role (arn:aws:iam::aws:policy/AdministratorAccess).

eksctl create iamserviceaccount --cluster=demo-cluster --name=crossplane --namespace=crossplane-system --attach-policy-arn=arn:aws:iam::aws:policy/AdministratorAccess --region eu-west-1 --override-existing-serviceaccounts --approve

2023-09-08 12:41:20 [ℹ]  1 iamserviceaccount (crossplane-system/crossplane) was included (based on the include/exclude rules)
2023-09-08 12:41:20 [!]  metadata of serviceaccounts that exist in Kubernetes will be updated, as --override-existing-serviceaccounts was set
2023-09-08 12:41:20 [ℹ]  1 task: {
    2 sequential sub-tasks: {
        create IAM role for serviceaccount "crossplane-system/crossplane",
        create serviceaccount "crossplane-system/crossplane",
    } }2023-09-08 12:41:20 [ℹ]  building iamserviceaccount stack "eksctl-demo-cluster-addon-iamserviceaccount-crossplane-system-crossplane"
2023-09-08 12:41:20 [ℹ]  deploying stack "eksctl-demo-cluster-addon-iamserviceaccount-crossplane-system-crossplane"
2023-09-08 12:41:20 [ℹ]  waiting for CloudFormation stack "eksctl-demo-cluster-addon-iamserviceaccount-crossplane-system-crossplane"
2023-09-08 12:41:50 [ℹ]  waiting for CloudFormation stack "eksctl-demo-cluster-addon-iamserviceaccount-crossplane-system-crossplane"
2023-09-08 12:41:51 [ℹ]  serviceaccount "crossplane-system/crossplane" already exists
2023-09-08 12:41:51 [ℹ]  updated serviceaccount "crossplane-system/crossplane"
Enter fullscreen mode Exit fullscreen mode

As this role is associated with our service account, we can proceed with creating a providerconfig for it.

apiVersion: aws.upbound.io/v1beta1
kind: ProviderConfig
metadata:
  name: irsa
spec:
  credentials:
    source: IRSA
Enter fullscreen mode Exit fullscreen mode

Now, we can reference this provider we have configured in any ec2 resource we want to create. You can find all supported ec2-related resources in the documentation.

Creating a VPC and a Subnet with Crossplane 

apiVersion: ec2.aws.upbound.io/v1beta1
kind: VPC
metadata:
  labels:
    name: vpc-example
  name: vpc-example
spec:
  forProvider:
    cidrBlock: 10.1.0.0/16
    region: eu-west-1
  providerConfigRef:
    name: provider-aws-ec2

---
apiVersion: ec2.aws.upbound.io/v1beta1
kind: Subnet
metadata:
  labels:
    name: subnet-example
  name: subnet-example
spec:
  forProvider:
    availabilityZone: eu-west-1b
    cidrBlock: 10.1.5.0/24
    region: eu-west-1
    vpcIdSelector:
      matchLabels:
        name: vpc-example
  providerConfigRef:
    name: provider-aws-ec2
Enter fullscreen mode Exit fullscreen mode

Save the above contents in a yaml file and run kubectl apply -f on it. In a couple of seconds, both resources will be created.

Creating links between resources in Crossplane references the matchLabels property, respecting Kubernetes' design.

At the beginning of a run, both resources will start with a Synced state of False, and because the subnet has a dependency on the VPC, it will actually wait in the False Synced state until the VPC reaches the ready state. This happens because the subnet needs to get the VPC id, and for a resource to have an id, it must be provisioned successfully.

Installing Terraform

To install Terraform, simply go here and download the correct version for your operating system and add it to your PATH.

Creating a VPC and Subnet with Terraform 

provider "aws" {
  region = "eu-west-1"
}
resource "aws_vpc" "this" {
    cidr_block = "10.1.0.0/16"
}

resource "aws_subnet" "this" {
    cidr_block = "10.1.5.0/24"
    vpc_id     = aws_vpc.this.id
}
Enter fullscreen mode Exit fullscreen mode

Terraform will also handle the dependencies for you, and you can easily create links between resources by using a reference to the resource you want to depend on.

In our example, the subnet will be created after the vpc because we are explicitly linking it to the vpc with vpc_id = aws_vpc.this.id.

Spacelift integration with Terraform

Elevate your workflow with Spacelift. Head over to your Spacelift account, but if you don't have one, you can create a free account here.

Prior to everything else, I've pushed the above Terraform code to a GitHub repository to use it directly in Spacelift. You can find the code here.

Next, head over to Cloud Integrations and select AWS. Paste in a role you would like to use, and ensure the role is configured as presented in the documentation. This will help you avoid static credentials inside your workflow.

terraform crossplane cloud integration

Next, head over to Stacks and click on Create Stack.

terraform crossplane create stack

Give your Stack a name, and select a space for it. After that, pick the GitHub repository and the branch you have the code on:

terraform crossplane create stack repo

The next step is selecting a backend. Spacelift supports Terraform, Terragrunt, Kubernetes, Ansible, Pulumi, and CloudFormation.

For this example, I will use Terraform with version 1.5.7. 

terraform crossplane create stack version

I will let Spacelift manage my state because I don't want to create an S3 bucket or implement locking mechanisms. With Spacelift, you don't have to worry about any of those.

To spice things up, I also want to leverage a third-party security tool in my workflow, so I will use tfsec for that. Spacelift not only enables you to write policies for your code and other points inside your workflow, but it also allows you to declare policies for third-party tools using the custom inputs feature.

For this example, I won't do that, but I will show you how easy it is to add a third-party tool to your workflow.

terraform crossplane custom inputs

You can control what happens before and after all runner stages: Initialization, Planning, Applying, Performing, Destroying, Finally.

Now that I've put everything in place, I will save my stack.

Before running for the first time, I will associate the Cloud Integration I created previously with this stack to take advantage of the dynamic credentials. To do that, head over to Settings → Integrations and select the Integration you have created by clicking the Attach button.

terraform crossplane attach integration

We are ready to run the code now, and because we've added the tfsec step after the initialization, this is what we are going to see:

terraform vs crossplane tfsec

Having the "-s" option in place is not going to make our workflow error out; you will just see the output in the after init step.

The plan looks like this:

crossplane example

You can choose to view the plan in a more human-readable format, or you can opt to view the terraform plan as you would normally do in the console:

terraform crossplane plan

You have the option to either Confirm or Discard the run. Confirming it will leverage the terraform apply command.

terraform crossplane apply

As mentioned before, your Terraform workflow can be greatly elevated by leveraging Spacelift, as you can easily embed third-party tools, define policies in different decision points in the application, integrate easily with cloud providers for dynamic credentials, and more.

Spacelift integration with Kubernetes

You can use the integration that Spacelift has with Kubernetes to run your Kubernetes workflows. This means that you can leverage this integration to run your Crossplane code, too.

The code I'm using can be found here.

Let's dive into it and create a new stack. You do this in the same way you did for the Terraform one, but you will need to change the backend to Kubernetes:

crossplane kubernetes

In the define behavior tab, ensure you are using an image that has kubectl installed and use one of the available authentication methods for your Kubernetes cluster.

terraform crossplane kubernetes cluster

You will need to reuse the cloud integration we've done for the Terraform part to leverage dynamic credentials for AWS. 

The role that you are using for the cloud integration should also be added to the Kubernetes config_map aws-auth to ensure it can access the cluster. 

This can be done by manually editing the config_map or by running the following command:

eksctl create iamidentitymapping\
    --cluster $cluster_name\
    --region $region\
    --arn $role\
    --group system:masters\
    --no-duplicate-arns\
    --username $user
Enter fullscreen mode Exit fullscreen mode

If you don't have eksctl installed, you can install it by following the guide from here.

By following the installation guide of Crossplane on the EKS cluster from the beginning of the post, you can now create IaC resources in your AWS account using Crossplane from Spacelift.

I will again reuse the code that creates an AWS VPC and a Subnet:

terraform crossplane vpc

This is the plan you get in the log format, and the apply result is similar to this:

subnet.ec2.aws.upbound.io/subnet-example created vpc.ec2.aws.upbound.io/vpc-example created
Enter fullscreen mode Exit fullscreen mode

terraform crossplane run view

Let's create a policy that checks VPC and Subnet masks to see if they match what we want. For the VPC we will use a /16 matching, and for the subnet we will use a /24.

package spacelift

cidr_block_vpc = block {
    resources := input.kubernetes.items[_].after
    resources.kind == "VPC"
    ip_parts := split(resources.spec.forProvider.cidrBlock, "/")
    block := ip_parts[1]
}

cidr_block_subnet = block {
    resources := input.kubernetes.items[_].after
    resources.kind == "Subnet"
    ip_parts := split(resources.spec.forProvider.cidrBlock, "/")
    block := ip_parts[1]
}

warn[msg] {
    cidr_block_vpc != "16"
    msg := sprintf("Your VPC cidr block mask /%v is different than /16", [cidr_block_vpc])
}

warn[msg] {
    cidr_block_subnet != "25"
    msg := sprintf("Your Subnet cidr block mask /%v is different than /25", [cidr_block_subnet])
}

sample = true
Enter fullscreen mode Exit fullscreen mode

Go to Policies → Create Policy, select plan policy, and paste in the above code.

Now, go back to your Stack → Settings → Policy and attach the policy you've just created.

Let's run the code again and see if we get any warnings.

terraform crossplane policy

We are getting a warning because the subnet cidr block's mask is /24 instead of /25.

Let's switch the warning with a deny for the subnet rule and rerun to see if anything changes:

crossplane warning

Our run will fail because it was denied by the plan policy. 

What should I choose Terraform or Crossplane?

Usually, a small software engineering team that focuses on developing microservices would choose a tool such as Crossplane or  Kubernetes operator instead of Terraform to define the underlying infrastructure for their services.

But imagine you are running K8s in the cloud: How are you going to create your cluster, the network, the IAM roles, and other things related to the core infrastructure of your product? Well, you can't use Crossplane or a K8s operator for that because the cluster doesn't exist yet.

One option is to create these manually, but this defeats the purpose of infrastructure management. Another option is to use a CLI or an SDK, but you will still need to implement methods for creating infrastructure, getting existing infrastructure, deleting it, and changing it. That is time-consuming and error-prone.

You could potentially use AWS CloudFormation, Azure Bicep, or GCP Deployment Manager for that, but switching to a new cloud provider would be harder to do, as you would need to learn a new tool for managing your infrastructure. That's where Terraform comes into play, helping you learn a single configuration language for all of the cloud providers.

Other factors you need to take into account when choosing the best IaC tool for your needs include:

  • Scope -- Crossplane states that it has 100% coverage of each provider API, having resource parity with Terraform. Although this is true for the major cloud providers, in Terraform many providers come up on a daily basis, so having 100% coverage for Crossplane can be impossible to achieve.
  • Complexity and learning curve - To define their configurations, Terraform uses HCL, and Crossplane uses yaml. Terraform's learning curve is less steep due to a big community and the many tutorials available. Crossplane's complexity also derives from the fact that you have to manage a K8s cluster for it.
  • Integration and Scalability - Terraform is more popular and people have invested considerable resources in it. Many third-party tools support Terraform's workflow, from many points of view (security, governance, deployments, linting, etc.). Crossplane is an emerging tool that's still at CNCF's Incubating level, so it's expected to rise over the coming years.
  • Maturity, community, and support - Terraform has been around since 2014, so it has a large community and many people involved in helping others get started or resolving their problems. On the other hand, Crossplane is newer and it has a smaller, albeit growing community.
  • Cost - Crossplane requires a K8s cluster, so you will need to pay for the cluster's underlying infrastructure to be able to create your resources. Terraform, on the other hand, can be 100% free if you run it locally or you leverage the generous always-free tier that Spacelift offers.
  • Licensing model - Crossplane is open-source, and Terraform has changed its license model to BUSL since Aug 10, 2023.  If the open-source license was a selling point for you, a new player on the block called OpenTofu, which started as a Terraform fork after the BUSL announcement, is and will always remain open-source.

In the end, there is no correct answer to choosing between Terraform and Crossplane. It will always depend on your use case.

Key points

Your choice of IaC tool to manage your infrastructure will always depend on your use case, how much you want to spend, and the size of your company.

Both Terraform and Crossplane have benefits and drawbacks, but when you are choosing between them, you must consider your team's experience, the learning curve, and whether or not you need a mature tool.

To make management easier, using an IaC orchestrator, such as Spacelift, will greatly enhance your workflow.

If you want to learn more about Spacelift, create a free account or book a demo with one of our engineers.

Image
How Best Filter Coffee Machine Has Become The Most Sought-After Trend In 2024
Image
AWS Logging, Monitoring & Auditing with AWS CloudWatch and CloudTrail!

aws, cloudwatch, cloudtrail, observability
Image
10 Things Competitors Inform You About Best Asbestos Lawyers
Image
5 Killer Quora Questions On Mental Health Clinic
Image
Panduan Menang Main Judi Online Kartu
Image
New And Innovative Concepts Happening With Accident & Injury Lawyers
Image
Find Out More About Cheap 2 Seater Sofa While Working From At Home
Image
Cat Flap Installation - How to Choose and Install a Cat Flap
Image
How Couch Sale UK Has Transformed My Life The Better
Image
We've Had Enough! 15 Things About Truck Key Replacement Near Me We're Sick Of Hearing
Image
15 Of The Best Documentaries On Accident Lawyer Savannah
Image
Jenis Keuntungan Judi Online24jam Deposit Pulsa
Image
Resume/CV Builder

codepen
Image
15 Amazing Facts About Accident Lawyers In Virginia That You've Never Heard Of
Image
10 Meetups About Private Mental Health Clinic London You Should Attend
Image
How to Train Your Cat to Use a Window Cat Flap
Image
tehnik raih kemenangan di games slots online playtech
Image
kekeliruan yang banyak dikerjakan dalam slots online
Image
Mental Health: 11 Things You're Leaving Out
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .