Learning Go by examples: part 12 - Deploy Go apps in Go with Pulumi

Aurélie Vache - Aug 21 '23 - - Dev Community

In previous articles we created a HTTP REST API server, and another kind of applications and we deployed them manually locally.

Deploying applications manually is cool but today, we will try and use Pulumi to deploy, programmatically, in Go of course ^^, our awesome apps.

Ready?

Pulumi

Pulumi

Pulumi is an Infrastructure as code (IasC) tool that allows you to build your infrastructures with a programming language. It supports a variety of programming languages: Python, Node.js, Go, Java, .Net...

Like Terraform, Pulumi have an architecture based on providers/plugins. There are official providers (AWS, GCP, Kubernetes, Docker...) but it is possible to create our own providers too.

How Pulumi is working?

Concretely, users defined the desired state in Pulumi programs and Pulumi create the desired resources.

To provision, update or delete infrastructures, Pulumi have an intuitive Command Line Interface (CLI). If you are familiar with Docker Compose CLI and Terraform CLI, you will adopt Pulumi CLI too.

Let's install the Pulumi CLI.
In this guide we will install it with brew but you can install in many ways, follow the installation guide. 



$ brew install pulumi/tap/pulumi
Enter fullscreen mode Exit fullscreen mode

Let's check the CLI is correctly installed locally:



$ pulumi version
v3.77.1
Enter fullscreen mode Exit fullscreen mode

Pre-requisites

In this article, we will use the feature to save locally the Pulumi state. If you want to save it in the Cloud, don't use the --local flag in pulumi login command, instead create an account on Pulumi and retrieve an access token.

On my side I am using a free Pulumi account and save locally the state.

What do we want?

An Infrastructure as Code (IaC) tool is originally used to deploy infrastructures in Cloud providers but we can also handle (deploy, change and destroy) our Go apps and that's what we will do in this article.

We will deploy two applications:

  • our cute Gophers API that list existing Gophers, display information, create, update and delete a Gopher
  • a Node.js HMI Gophers API Watcher that displays our cute Gophers (who don't love UI? ^^)

As you maybe know, I like to run apps in containers so we will run our apps in containers.

Pre-requisites - Create a Docker image from a Go app

We will use the Docker provider in Pulumi to deploy and run our applications so before we need to have Docker images with our apps.

Let's do it!

Uh... come on Aurélie ... You want really to explain deployment with Pulumi and Docker images creation in only one article?

Yes, with docker init command we can generate necessary Docker files and create our images easily without headaches or "marabout tips".

I have already explained it in video:

So thanks to the docker init command I generated a Dockerfile, built (for different platform and architecture), tagged and pushed the image.

Our Gophers API is available on Docker Hub (with several tags, depending on your host platform): https://hub.docker.com/r/scraly/gophers-api

And the Gophers API watcher too: https://hub.docker.com/r/scraly/gophers-api-watcher

Initialization

First of all, we can create our repository in GitHub (in order to share and open-source it).

For that, I logged in GitHub website, clicked on the repositories link, click on "New" green button and then I created a new repository called “pulumi-gophers”.

Now, in your local computer, git clone this new repository where you want:



$ git clone https://github.com/scraly/pulumi-gophers.git
$ cd pulumi-gophers
Enter fullscreen mode Exit fullscreen mode

Login (the state will be saved locally):



$ pulumi login --local

Logged in to scraly-pulumigophers-o1kkpsmgr3k as gitpod (file://~)
Enter fullscreen mode Exit fullscreen mode

Initialize our project:



$ pulumi new go --force

This command will walk you through creating a new Pulumi project.

Enter a value or leave blank to accept the (default), and press <ENTER>.
Press ^C at any time to quit.

project name: (pulumi-gophers) 
project description: (A minimal Go Pulumi program) 
Created project 'pulumi-gophers'

Please enter your desired stack name.
To create a stack in an organization, use the format <org-name>/<stack-name> (e.g. `acmecorp/dev`).
stack name: (gophers) 
Created stack 'gophers'

Installing dependencies...

go: downloading github.com/pulumi/pulumi/sdk/v3 v3.60.1
go: downloading golang.org/x/net v0.7.0
...
go: downloading github.com/kr/text v0.2.0
Finished installing dependencies

Your new project is ready to go! 

To perform an initial deployment, run `pulumi up`
Enter fullscreen mode Exit fullscreen mode

The command create a gophers stack and the code organization of your project:



$ tree
.
├── go.mod
├── go.sum
├── main.go
├── Pulumi.yaml
└── README.md
Enter fullscreen mode Exit fullscreen mode

Create our application (Pulumi Go program)

Our application will:

  • retrieve Gophers API Docker image
  • retrieve Gophers API Watcher Docker image
  • create a Docker network (thanks to that our containers should communicate with each other)
  • create a gophers-api container and run it
  • create a gophers-api-watcher containr and run it

For that, we will use the Pulumi official Docker provider.

Let's install Pulumi SDK and Pulumi Docker provider in order to use it in our code:



$ go get github.com/pulumi/pulumi-docker/sdk/v3@v3.6.1
$ go get github.com/pulumi/pulumi/sdk/v3@v3.44.2
Enter fullscreen mode Exit fullscreen mode

Good, now we can create a main.go file and copy/paste the following code into it.

Go code is organized into packages. So, first, we initialize the package, called main, and all dependencies/libraries we need to import and use in our main file:



package main

import (
    "fmt"

    "github.com/pulumi/pulumi-docker/sdk/v3/go/docker"
    "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
    "github.com/pulumi/pulumi/sdk/v3/go/pulumi/config"
)
Enter fullscreen mode Exit fullscreen mode

With the imports added, you can start creating the main() function that contains the intelligence of our app:



func main() {
    pulumi.Run(func(ctx *pulumi.Context) error {
Enter fullscreen mode Exit fullscreen mode

When you will execute pulumi up command to deploy our applications, Pulumi will run all the code we will wrote in the pulumi.Run(func(ctx *pulumi.Context) error { block code.

Let's write our program.

First, we define the configuration: 




        //Configuration
        protocol := "http://"
        //tag := "latest" //for linux/arm64
        tag := "linux-amd64" //if you run this program in a linux/amd64 arch, on GitPod for example ;-)

        cfg := config.New(ctx, "")
        gophersAPIPort := cfg.RequireFloat64("gophersAPIPort")
        gophersAPIWatcherPort := cfg.RequireFloat64("gophersAPIWatcherPort")
Enter fullscreen mode Exit fullscreen mode

Then, we pull the Gophers API Docker image from the Docker Hub with the tag you defined in the configuration:



        // Pull the Gophers API image
        gophersAPIImageName := "gophers-api"
        gophersAPIImage, err := docker.NewRemoteImage(ctx, fmt.Sprintf("%v-image", gophersAPIImageName), &docker.RemoteImageArgs{
            Name: pulumi.String("scraly/" + gophersAPIImageName + ":" + tag),
        })
        if err != nil {
            return err
        }
        ctx.Export("gophersAPIDockerImage", gophersAPIImage.Name)
Enter fullscreen mode Exit fullscreen mode

Then, we pull the Gophers API Watcher Docker image from the Docker Hub with the tag you defined in the configuration:



        // Pull the Gophers API Watcher (frontend/UI) image
        gophersAPIWatcherImageName := "gophers-api-watcher"
        gophersAPIWatcherImage, err := docker.NewRemoteImage(ctx, fmt.Sprintf("%v-image", gophersAPIWatcherImageName), &docker.RemoteImageArgs{
            Name: pulumi.String("scraly/" + gophersAPIWatcherImageName + ":" + tag),
        })
        if err != nil {
            return err
        }
        ctx.Export("gophersAPIWatcherDockerImage", gophersAPIWatcherImage.Name)
Enter fullscreen mode Exit fullscreen mode

Our containers will need to connect to each other, so we will need to create a Docker Network:



        // Create a Docker network
        network, err := docker.NewNetwork(ctx, "network", &docker.NetworkArgs{
            Name: pulumi.String(fmt.Sprintf("services-%v", ctx.Stack())),
        })
        if err != nil {
            return err
        }
        ctx.Export("containerNetwork", network.Name)
Enter fullscreen mode Exit fullscreen mode

Create the Gophers API container:




        // Create the Gophers API container
        _, err = docker.NewContainer(ctx, "gophers-api", &docker.ContainerArgs{
            Name:  pulumi.String(fmt.Sprintf("gophers-api-%v", ctx.Stack())),
            Image: gophersAPIImage.RepoDigest,
            Ports: &docker.ContainerPortArray{
                &docker.ContainerPortArgs{
                    Internal: pulumi.Int(gophersAPIPort),
                    External: pulumi.Int(gophersAPIPort),
                },
            },
            NetworksAdvanced: &docker.ContainerNetworksAdvancedArray{
                &docker.ContainerNetworksAdvancedArgs{
                    Name: network.Name,
                    Aliases: pulumi.StringArray{
                        pulumi.String(fmt.Sprintf("gophers-api-%v", ctx.Stack())),
                    },
                },
            },
        })
        if err != nil {
            return err
        }
Enter fullscreen mode Exit fullscreen mode

Create the Gophers API Watcher container:



        // Create the Gophers API Watcher container
        _, err = docker.NewContainer(ctx, "gophers-api-watcher", &docker.ContainerArgs{
            Name:  pulumi.String(fmt.Sprintf("gophers-api-watcher-%v", ctx.Stack())),
            Image: gophersAPIWatcherImage.RepoDigest,
            Ports: &docker.ContainerPortArray{
                &docker.ContainerPortArgs{
                    Internal: pulumi.Int(gophersAPIWatcherPort),
                    External: pulumi.Int(gophersAPIWatcherPort),
                },
            },
            Envs: pulumi.StringArray{
                pulumi.String(fmt.Sprintf("PORT=%v", gophersAPIWatcherPort)),
                pulumi.String(fmt.Sprintf("HTTP_PROXY=backend-%v:%v", ctx.Stack(), gophersAPIPort)),
                pulumi.String(fmt.Sprintf("PROXY_PROTOCOL=%v", protocol)),
            },
            NetworksAdvanced: &docker.ContainerNetworksAdvancedArray{
                &docker.ContainerNetworksAdvancedArgs{
                    Name: network.Name,
                    Aliases: pulumi.StringArray{
                        pulumi.String(fmt.Sprintf("gophers-api-watcher-%v", ctx.Stack())),
                    },
                },
            },
        })
        if err != nil {
            return err
        }

        return nil
    })
}
Enter fullscreen mode Exit fullscreen mode

And that's it! We defined everything we want in our infrastructures: 2 applications running in containers thanks to Pulumi.

The configuration

As you saw, we didn't hardcode the apps port number:



        cfg := config.New(ctx, "")
        gophersAPIPort := cfg.RequireFloat64("gophersAPIPort")
        gophersAPIWatcherPort := cfg.RequireFloat64("gophersAPIWatcherPort")
Enter fullscreen mode Exit fullscreen mode

Instead we define them as config parameters. The program will get them in a Pulumi.<your-stack-name>.yaml file.

To define them and generate the config file, execute the following commands:



$ pulumi config set gophersAPIPort 8080
$ pulumi config set gophersAPIWatcherPort 8000
Enter fullscreen mode Exit fullscreen mode

After editing our main.go file and defining our configuration fields and values, it's time to ask to Go to download and install all the Go providers and dependencies:



$ go mod tidy
Enter fullscreen mode Exit fullscreen mode

Let's deploy our apps

Now we can deploy our apps, to do that just execute the pulumi up comand.
This will display the plan/the preview of the desireed state. A prompt will ask you to choose the stack (dev by default) and to confirm of you want to perform/apply the changes.



$ pulumi up
Please choose a stack, or create a new one: gophers
Previewing update (gophers)

View in Browser (Ctrl+O): https://app.pulumi.com/scraly/pulumi-gophers/gophers/previews/cb2a49a5-e17e-4e58-9525-a1931b214b23

     Type                         Name                       Plan       
 +   pulumi:pulumi:Stack          pulumi-gophers-gophers     create     
 +   ├─ docker:index:RemoteImage  gophers-api-watcher-image  create     
 +   ├─ docker:index:RemoteImage  gophers-api-image          create     
 +   ├─ docker:index:Network      network                    create     
 +   ├─ docker:index:Container    gophers-api-watcher        create     
 +   └─ docker:index:Container    gophers-api                create     


Outputs:
    containerNetwork            : "services-gophers"
    gophersAPIDockerImage       : "scraly/gophers-api:linux-amd64"
    gophersAPIWatcherDockerImage: "scraly/gophers-api-watcher:linux-amd64"

Resources:
    + 6 to create

Do you want to perform this update? yes
Updating (gophers)

View in Browser (Ctrl+O): https://app.pulumi.com/scraly/pulumi-gophers/gophers/updates/3

     Type                         Name                       Status              
 +   pulumi:pulumi:Stack          pulumi-gophers-gophers     created (9s)        
 +   ├─ docker:index:Network      network                    created (2s)        
 +   ├─ docker:index:RemoteImage  gophers-api-watcher-image  created (8s)        
 +   ├─ docker:index:RemoteImage  gophers-api-image          created (5s)        
 +   ├─ docker:index:Container    gophers-api                created (0.97s)     
 +   └─ docker:index:Container    gophers-api-watcher        created (1s)        


Outputs:
    containerNetwork            : "services-gophers"
    gophersAPIDockerImage       : "scraly/gophers-api:linux-amd64"
    gophersAPIWatcherDockerImage: "scraly/gophers-api-watcher:linux-amd64"

Resources:
    + 6 created

Duration: 13s
Enter fullscreen mode Exit fullscreen mode

We can check if images have been successfully pulled from the registry:



$ docker image ls
REPOSITORY                   TAG           IMAGE ID       CREATED        SIZE
scraly/gophers-api-watcher   linux-amd64   ee8c626fdeab   3 hours ago    288MB
scraly/gophers-api           linux-amd64   83e5cf52694c   3 hours ago    22.6MB
scraly/gophers-api-watcher   latest        4de2009ea463   23 hours ago   286MB
scraly/gophers-api           latest        0e32fa8f8e18   4 months ago   22.3MB
Enter fullscreen mode Exit fullscreen mode

And check if containers are running as well:



 $ docker container ls
CONTAINER ID   IMAGE                        COMMAND                  CREATED         STATUS         PORTS                    NAMES
ba0b23d0af11   scraly/gophers-api-watcher   "docker-entrypoint.s…"   9 minutes ago   Up 9 minutes   0.0.0.0:8000->8000/tcp   gophers-api-watcher-gophers
7b6d448f3d01   scraly/gophers-api           "/bin/server"            9 minutes ago   Up 9 minutes   0.0.0.0:8080->8080/tcp   gophers-api-gophers
Enter fullscreen mode Exit fullscreen mode

Let's test it locally

Ahh I like this moment where we will be able to test what we have created!

First, let's test our API. 



$ curl localhost:8080/gophers
[{"displayname":"5th Element","name":"5th-element","url":"https://raw.githubusercontent.com/scraly/gophers/main/5th-element.png"}]
Enter fullscreen mode Exit fullscreen mode

Cool, and now let's display our cute HMI, for that go to localhost:8000/demo with your favorite browser:

Gophers API Watcher

Awesome, it's working (and our Gopher is so cute ^^)!

Now, if you want, you can now play with the Gophers API (add and edit existing gophers) and watch them appears in the cute HMI done by Horacio Gonzalez 🙂.

Cleanup

To easily destroy created resources, you can use pulumi destroy command.



$ pulumi destroy
Please choose a stack: gophers
Previewing destroy (gophers)

View in Browser (Ctrl+O): https://app.pulumi.com/scraly/pulumi-gophers/dev/previews/2344bad2-xxxx-xxxx-xxxx-846c828f7102

     Type                         Name                       Plan       
 -   pulumi:pulumi:Stack          pulumi-gophers-gophers     delete     
 -   ├─ docker:index:Container    gophers-api                delete     
 -   ├─ docker:index:Container    gophers-api-watcher        delete     
 -   ├─ docker:index:Network      network                    delete     
 -   ├─ docker:index:RemoteImage  gophers-api-image          delete     
 -   └─ docker:index:RemoteImage  gophers-api-watcher-image  delete     


Outputs:
  - containerNetwork            : "services-gophers"
  - gophersAPIDockerImage       : "scraly/gophers-api:linux-amd64"
  - gophersAPIWatcherDockerImage: "scraly/gophers-api-watcher:linux-amd64"

Resources:
    - 6 to delete

Do you want to perform this destroy? yes
Destroying (gophers)

View in Browser (Ctrl+O): https://app.pulumi.com/scraly/pulumi-gophers/gophers/updates/2

     Type                         Name                       Status              
 -   pulumi:pulumi:Stack          pulumi-gophers-gophers     deleted             
 -   ├─ docker:index:Container    gophers-api-watcher        deleted (0.26s)     
 -   ├─ docker:index:Container    gophers-api                deleted (0.51s)     
 -   ├─ docker:index:RemoteImage  gophers-api-watcher-image  deleted (0.64s)     
 -   ├─ docker:index:Network      network                    deleted (2s)        
 -   └─ docker:index:RemoteImage  gophers-api-image          deleted (0.86s)     


Outputs:
  - containerNetwork            : "services-gophers"
  - gophersAPIDockerImage       : "scraly/gophers-api:linux-amd64"
  - gophersAPIWatcherDockerImage: "scraly/gophers-api-watcher:linux-amd64"

Resources:
    - 6 deleted

Duration: 6s

The resources in the stack have been deleted, but the history and configuration associated with the stack are still maintained. 
If you want to remove the stack completely, run `pulumi stack rm gophers`.
Enter fullscreen mode Exit fullscreen mode

Known issues

creating failed

The first time you will "play" with Pulumi, specially the Docker provider, you can face to the issue I had:



Do you want to perform this update? yes
Updating (gophers)

View in Browser (Ctrl+O): https://app.pulumi.com/scraly/pulumi-gophers/gophers/updates/1

     Type                         Name                       Status              
 +   pulumi:pulumi:Stack          pulumi-gophers-gophers     created (9s)        
 +   ├─ docker:index:Network      network                    created (2s)        
 +   ├─ docker:index:RemoteImage  gophers-api-watcher-image  created (8s)        
 +   ├─ docker:index:RemoteImage  gophers-api-image          created (5s)        
 +   ├─ docker:index:Container    gophers-api                ** creating failed**    1 error
Enter fullscreen mode Exit fullscreen mode

creating failed... OK...

I admit that when you have this problem, it can be disappointing.

A way, to find what is happening is to add the --debug flag to the pulumi up command.

Another way is to try, simply, to run the container locally:



$ docker run scraly/gophers-api:latest 
WARNING: The requested image's platform (linux/arm64/v8) does not match the detected host platform (linux/amd64/v3) and no specific platform was requested
exec /usr/local/bin/docker-entrypoint.sh: exec format error
Enter fullscreen mode Exit fullscreen mode

As you can see, I built and pushed my image into another platform/architecture (mac m1) than the server/machine I'm running it on (ubuntu).

fire gopher

So the solution was to use, on my mac, docker buildx build command with the platform flag, to build and push the image with the good platform, like this:




$ docker buildx build --platform linux/amd64 -t scraly/gophers-api:linux-amd64 . --push
Enter fullscreen mode Exit fullscreen mode




What am I thinking about Pulumi

Before to conclude this blog post I need to tell you my thought about Pulumi.

I am doing a lot of Terraform since 2017, I trained my ex colleagues, used it in many projects, for several Cloud providers (AWS, OVHcloud...), and even maintaning a Terraform provider daily. So I admit I thought I don't need Pulumi because I know and uses already an IaC tool.

As you know, I am curious, so I wanted, since several years to test it but without time to do it. I wanted to try it for a concrete need and I found one: deploying a Kubernetes cluster & a node pool (and other resources) on OVHcloud.

The journey was not easy, I had several troubles (mainly the tf2pulumi converter and then the existing Pulumi OVH community provider). I lost several hours and days, but didn't give up and thanks to Engin we finally found solutions.

I think the experience with Pulumi you can have, will depends on the provider(s) you will use.

If you already know Terraform, you will easily understand the Pulumi concepts. If you are a developer, I think it can be easier for you to write your infrastructure in your favorite language, instead of write it in HCL (Hashicorp Configuration Language).
But, not all the company who have a Terraform provider have a Pulumi provider yet, it's a fact.

On my side the journey was not easy, but I am stubborn and hopefully Engin Diri helped me. Without his help, I would probably have given up or postponed my umpteenth attempt for several months.

Conclusion

As you have seen in this article and previous articles, it's possible to create applications in Go and even now deploying them in Go too with Pulumi.

All the code of our app is available in: https://github.com/scraly/pulumi-gophers

So if you want, now, you can use Pulumi, or another tool, to deploy your apps and automatize this task.
For me, there is no magic wand, no magic tool and technology better then other, depending on your team, the context, the need, use the technology, the tool or the language you want :-).

In the following articles we will create others kind/types of applications in Go.

Hope you'll like it.

Image
A Look Into The Secrets Of Small Built In Microwave
Image
Unlock the Secrets to Maximizing Your Winnings with Altcoins in Cryptocasinos

crypto, blockchain, web3, eth
Image
Top Salesforce Consulting Trends You Can’t Afford to Miss

salesforce, softwaredevelopment, webdev, ai
Image
Discover the Thrilling Evolution of Live Dealer

gambling, crypto, blockchain, web3
Image
Error: externally-managed-environment on python package installation

python, tutorial
Image
keuntungan main pada agen slots online memiliki pengalaman
Image
DATA EXPLORATION WITH PANDAS: A BEGINNER'S GUIDE

datascience, python, pandas
Image
The Role of Ghostwriting in Politics: Crafting Public Statements and Speeches
Image
Bank Accounts Types - Learn The Differences Before Opening An Account
Image
Social Injustice: Causes, Consequences, And Solutions
Image
Understanding Namespaces in Kubernetes: Imperative vs Declarative Approaches & Resources Not Specific to Namespaces
Image
What Is Social Justice? Definitions And Examples Of Social Injustice
Image
Akshay Kumar: 5 Nice Health Classes Everyone Ought To Study From Akshay Kumar
Image
Using a Cat Flap in Bifold Doors
Image
Unlocking the Internet: The Ultimate Guide to the Best Budget VPNs for Every Wallet
Image
Looking for a Dynamic Environment? Try BPO!
Image
Kryptowaluta: Przyszłość Finansów czy Spekulacyjna Bańka?
Image
The 10 Most Infuriating Personal Injury Claim-Related FAILS Of All Time Could Have Been Prevented
Image
🔍✨ Demystifying Java Bytecode: A Peek Under the Hood of the JVM 🔧🛠️
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .