Learn How to Setup a CI/CD Pipeline from Scratch

Pavan Belagatti - Mar 1 '23 - - Dev Community

In this tutorial, we will take an example of a Go application and setup a CI/CD pipeline.

Go is becoming increasingly popular amongst developers for its ability to simplify and secure the building of modern applications. The language was created by Google and has gained traction due to its open-source nature and ability to write programs in the Go language. Go also provides users with the freedom to build their own front-end websites and applications, as well as making it easy to develop, maintain and use. Enterprises can rely on Go to help build and scale cloud computing systems while enjoying its powerful concurrency features. Furthermore, Go offers high performance without utilizing too many resources.

Today, we will create a simple Go application and set up a CI/CD pipeline for the same. Let's Go!

Prerequisites

  • Create a free SingleStore account. For this tutorial, we'll be using SingleStore as our database solution. SingleStore is a high-performance, in-memory database that supports both SQL and NoSQL data models.

  • Create a free Harness cloud account to setup CI/CD

  • Download & install Go quickly from here

  • Kubernetes cluster access from any cloud provider to deploy our application (you can also use Minikube or Kind to create a single node cluster).

  • Docker, preferably Docker Desktop

Tutorial

The example repository is accessible here; feel free to fork it or just follow along. I will not go into many details about the application code itself. It is a sample “Hello World” app that prints the text “Hello World” on the local host 8080.

Here is the code for the main.go file:



package main

import (
    "fmt"
    "log"
    "net/http"
)

func homePage(w http.ResponseWriter, r *http.Request) {
    fmt.Fprintf(w, "Home Page")
}

func wsEndpoint(w http.ResponseWriter, r *http.Request) {
    fmt.Fprintf(w, "Hello World")
}

func setupRoutes() {
    http.HandleFunc("/", homePage)
    http.HandleFunc("/ws", wsEndpoint)
}

func main() {
    fmt.Println("Hello World")
    setupRoutes()
    log.Fatal(http.ListenAndServe(":8080", nil))
}


Enter fullscreen mode Exit fullscreen mode

The application also has a test file main_test.go with simple test case.



package main

import (
    "net/http"
    "net/http/httptest"
    "testing"
)

func TestHomePage(t *testing.T) {
    req, err := http.NewRequest("GET", "/", nil)
    if err != nil {
        t.Fatal(err)
    }
    rr := httptest.NewRecorder()
    handler := http.HandlerFunc(homePage)
    handler.ServeHTTP(rr, req)
    if status := rr.Code; status != http.StatusOK {
        t.Errorf("handler returned wrong status code: got %v want %v",
            status, http.StatusOK)
    }
    if rr.Body.String() != "Home Page" {
        t.Errorf("handler returned unexpected body: got %v want %v",
            rr.Body.String(), "Home Page")
    }
}

func TestWsEndpoint(t *testing.T) {
    req, err := http.NewRequest("GET", "/ws", nil)
    if err != nil {
        t.Fatal(err)
    }
    rr := httptest.NewRecorder()
    handler := http.HandlerFunc(wsEndpoint)
    handler.ServeHTTP(rr, req)
    if status := rr.Code; status != http.StatusOK {
        t.Errorf("handler returned wrong status code: got %v want %v",
            status, http.StatusOK)
    }
    if rr.Body.String() != "Hello World" {
        t.Errorf("handler returned unexpected body: got %v want %v",
            rr.Body.String(), "Hello World")
    }
}


Enter fullscreen mode Exit fullscreen mode

The Dockerfile you see in the repo will be used to build and push our application as an image to the Docker Hub.



FROM golang:1.16.4-buster AS builder

ARG VERSION=dev

WORKDIR /go/src/app
COPY main.go .
RUN go build -o main -ldflags=-X=main.version=${VERSION} main.go 

FROM debian:buster-slim
COPY --from=builder /go/src/app/main /go/bin/main
ENV PATH="/go/bin:${PATH}"
CMD ["main"]


Enter fullscreen mode Exit fullscreen mode

The next thing is we will build the image and push it to the Docker Hub using the command,



docker buildx build --platform=linux/arm64 --platform=linux/amd64 -t docker.io/<docker hub username>/<image name>:<tag> --push -f ./Dockerfile .


Enter fullscreen mode Exit fullscreen mode

Once the build and push are successful, you can confirm it by going to your Docker Hub account.

Golang image on DockerHub

We will be deploying our application on a Kubernetes cluster.
You can see the deployment.yaml and service.yaml files in the forked repo, which define the deployment and service to help us deploy and expose our application. At this point, make sure your Kubernetes cluster is up and running.

Our deployment.yaml file is shown below



apiVersion: apps/v1
kind: Deployment
metadata:
  name: go-app-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: go-app
  template:
    metadata:
      labels:
        app: go-app
    spec:
      containers:
      - name: go-app
        image: pavansa/golang-hello-world:latest
        ports:
        - containerPort: 8080
        env:
        - name: PORT
          value: "8080"


Enter fullscreen mode Exit fullscreen mode

The service.yaml file is shown below,



apiVersion: v1
kind: Service
metadata:
  name: go-app-service
spec:
  selector:
    app: go-app
  ports:
    - name: http
      protocol: TCP
      port: 80
      targetPort: 8080
  type: LoadBalancer


Enter fullscreen mode Exit fullscreen mode

It is time to set up a Harness account to do CI/CD. Harness is a modern CI/CD platform with AI/ML capabilities.

Create a free Harness account and your first project. Once you sign-up at Harness, you will be presented with the new CI/CD experience and capabilities.

Add the required connectors, such as Harness Delegate, your GitHub repo, Docker Hub account and secrets. Delegate in Harness is a service/software you need to install/run on the target cluster [Kubernetes cluster in our case] to connect your artifacts, infrastructure, collaboration, verification and other providers with the Harness Manager. When you set up Harness for the first time, you install a Harness Delegate.

Continuous Integration

After signing in at Harness, it will first ask you to create a project.

start with the Project

Invite collaborators to the project if you want.
Invite collaborators

Select the Continuous Integration module.
Continuous Integration module

Create your first pipeline.
first pipeline

Connect with your source control management like GitHub, where the application code is present.
scm connect

Next, configure your pipeline with the proper language of the project.
configure pipeline

When you save and continue, you see the below default setup in the pipeline studio.
pipeline studio

When you click on the 'Go Build App' under execution, you will see the below setup details.
go build app

default settings

Let's modify the previous/above step, name it as 'Test Go App' add the following code in the command tab, and save and run the pipeline.

modify pipeline

run pipeline

You should see a successful output:)
successful output

We successfully created a CI pipeline for our application where the build and testing of the code happens. Let's extend the idea of deploying this error-free application code our target environment, i.e Kubernetes.

Continuous Delivery and Deployment

It is time to deploy our Go application, create the deployment stage.
app deploy

Add a name to your stage, select the deployment type as 'Kubernetes', and click 'Set up Stage'.
deploy details

This is what you should be seeing after creating the deploy stage.
service studio

We need to create a service. Hence, click on 'add service'. In the next step, we need to add a name to our service and manifest details.
service details

Click on 'Add manifest' to add the details.
manifest details add

Then select K8S manifest and continue.
k8s manifest

k8s type mani

Specify the K8S manifest store. We know that our manifest files are present in GitHub. Hence select GitHub.
manifest store

Add a new GitHub connector to connect your manifest files.
github connector files

Specify all the details step by step.
source details

Add credentials through inbuilt secrets.
cred secrets

Connect with your delegate.
connect delegate

Make sure the connection of your manifest is successful with your delegate.
successful delegate connection

Now, add the manifest details from your GitHub repo.
mani details from repo
Save everything and continue.
You should see the service with manifest details.

service with manifest details

Your pipeline studio will look like this, with your added service.
service added

Add a new environment, save and continue.
new environment

Similarly, add new infrastructure. Select infrastructure type as 'Kubernetes' and add the cluster details.
infra kube type

cluster details

Save and continue.

In the next step, you need to select the deployment strategy type. We are selecting 'Rolling' as our deployment strategy.
deployment strategy

We are all done and this is how our CD pipeline looks like.
CD pipeline

Now, save everything and run the pipeline.
run CICD pipeline

You should see a successful pipeline execution starting from CI and then CD step by step.
CICD successful

Automate CI/CD

The last step is to automate our CI/CD pipeline by creating Triggers. Let's do it.

In the pipeline studio, you can see the 'Triggers' tab.
Triggers tab

Click the Triggers tab and create a new trigger.
new trigger

Add the GitHub trigger, whenever someone pushes a new code to the main branch, the pipeline should trigger automatically.
GitHub trigger

trigger details
trigger added
more trigger details

Save everything and create the trigger.

You should see the created trigger under the Trigger tab.

Now, let's confirm if our CI/CD is automated and working properly. Add a readme file to our GitHub repo and see if it triggers our pipeline.
updated readme file

You see, the pipeline triggered when a new code commit happened.
automated pipeline

We have successfully automated the CI/CD process for our Go application using Harness.

Also, checkout my other articles on continuous integration and deployment.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .