AWS Lambda with Go, initial boilerplate

szymon-szym - Oct 16 - - Dev Community

Photo by Lukáš Vaňátko on Unsplash

Introduction

Go has always been one of my favorite languages due to its simplicity. Lately, I decided to figure out what it takes to create a simple boilerplate serverless project with lambda functions written in Go. I was curious about the tooling and developer experience.

Goal

I want to create a REST API, that uses postgres db as a data layer. My initial requirements are the following

  • Spec to be defined with OpenApi, and models generated from it
  • Use AWS SAM
  • Each endpoint is to be handled by a separate lambda function
  • Local development as easy as possible
  • Same for deployment

Prerequisites

You would need to have Go installed, as well as AWS SAM. If you deploy your project to AWS you might be billed for the resources you are creating, so remember to clear your resources when you don't need them.
For DB I use supabase

Building the project

The code is available in this repo

Let's start by running sam init. I picked the Hello World template, Go with provided al.2023 env. Previously there was a managed runtime for Go, but nowadays it is deprecated.

OpenApi schema

Having API schema defined as OpenApi spec has a few obvious advantages. We can use it to generate documentation, create clients, servers, etc. I also use it for defining the shape of the AWS HttpApi Gateway.

My schema is straightforward. The only interesting part is x-amazon-apigateway-integration property, which allows connection with lambda integration. The setup is language-agnostic.

You can find schema file in the repo

SAM template

HttpApi and secret 

# template.yaml
# ....
ItemsAPI:
    Type: AWS::Serverless::HttpApi
    Properties:
      StageName: Prod
      DefinitionBody:
        'Fn::Transform':
          Name: 'AWS::Include'
          Parameters:
            Location: './api.yaml'
      FailOnWarnings: false
DBSecret:
    Type: AWS::SecretsManager::Secret
    Properties:
      Name: my-db-secret
      Description: Postgres config string
      SecretString: 'host=172.17.0.1 port=5431 user=admin password=root dbname=lambdas sslmode=disable'
# ....
Enter fullscreen mode Exit fullscreen mode

As mentioned above, there is nothing specific to Go here. The HttpApi Gateway is created based on OpenApi.

There is also a secret for storing connection string. I will update its value after the deployment

Lambda Function

AWS SAM support for Go is pretty awesome. I can point the CodeUri to the folder with lambda handler and define the build method as go1.x

Lambda functions built in Go use provided.al2023 runtime, as they produce a single self-containing binary.

The definition of the function looks like this:

# template.yaml
# ....
  GetItemFunction:
    Type: AWS::Serverless::Function
    Metadata:
      BuildMethod: go1.x
    Properties:
      Tracing: Active
      CodeUri: lambda_handlers/cmd/get_item/
      Handler: bootstrap
      Runtime: provided.al2023
      Architectures:
        - x86_64
      Environment:
        Variables:
          DB_SECRET_NAME: !Ref DBSecret
          API_STAGE: Prod
      Events:
        HttpApiEvents:
          Type: HttpApi
          Properties:
            Path: /item/{id}
            Method: GET
            ApiId: !Ref ItemsAPI
      Policies: 
        - AWSLambdaBasicExecutionRole
        - AWSSecretsManagerGetSecretValuePolicy:
            SecretArn: !Ref DBSecret
# ....
Enter fullscreen mode Exit fullscreen mode

Thanks to the SAM magic, the connection between HttpApi Gateway and the lambda function will be established with all required permissions.

Function code

Project structure

To be honest, the folder structure is probably not idiomatic. But I tried to follow general Go patterns

lambda_handlers
|--/api
|--/cmd
|--/internal
|--/tools
|--go.mod
|--go.sum
Enter fullscreen mode Exit fullscreen mode

cmd is the main folder with actual lambda handlers
internal holds the code shared between handlers
tools defines additional tools to be used in the projects
api for openapi generator config and generated models

Function handler

The initial boilerplate for the lambda handler looks like this:

// ...
func handleRequest(ctx context.Context, request events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error) {
    // handler logic
}

func main() {
    lambda.Start(handleRequest)
}
Enter fullscreen mode Exit fullscreen mode

Usually, the first question to ask is where to put the initialization of AWS SDK clients, database connection, and other things, we want to deal with during the cold start.

We have options here. First is to follow the pattern from AWS documentation example and initialize services inside the init() function. I don't like this approach, because it makes it harder to use the handler in the unit tests.

Thanks to the fact that lambda.Start() method takes a function as an input, I can wrap it in the custom struct, and initialize it with the services I need. In my case, the code looks this way:

package main

// imports ...

type GetItemsService interface {
    GetItem(id int) (*api.Item, error)
}

type LambdaHandler struct {
    svc GetItemsService
}

func InitializeLambdaHandler(svc GetItemsService) *LambdaHandler {
    return &LambdaHandler{
        svc: svc,
    }
}

func (h *LambdaHandler) HandleRequest(ctx context.Context, request events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error) {

    strId, err := helpers.GetPathParameter("id", request.PathParameters)

    if err != nil {
        return helpers.SendResponse("id is required", 400), nil
    }

    id, err := strconv.Atoi(*strId)

    if err != nil {
        return helpers.SendResponse("id must be an integer", 400), nil
    }

    log.Printf("id: %d", id)

    result, err := h.svc.GetItem(id)

    if err != nil {
        if err.Error() == "Item not found" {
            return helpers.SendResponse("Item not found", 404), nil
        }
        return helpers.SendResponse("error", 500), nil
    }

    jsonRes, err := json.Marshal(result)

    if err != nil {
        log.Printf("error marshalling response: %s", err.Error())
        return helpers.SendResponse("internal server error", 500), nil
    }

    return helpers.SendResponse(string(jsonRes), 200), nil
}

func main() {

    dbSecretName := os.Getenv("DB_SECRET_NAME")

    log.Printf("dbSecretName: %s", dbSecretName)

    cfg, err := awssdkconfig.InitializeSdkConfig()

    if err != nil {
        log.Fatal(err)
    }

    secretsClient := awssdkconfig.InitializeSecretsManager(cfg)

    connString, err := secretsClient.GetSecret(dbSecretName)

    if err != nil {
        log.Fatal(err)
    }

    conn, err := db.InitializeDB(connString)

    if err != nil {
        log.Fatal(err)
    }

    defer conn.Close()

    log.Println("successfully connected to db")

    svc := items.InitializeItemsService(conn)

    handler := InitializeLambdaHandler(svc)

    lambda.Start(handler.HandleRequest)
}
Enter fullscreen mode Exit fullscreen mode

In the main() function (which runs during cold start) I get the secret from secretsmanager and then initialize the connection with DB. Both functionalities are defined inside internal folders as common helpers so they can be reused in other handlers. Finally my ItemsService is initialized with the created db connection, and used for creating a lambda handler.

HandleRequest parses ID from the path parameter, and calls ItemsService to get an item from DB.

Internal modules

As the function is simple, there is not much business logic around. The ItemsServise simply calls db for the specific item

package items

import (
    "database/sql"
    "errors"
    api "lambda_handlers/api"
    "log"
)

type ItemsService struct {
    conn *sql.DB
}

func InitializeItemsService(conn *sql.DB) *ItemsService {
    return &ItemsService{
        conn: conn,
    }
}

func (svc ItemsService) GetItem(id int) (*api.Item, error) {

    log.Printf("Getting item id %v", id)

    query := `SELECT * FROM items WHERE id = $1`

    var item api.Item

    err := svc.conn.QueryRow(query, id).Scan(&item.Id, &item.Name, &item.Price)

    log.Printf("Got item id %v", id)

    if err != nil {
        log.Printf("Error getting item %v: %v", id, err)
        if err == sql.ErrNoRows {
            return nil, errors.New("Item not found")
        }
        return nil, err
    }

    return &item, nil

}
Enter fullscreen mode Exit fullscreen mode

At this point, we don't need anything more here.

Tools

My goal is to use additional tools, which could be attached to the project dependencies, so there is no need to rely on tools installed on the developer's machine.

In Go one way of doing it is to keep oapi-codegen in the tools package

//go:build tools
// +build tools

package main

import (
    _ "github.com/oapi-codegen/oapi-codegen/v2/cmd/oapi-codegen"
)
Enter fullscreen mode Exit fullscreen mode

And call it from inside api_gen.go

//go:generate go run github.com/oapi-codegen/oapi-codegen/v2/cmd/oapi-codegen --config=cfg.yaml ../../api.yaml

package api
Enter fullscreen mode Exit fullscreen mode

This way I can run go generate without installing oapi-codegen binaries separately.

Local development

Build

My build process requires two steps: generating models from OpenAPI, and building the projects itself. I let AWS SAM deal with the latter.

Here is my Makefile

.PHONY: build local deploy

generate:
    cd lambda_handlers && go generate ./...

build: generate
    rm -rf .aws-sam
    sam build

local: build
    sam local start-api --env-vars parameters.json

deploy: build
    sam deploy
Enter fullscreen mode Exit fullscreen mode

Initial deployment

For me, the easiest way to test API Gateway locally is to run sam local start-api
Since our function relies on environment variables, I created paramters.json file to pass env vars to sam local

When developing for serverless, at some point you might want to start using cloud resources even for local development. In my case, I will utilize the secrets manager right away to store the connection string for DB. It means, that I need to deploy the stack first, so I can use it in the local development.

I run make deploy but for now I don't check the whole deployment, just grab a secret name from the console. I also need to update the secret in the console, so it holds the correct connection string.

For testing, I have created a DB on supabase and seeded it with a few dummy records

Local testing

After running make local I can test API locally

Image description

Since Go is a compiled language, after each change I need to rebuild the project and run start-api again. Considering the Go compiler's amazing speed, it is not a big deal.

Test on AWS

The API Gateway URL was printed out in the console after deployment, and it can be also grabbed from the AWS console directly.

I call the endpoint, and it works as expected:

Image description

Cold start is a bit long, as initialization takes ~300 ms, mostly because it includes taking the secret and establishing a connection to the DB. But to be honest it is more than a decent result.

Summary

The given project is a starting point for creating a serverless REST API in Go. It uses OpenAPI for the schema and AWS SAM for managing deployment and local testing.

I've used an external postgres db and AWS SDK for getting connection string from secrets manager.

There are also unit tests for lambda handler and items service

Most of the time I've spent configuring the AWS part, which would be the same for all languages. The Go code is pretty straightforward (for this simple use case).

Image
Weather App with Tolgee
Image
Card Fight: A Python Terminal Game

python, gamedev, webdev
Image
The Top 5 Reasons Why People Are Successful In The Wall Mounted Electric Fireplace Heater Industry
Image
Are You Sick Of Bifold Door Repair? 10 Inspirational Sources That Will Invigorate Your Love
Image
tehnik menang slots online dengan sangat kencang
Image
최고의 카지노 사이트 탐방: 바다를 건너는 바카라와 라이브 카지노의 매력!
Image
What Experts Say You Should Be Able To
Image
15 Reasons You Shouldn't Be Ignoring Built In Electric Fireplace Wall
Image
Cat Flaps For French Doors
Image
This Is What Mental Health Practitioners Will Look In 10 Years Time
Image
What's The Job Market For Asbestos Lawyers Professionals Like?
Image
How to Tackle Discord Unstable Internet Connection for a Better Experience

internetconnection, proxy
Image
VSCode language mode association for subfolders and special characters folders

vscode, python
Image
16 Facebook Pages You Must Follow For Self Propelled Wheelchairs Marketers
Image
A The Complete Guide To Self Propelled Wheelchair Uk From Start To Finish
Image
Trik Jadi Bandar Judi Online Sangat Gampang
Image
111111
Image
Discover the Best German Proxy Websites for Secure Browsing

swiftproxy, germanproxy
Image
Everything You Need To Know About Asbestos Mesothelioma
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .