To read this full New Relic blog, click here.

Are you building modern apps in Golang? This multi-part blog series will help you instrument Golang faster by providing tips, guidance, and best practices with New Relic.

Typically, the content in this blog would be presented in a workshop, but we've made it a blog series so anyone can follow along "hands-on," using sample code, Instruqt, this blog, and videos. For the best experience:

See and run the full code that I'll be walking through in this multi-part series

Follow the hands-on lab in Instruqt

Read along with this blog and follow the videos below

The following two videos will give an overview of Instruqt and some instructions about how to use it if you haven't done that before.

Overview of Instruqt

Top Tips for Instrumenting Golang | New Relic

Instruqt UI navigation

Top Tips for Instrumenting Golang | New Relic

Now that you're set up, let's learn a little more about instrumentation of Golang.

An introduction to Golang instrumentation

Golang is increasingly becoming popular and loved by many developers when building modern cloud-native services. Golang, also known as Go, is a compiled, high-performance language meant to be simple, easy to learn, and effective to use. Since its inception back in 2009, Go has continued to grow significantly, as seen by the latest Stack Overflow 2022 survey.

While developers continue to build more services in Go, finding effective ways to instrument better for Go can be confusing and frustrating. Interpreter languages like PHP, Ruby, Python, and JavaScript, don’t require a pre-runtime translation of code, meaning you can automatically instrument your applications. But, Go, like C and Rust, is a compiled language, meaning it relies on translators that generate machine code from source code. So you can't use a dynamic and automatic instrumentation technique like bytecode injection.

Manual instrumentation is required to get better visibility to find the root cause of an issue.

Tip 1: Learn why you should instrument

The answer is simple: you’ll want to quickly identify issues when there’s a production incident or an outage. It would be great if finding a root cause of an outage was easy and could be followed quickly with a fix and a plan for an effective remediation path. But, the reality is engineers often firefight among an abundance of noise and staggering amounts of information. They traverse multiple tools just to find an answer.

Often, you might rely on infrastructure metrics or troll through endless messy logs to understand and triage a problem. But it can be hard to understand what’s happening inside the application. Without the additional help from software engineers to instrument their Go apps, Ops engineers often resort to guesswork or intuition when an incident or outage occurs.

So, before you start your instrumentation effort, start with a simple rule: Ask yourself, "What is the minimal viable instrumentation (or MVI) that I can instrument to help my fellow Ops engineers quickly identify issues?"

Admittedly, most of the time engineers are only interested in identifiers. For example, if there is a failed transaction, an engineer wants to know which userID, transactionID, or some form of contextual information can pinpoint exactly the affected request for a remediation path. But relying on status codes such as 404 or 500 isn’t going to be sufficient anymore. You'll need additional contextual metadata to work effectively as your environment starts to get complex.

Here are the four key identifiers you should consider as part of your instrumentation.

Request Identifier: correlation ID between requests (such as accountID)

State Identifier: current operations or status (such as completed)

Environment Identifier: operating domain metadata (such as software version)

Parameters Identifier: variables used to pass information (such as total or account)

These identifiers are the building blocks to improve observability.

After you've decided on your identifiers, plan ahead with your top five most important software functions that you should expose with telemetry, using the New Relic Go agent, and add more as you need over time.

Tip 2: Start the flow of data into New Relic using StartTransaction

The following video will step through tips two and three, followed by further explanation about this process in this blog.

Top Tips for Instrumenting Golang | New Relic

When you start using the New Relic Go agent for the first time, there’s a high possibility you won’t see any data, which is different than interpreted languages such as PHP, Ruby, Python, and JavaScript.

For interpreted languages, the New Relic SDK begins instrumenting supported framework functions, methods, or classes. But for Go, you need to start instrumentation by writing a StartTransaction.

txn := app.StartTransaction("transaction_name")
defer txn.End()

Note these two pieces of helpful documentation:

Instrument and monitor Go transactions
Install New Relic for Go

The app statement refers to the variable assigned during the New Relic configuration process, while the defer statement defers the segment ending until the function closes. This means that setting up New Relic for your Golang app requires you to add New Relic methods to your source code manually. Here is a simple instrumentation of a popular function: print().

func main() {
   app, err := newrelic.NewApplication(
       newrelic.ConfigAppName("Short Lived App"),
       newrelic.ConfigLicense(os.Getenv("NEW_RELIC_LICENSE_KEY")),
       newrelic.ConfigDebugLogger(os.Stdout),
   )
   if nil != err {
       fmt.Println(err)
       os.Exit(1)
   }

   // Wait for the application to connect.
   if err := app.WaitForConnection(5 * time.Second); nil != err {
       fmt.Println(err)
   }

   // Do the tasks at hand.  Perhaps record them using transactions and/or
   // custom events.
   tasks := []string{"white", "black", "red", "blue", "green", "yellow"}
   for _, task := range tasks {
       txn := app.StartTransaction("task")
       time.Sleep(10 * time.Millisecond)
       txn.End()
       app.RecordCustomEvent("task", map[string]interface{}{
           "color": task,
       })
   }

   // Shut down the application to flush data to New Relic.
   app.Shutdown(5 * time.Second)
}

Note the additional code segment - app.WaitForConnection and app.Shutdown, where we wait for the application to connect and shut down for five seconds. You need to do this initially because the harvest cycle in the New Relic Go agent is five seconds. This is only required if you are instrumenting a small sample app or a short-lived app in your development environment. For production deployment, you won’t need to use this code segment if your app runs longer than five seconds.

For a full code sample in one file, see this repository on GitHub.

Tip 3: Embed observability practices

As you start instrumenting more with your Go apps, take some time to discuss and review incidents with your team or engineering leads. Conduct a post-mortem or retrospective to understand what additional metadata engineers need from the app to triage issues more effectively.

Observability is about better understanding the inner workings of a system through exposed external outputs—getting one step closer to the why. Observability is also a scientific engineering approach to refine, experiment, learn, and add observations over time, so you can gain more confidence in your system.

As mentioned earlier, this is a great opportunity to embed observability practices as part of your instrumentation and development cycle. Observability is not about monitoring metrics or using an existing operations approach to look at hundreds or thousands of dashboards or alerts.

Often engineers will debate whether metrics, tracing, or logs are the best, but each telemetry type has its own pros and cons. Ultimately observability is all about encouraging deep collaboration between Dev and Ops, separating noise and signals. You gain more confidence over time in your systems by exposing contextual information either through better usage of metrics, tracing, or logs.

Unlike automatic instrumentation, the good news about manual instrumentation for Go is that there are no hidden, weird, or sudden surprises. Starting fresh allows you to embed observability concepts, knowing exactly what the instrumentation is doing, and you have absolute control over what you instrument in your app. The package from New Relic is stable, open, and available so you can understand what the agent is doing underneath the hood.

For more information, don't forget to visit the official page for the New Relic Go agent.

To read this full New Relic blog, click here.