TL;DR: This article explores advanced error handling in Go, focusing on contextual errors that provide rich details for debugging by embedding key-value metadata into errors. It demonstrates implementing these errors using a dedicated package, offering insights into better error resolution and organized logging. The approach enhances error understanding in Go applications, balancing detail and simplicity.

Before reading this post, consider checking the first article about error handling in Golang: Domain-Centric Approach to Error Handling using Go

Introduction

In the previous blog post, we explored the domain errors approach to error handling in Go, ensuring that error messages are both clear and user-centric. However, while this approach effectively addresses one aspect of error handling, it falls short in another critical area - providing sufficient contextual information. The absence of context can create significant challenges for developers and QA specialists as they attempt to investigate and understand the underlying problems in an application.

In this post, we will try to tackle this issue. We'll start by taking a brief look at the standard approach to error handling in Go, then go through a typical example that highlights the limitations of this method. Finally, we will introduce an approach that could help solve this problem, or at least make it more manageable and future-proof.

Limitations of the Standard Way

When using traditional Go error handling, providing useful context is pretty difficult. Errors are just simple text messages without any structured information. An error is typically created in a deeply nested service (e.g. a database repository) and bubbles up through several layers before finally being handled. By the time it reaches the surface, the original context - the 'why' and 'how' of the error - is often lost.

To address this lack of context, a common practice is to embed context directly into the error message using formatted errors. Typically, this is done by using statements that include additional information with the error, such as:

fmt.Errorf("failed to store entity with ID=%s: %w",
    entity.ID, err)

fmt.Errorf("entity %s is empty", entity.ID)

fmt.Errorf("failed to process entity with id=%s, 
    userID=%s: %v", entity.ID, user.ID, err)

However, this approach leads to its own set of problems. As an error passes through various layers, accumulating additional context, the final error string becomes increasingly complex and convoluted:

failed to process request, requestID=491002, userID=4881:
doWork called, failed to do work, entityID=482003, 
userID=4881: failed to fetch entity with ID=482003 from 
the database: database connection lost

Such error strings, although rich in information, are not great:

Long, nested error messages become increasingly difficult to read and understand. The deeper the error is nested, the harder it is to quickly determine the root cause or the primary message.
These verbose error strings are hard to parse by logging analysis tools. Extracting specific pieces of information from these long strings can be problematic and error-prone.
As errors bubble up through the application layers, the same information often gets repeated. For example, user IDs or entity IDs might be appended at multiple levels, leading to redundancy in the error message.

Exploring a Practical Example

To illustrate the practicality and necessity of contextual errors in Go, let's consider a common scenario in a typical Go application with a simple layered architecture: an HTTP handler, a Domain Service, and a Database Repository.

HTTP Handler (package api)

This layer handles incoming HTTP requests, interacts with domain services, and responds to the client. It's the entry point for most operations initiated by the user. For simplicity, in the exampleHandleSomeRequest does not operate with http.ResponseWriter and http.Request directly, leaving it to a more generic caller.

package api

type DomainService interface {
    DoWork(ctx context.Context, userID string) error
}

type API struct {
    domainService DomainService
}

func (a *API) HandleSomeRequest(ctx context.Context, userID string) *Response {
    err := a.domainService.DoWork(ctx, userID)
    if err != nil {
        slog.Error("error doing work", "error", err)
        return NewErrorResponse(err)
    }

    return nil
}

Domain Service (package domain)

This layer contains the business logic. It's responsible for executing domain-specific operations, often interacting with the repository using an interface.

package domain

type Repository interface {
    LocateEntity(ctx context.Context, userID string) (*Entity, error)
    SaveEntity(ctx context.Context, entity *Entity) error
}

type Service struct {
    repo Repository
}

func (s *Service) DoWork(ctx context.Context, userID string) error {
    // find specific entity which might be different depending 
    // on the user internal state on the moment of the request
    entity, err := s.repo.LocateEntity(ctx, userID)
    if err != nil {
        return err
    }

    // perform some work on the entity
    if err := entity.DoImportantEntityWork(); err != nil {
        return err
    }

    // save the entity back in the database
    if err := s.repo.SaveEntity(ctx, entity); err != nil {
        return err
    }

    return nil
}

As you may see, the process of error propagation to higher layers is quite straightforward and doesn't provide much extra information. For example, in the absence of any context, an HTTP handler would not be able to log any specific details of the entity that caused the error. This information is only available in the domain layer and is not transferred to higher layers.

Let's take a look at how to address this problem and improve this code without adding this information right in the error message.

Introducing Structured Contextual Errors

Structured contextual errors differ from traditional error wrapping by including additional metadata in a structured format, typically key-value pairs. This metadata can be represented by various data relevant to the error, such as entity IDs, user details, input parameters, or internal state information. By embedding this information directly into the error, we create a self-contained error object that carries its context along with it.

Let's refer back to our previous example and improve our error handling by implementing structured contextual errors. Instead of returning a generic error message, we will wrap it with additional context when an error occurs with a simple contract likeWithMetadata(err error, pairs …any) which adds provided metadata to the error and returns an enriched version.

package domain

// … [Other code] …

func (s *Service) DoWork(ctx context.Context, userID string) error {
    entity, err := s.repo.LocateEntity(ctx, userID)
    if err != nil {
        return metaerr.WithMetadata(err, "operation", "locate_entity")
    }

    if err := entity.DoWork(); err != nil {
        return metaerr.WithMetadata(err, "operation", "do_work", 
            "entity_id", entity.ID)
    }

    if err := s.repo.SaveEntity(ctx, entity); err != nil {
        return metaerr.WithMetadata(err, "operation", "save_entity", 
            "entity_id", entity.ID)
    }

    return nil
}

In the HTTP handler layer, we can now include structured metadata when logging errors, making the logs more informative. It can be done with a handy GetMetadata(err) []any function that extracts all metadata gathered throughout the call chain and returns it as a slice of key-value pairs suitable for a standard Go structured logging library.

package api

// … [Other code] …

func (a *API) HandleSomeRequest(ctx context.Context, userID string) *Response {
    err := a.domainService.DoWork(ctx, userID)
    if err != nil {
        slog.With(metaerr.GetMetadata(err)...).
            With("user_id", userID, "error", err).
            Error("error doing work")
        return NewResponse(err)
    }

    return nil
}

MetaErrors Implementation

Having our desired contract in place, let's take a look at a potential implementation of the metaerr package.

First thing we need a structure type that will represent an error with attached metadata and will support all required error-related interfaces. Let's name it errMetadata and make it private to the package, as there should not be any need to reference it from outside. This struct is the cornerstone of the metaerr package and will enable us to store additional error context.

// errMetadata represents an error with attached structured metadata
type errMetadata struct {
    // err is the original error
    err error
    // metadata is the container for structured error context
    metadata []any
}

// Error returns the original error message, 
// ensuring compatibility with the standard error interface.
func (e *errMetadata) Error() string {
    return e.err.Error()
}

// Unwrap allows errors wrapped by errMetadata to be compatible with
// standard error unwrapping mechanism
func (e *errMetadata) Unwrap() error {
    return e.err
}

The WithMetadata function is responsible for wrapping an existing error with additional context. It takes an error and a series of key-value pairs and then wraps this error in the errMetadata struct along with the provided metadata.

package metaerr

// WithMetadata creates a new error with metadata
func WithMetadata(err error, pairs ...any) error {
    return &errMetadata{
        err:      err,
        metadata: pairs,
    }
}

The GetMetadata function is designed to extract metadata from errors in the form of a slice, suitable for the Go standard structured logger (slog). In the future, more specific methods may be introduced, such as returning key-value pairs as a map or merging values for duplicated keys. However, for now, let's limit the example to logging purposes and keep things simple.

// GetMetadata returns metadata from the error chain
// if there is no metadata in the chain, it will return an empty slice
func GetMetadata(err error) []any {
    data := make([]any, 0)

    // we will iterate over all errors in the chain
    // and merge metadata from all of them
    for err != nil {
        // if current error is a metadata error
        // we will add its metadata to the existing
        // already collected metadata
        if e, ok := err.(*errMetadata); ok {
            data = append(data, e.metadata...)
        }

        err = errors.Unwrap(err)
    }

    return data
}

With the metaerr package, error handling in our example application becomes more informative. When an error is created or passed through the DoWork function in the domain service, WithMetadata is used to attach relevant context. At the API layer, logging these errors with GetMetadata helps to capture a complete picture of the error scenario, including all important contextual details.

Benefits of Contextual Error Handling

The primary benefit of contextual errors is the rich detail they provide, which is invaluable for debugging:

Unlike traditional error messages that often require developers to dig through logs or replicate issues, contextual errors can provide immediate insights into what went wrong and under what conditions.
Logging systems can leverage the structured nature of these errors to create more organized and searchable logs, making it easier to filter and analyze them.
Logs that include detailed error metadata offer a more comprehensive view of issues, aiding in post-mortem analysis and continuous improvement processes.

Some Rules Of Thumb

To effectively utilize contextual errors in Go applications, consider the following rules of thumb:

Only include metadata that is useful for understanding the error. Avoid cluttering errors with irrelevant or redundant information.
Add information at the point where it is first obtained. For example, a parameter passed to a function should be added to the error in the function where it was created, not necessarily where the error is first encountered. It will help to reduce the risk of adding the same parts of metadata in multiple places.
Use consistent key names for metadata across different parts of the application. This consistency is crucial for effective log analysis.
Be mindful of how errors are wrapped. Preserve the original error using Go's wrapping mechanism (%w with fmt.Errorf) to maintain the error chain and gather metadata.
Contextual errors should complement, not replace, existing error-handling strategies. They should fit seamlessly into the current architecture.
Be cautious about including sensitive information in error metadata, especially when logs might be stored or processed in less secure environments.

Conclusion

By integrating structured contextual errors, particularly through the implementation of a library similar to the discussed one, developers can achieve a deeper understanding and more efficient resolution of issues that arise in their applications.

Key Takeaways

Contextual errors provide information that goes beyond the traditional error message, offering insights into the application's state at the time of the error.
With detailed metadata attached to errors, the debugging process becomes more straightforward, leading to quicker resolutions and enhanced application reliability.
Structured error metadata leads to more organized and actionable logs, aiding in both real-time issue resolution and long-term analysis.
Implementing contextual errors effectively involves a balance of including relevant metadata, maintaining consistency, and ensuring performance and simplicity are not compromised.

Potential pitfalls

Keep in mind that metadata is not a replacement for typed errors, it is just a context. If you need to provide some specific information to a client, do it explicitly with typed domain errors, generic metadata is not a good fit for domain-related information.
Adding too much metadata can lead to performance overhead. Ensure that the process of attaching metadata is efficient and doesn't significantly impact application performance.
The error-handling logic can become more complex with the addition of metadata. Strive for a balance between detail and simplicity.

Final Thoughts

Complexities of error management in Go demonstrate that the decision to implement contextual and structured error handling is both a practical and strategic choice. It's about enhancing the code we write and the experiences of those who interact with our applications - developers, support teams, or quality assurance specialists.

Embracing contextual structured error handling and logging is a step in that direction, offering a method to handle errors as sophisticated and nuanced as the applications we build.

I hope you found this exploration insightful and that it inspires you to implement these practices in your Go projects. Your feedback and experiences are invaluable, so feel free to share your thoughts and how you have applied these concepts in your work.

Stay tuned for more discussions on Go programming, software architecture, and beyond. Happy coding!

Find me on Linkedin: https://www.linkedin.com/in/alex-demin-dev/

Context Matters: Advanced Error Handling Techniques in Go