Cover image by "NESA by Makers" on Unsplash.

Each developer relations program has a different opinion on what should be north star metrics to measure the success of their platform. Some metrics are valid while others can be what are called vanity metrics. This post discusses which metrics you should or should not be tracking.

What to measure

The goal of developer relations is to ensure third-party developers are able to leverage your platform to create something of value. Value can be subjective but some examples include shipping a new integration or plugin that increases the usability of your products or integrating your APIs and SDKs into their web or mobile apps to deliver a better experience for their customers.

False north star metrics

Because this is hard to quantify, some developer relations programs naively fall back to marketing metrics like page views and sign ups. The problem with these metrics is a new user who signs up for your platform didn’t actually create any value. You can easily hack these metrics by running a large Google AdWords or Facebook campaign to drive an influx of random sign-ups. Yet those users would never integrate and leverage the platform.

To really measure the success of a platform, you should look at two north star metric:

Weekly Active Tokens (or Weekly Active Users)
Time to First Hello World

1. Weekly Active Tokens (WAT)

In order to measure the success of a platform, we should instead look at the usage. This can be from the various APIs and SDKs that you have released for third-party developers to interact with your platform. The mere act of generating an API key does not measure usage since even non-developers can view API keys without any real use for them. Thus we turn to Weekly Active API Tokens. Since most APIs limits access to authenticated users, we are able to track how many distinct tokens are accessing our API platform in a given week.

Since a single customer can create multiple API keys with various scopes and expiration dates, we can further narrow this down to Weekly Active Integrated Companies. Everyone on your developer relations team should be looking at this metric when deciding on where to invest more time. In order to do so, you should have the instrumentation in place to tie metrics like tokens to developer demographic info such as any marketing channels that brought the developer to your site, company affiliation and size, and even social info such as GitHub stars. With this info, you can break down WAT to find what is contributing the most to your north star metric.

2. Time to First Hello World (TTFHW)

Followed by Weekly Active Tokens, Time to First Hello World is a good proxy for the overall success of your platform. Not only does it include adoption and usage like WAT, but it also includes other functional areas outside DevRel such as marketing and support.

The longer a developer takes to get started with your platform, the less likely it will be successful. Documentation that is hard to understand, unclear next steps in onboarding, or API and SDKs that require a six-hour lecture just to understand the basics are all items that can contribute to higher than wanted TTFHW.

An effective DevRel program should be working to increase Weekly Active Tokens (WAT) while reducing Time to First Hello World (TTFHW).

Where does developer relations sit

Since the goal of developer relations is to ensure developers are able to create something of value with your platform. extra care must be taken to ensure the DevRel goals are aligned with the greater department. Being a relatively new field, each company has its own idea where DevRel should reside and where the budget should come from.

Marketing

At some companies, developer relations is an extension of marketing. Due to this, these DevRel teams have the luxury of a large advertising budget and can promote the platform everywhere from Facebook Ads to a large presence at conferences. Key metrics for the marketing team include page views, sign-ups, and Marketing Qualified Leads (MQLs). However, these metrics are misaligned with the developer relation’s team in helping developers engage with and find something of value with the platform. Thus, these teams are effectively a technical or developer marketing team without much of the relations part.

Engineering

At other companies, developer relations is an extension of the engineering team. These teams are responsible for up to date API documentation and potentially the SDKs themselves. They are also able to debug and fix issues developers may be running into or create additional guides and examples that can aid their developer community. These teams, unfortunately, spend most of their time engaging with other teammates at their company rather than the greater community due to the constant stream of SDK changes, documentation updates, debugging, etc.

Product

Product’s mission is to build a product customers (or developers) actually adopt and use. They are constantly testing various hypotheses and building out a feature road map based on product analytics data and qualitative feedback from customers. Developer relations teams are constantly collecting feedback from developers to influence road map while they are the first to know if there is any resentment from their community on product changes. In this capacity, developer relations is an extension of product and more “in the community” rather than focused on the building side and collaborating with internal stakeholders. While this is a biased belief from working with multiple platform companies and developer relations teams, we’ve seen DevRel can perform best when aligned with product rather than other areas like marketing and engineering. After all, product is also held accountable by engagement metrics similar to Weekly Active Tokens and Time to First Hello World (which product messaging and onboarding experience can have a large impact on).

Closing thoughts

Any new program needs to have the right metrics that guide the mission and goals of the program. Developer relations teams are not immune to accountability. Yet, the limited operating history of developer relations teams has created an ad hoc system of borrowed metrics from other departments based on where the DevRel budget comes from. Creating product metrics like Weekly Active Tokens ensure DevRel is aligned with helping developers engaging with and finding value in a new API or platform.

Moesif is the leading API Analytics service for API platforms.

This article was written by Derric Gilling, co-founder, and CEO of Moesif. It was originally published on the Moesif Blog.

Best practices for Developer Relations Programs to measure the success of an API platform