Stateful Serverless with Durable Functions

Christian Nwamba - Sep 7 '19 - - Dev Community

This article is part of #ServerlessSeptember. You'll find other helpful articles, detailed tutorials, and videos in this all-things-Serverless content collection. New articles are published every day — that's right, every day — from community members and cloud advocates in the month of September.

Find out more about how Microsoft Azure enables your Serverless functions at https://docs.microsoft.com/azure/azure-functions/.

What makes Azure Durable Functions powerful is their ability to retain the state of an API even after the request is concluded. The advantage of having this is, you can go back to the API at a later time and track how the state has evolved. This explains what we mean by Stateful, while the term Serverless explains when a cloud provider manages the allocation of resources and the provision of a server architecture. An excellent use case of Azure Durable Functions is for building timers and notifications.

So what are Durable Functions? A Durable Function consists of different types of Azure Functions that can be used together to resolve orchestration challenges in a serverless architecture.

There are two words we need to get out of the way first before we can understand what Durable Functions offer — orchestration and durable.

Orchestration was initially used to describe an assembly of well-coordinated events and actions. It is heavily borrowed in computing to describe smooth coordination of computer systems. The keyword is coordinate. We need to put two or more units of a system together in a coordinated way.

Durable is used to describe anything that does well in withstanding the passing of time. The combination of system coordination and durability gets you Durable Functions. It is one of the best offerings of Azure’s Serverless Function. Durable Functions based on what we now know has these two features:

  1. They can be used to assemble the execution of two or more functions and coordinate them, so race conditions do not occur (orchestration).

  2. Durable Functions remember things. This is what makes it so powerful. It breaks the number one rule of HTTP: stateless. Durable functions keep their state intact no matter how long they have to wait. Create a schedule for 1,000,000 years into the future, and a durable function executes after one million years while remembering the parameters that were passed to it on the day of trigger. That means Durable Functions are stateful.

The fact that a serverless function can effectively maintain its state through time provides a sphere of opportunities. We will utilize one of those features for sending notifications in this article.

If you’re searching for a more visual representation of what Azure’s Durable Functions offer, I would recommend Sarah’s article on Durable Functions.

I also made a visual representation of the behaviour of the Durable Functions we will be writing today:

Shows the touch-points of a serverless system.

A typical Durable Function conforms to the following flow:

  1. A data modification from an external system that triggers the orchestration using the HTTP Trigger.
  2. The trigger then calls the orchestration function, which schedules an event.
  3. When the time for execution is due, a call to the orchestration function made again, but this time, it skips the orchestration and calls the activity function. The activity function is the action performer. This is where the magic happens, e.g. "send email notification". ## Create orchestrated Durable Functions

Let me walk you through creating functions using VS Code. You need two things:

  1. An Azure account
  2. VS Code

Once you have both setup, we need them working together. You can do this using a VS Code extension and a Node CLI tool. Start with installing the CLI tool:

npm install -g azure-functions-core-tools

# OR

brew tap azure/functions
brew install azure-functions-core-tools
Enter fullscreen mode Exit fullscreen mode

Once the installation is complete, pop open your VSCode editor and install the Azure Functions extension. If you’re feeling up to it, read more about setting up Azure Functions from my previous article.

Now that you have all the setup done, let’s get into creating these functions. The functions we will be creating will map to the following folders.

Folder Function
schedule Durable HTTP Trigger
scheduleOrchestrator Durable Orchestration
sendEmail Durable Activity

Create a folder named serverless, open the folder using the VSCode editor and follow the instructions below:

We’ll start by creating the trigger:

  1. Click on the Azure extension icon and follow the image below to create the schedule function
    Shows the interface steps going from Browse to JavaScript to Durable Functions HTTP start to naming the function schedule.

  2. Since this is the first function, we chose the folder icon to create a function project. The icon after that creates a single function (not a project).

  3. Select the serverless folder.

  4. Select JavaScript as the language. You can select any other language of preference if you wish.

  5. Select Durable Functions HTTP starter. This is the trigger.

  6. Name the first function as schedule

Next, create the orchestrator. Instead of creating a function project, create a function instead.

  1. Click on the function icon:

  2. Select Durable Functions orchestrator.

  3. Name it scheduleOrchestrator and hit Enter.

  4. Select a storage account when asked for one, Orchestrator uses the storage to maintain the state of a function-in-process.

  5. Select a subscription in your Azure account. In my case, I chose the free trial subscription.

  6. Follow the few remaining steps to create a storage account.

Finally, repeat the previous step to create an Activity. This time, the following should be different:

  • Select Durable Functions activity.
  • Name it sendEmail.
  • No storage account will be needed.

Scheduling with a durable HTTP trigger

We will maintain the bootstrapped code in the serverless/schedule/index.js file. The contents of the file should look similar to the snippet below:

const df = require("durable-functions");
module.exports = async function (context, req) {
  const client = df.getClient(context);
  const instanceId = await client.startNew(req.params.functionName, undefined, req.body);
  context.log(`Started orchestration with ID = '${instanceId}'.`);
  return client.createCheckStatusResponse(context.bindingData.req, instanceId);
};
Enter fullscreen mode Exit fullscreen mode

Let’s run through the contents of the file:

  1. We’re creating a durable function on the client side that is based on the context of the request.
  2. We’re calling the orchestrator using the client's startNew() function. The orchestrator function name is passed as the first argument to startNew() via the params object. A req.body is also passed to startNew() as third argument which is forwarded to the orchestrator.
  3. Finally, we return a set of data that can be used to check the status of the orchestrator function, or even cancel the process before it's complete.

The URL to call the above function would look like this:

http://localhost:7071/api/orchestrators/{functionName}
Enter fullscreen mode Exit fullscreen mode

Where functionName is the name passed to startNew. In our case, it should be:

//localhost:7071/api/orchestrators/scheduleOrchestrator
Enter fullscreen mode Exit fullscreen mode

It’s also good to know that you can change how this URL looks.

Orchestrating with a Durable Orchestrator

The HTTP trigger startNew call calls a function based on the name we pass to it. That name corresponds to the name of the function and folder that holds the orchestration logic. The serverless/scheduleOrchestrator/index.js file exports a Durable Function. Replace the content with the following:

const df = require("durable-functions");
module.exports = df.orchestrator(function* (context) {
  const input = context.df.getInput()
  // TODO -- 1

  // TODO -- 2
});
Enter fullscreen mode Exit fullscreen mode

The orchestrator function retrieves the request body from the HTTP trigger using context.df.getInput().

Replace TODO -- 1 with the following line of code which might happen to be the most significant thing in this entire demo:

yield context.df.createTimer(new Date(input.startAt))
Enter fullscreen mode Exit fullscreen mode

In this line, we create a timer based on the start date provided in the request body received from the HTTP trigger.

Upon getting to this line, execution bails temporarily after creating the timer. When the set timer runs down, it comes back, skips this line and executes the next line. Replace the TODO -- 2 comment with the following line of code:

return yield context.df.callActivity('sendEmail', input);
Enter fullscreen mode Exit fullscreen mode

The function would call the activity function to send an email. We are also passing a payload as the second argument.

The body of the payload should look something like this:

{
  "description": "This is a test mail",
  "startAt": "2019-01-23",
  "title": "Test email",
  "email": "test@example.com"
}
Enter fullscreen mode Exit fullscreen mode

This is what the completed function would look like:

const df = require("durable-functions");

module.exports = df.orchestrator(function* (context) {
  const input = context.df.getInput()

  yield context.df.createTimer(new Date(input.startAt))

  return yield context.df.callActivity('sendEmail', input);
});
Enter fullscreen mode Exit fullscreen mode

Sending email with a durable activity

When a schedule is due, the orchestrator comes back to call the activity. The activity file lives in serverless/sendEmail/index.js. Replace what’s in there with the following:

const sgMail = require('@sendgrid/mail');
sgMail.setApiKey(process.env['SENDGRID_API_KEY']);

module.exports = async function(context) {
  // TODO -- 1
  const msg = {}
  // TODO -- 2
  return msg;
};
Enter fullscreen mode Exit fullscreen mode

It currently imports SendGrid’s mailer and sets the API key. You can get an API Key by following these instructions.

I am setting the key in an environmental variable to keep my credentials safe. You can safely store yours the same way by creating a SENDGRID_API_KEY key in serverless/local.settings.json with your SendGrid key as the value:

{
  "IsEncrypted": false,
  "Values": {
    "AzureWebJobsStorage": "<<AzureWebJobsStorage>",
    "FUNCTIONS_WORKER_RUNTIME": "node",
    "SENDGRID_API_KEY": "<<SENDGRID_API_KEY>"
  }
}
Enter fullscreen mode Exit fullscreen mode

Replace TODO -- 1 with the following line:

const { email, title, startAt, description } = context.bindings.payload;
Enter fullscreen mode Exit fullscreen mode

This pulls out the event information from the input from the orchestrator function using object destructuring. You can name attached to the input context.bindings.payload can be anything so go to serverless/sendEmail/function.json and change the value of the name field to payload:

{
  "bindings": [
    {
      "name": "payload",
      "type": "activityTrigger",
      "direction": "in"
    }
  ]
}
Enter fullscreen mode Exit fullscreen mode

Next, update TODO -- 2 with the following block to send an email:

const msg = {
    to: email,
    from: { email: 'chris@codebeast.dev', name: 'Codebeast Durable Functions' },
    subject: `Test: ${title}`,
    html: `<h4>${title} @ ${startAt}</h4> <p>${description}</p>`
  };
sgMail.send(msg);

return msg;
Enter fullscreen mode Exit fullscreen mode

Here is the complete version:

const sgMail = require('@sendgrid/mail');
sgMail.setApiKey(process.env['SENDGRID_API_KEY']);

module.exports = async function(context) {
  const { email, title, startAt, description } = context.bindings.payload;
  const msg = {
    to: email,
    from: { email: 'chris@codebeast.dev', name: 'Codebeast Durable Functions' },
    subject: `Test: ${title}`,
    html: `<h4>${title} @ ${startAt}</h4> <p>${description}</p>`
  };
  sgMail.send(msg);

  return msg;
};
Enter fullscreen mode Exit fullscreen mode

Deploying functions to Azure

Deploying functions to Azure is easy. It’s merely a click away from the VS Code editor. Click on the circled icon to deploy.

After deploying successfully, you should get a URL; store the URL safely as all requests to this function would be through this URL.

Testing

To test the function, we can make use of cURL or Postman to send HTTP requests to the trigger URL that was returned after deploying the function. First, we will test the function using cURL, run the command below in a terminal:

curl -d '{"description": "This is a test mail","startAt": "2019-01-23","title": "Test email","email": "test@example.com"}' -H "Content-Type: application/json" -X POST '<http://YOUR_HTTP_TRIGGER_URL>'
Enter fullscreen mode Exit fullscreen mode

A sample response would look something like this:

{
    "id": "a14af5b16c6a4817af5484ac6c6dbeaf",
    "statusQueryGetUri": "",
    "sendEventPostUri": "",
    "terminatePostUri": "",
    "rewindPostUri": "",
    "purgeHistoryDeleteUri": ""
}
Enter fullscreen mode Exit fullscreen mode

If you’re looking to test it using Postman, you can follow the annotated screenshot below:

Enter the URL returned in the last section in the address bar. Click the body tab, select the raw option and pick JSON(application/json) from the dropdown on the right.

Fill in a JSON formattted request body and click Send to run the request. Endeavour to test with a functional email address.

Image
2024 -Genesis Cloud Mining The best opportunity to start a business and get rich
Image
forex trading companies in india
Image
Must-Try Cloud-Native DevOps Tools

devops, cloud, kubernetes, terraform
Image
Revolutionizing User Interaction with Voice Assistant Technology

ai, chatgpt, voicebot, voiceui
Image
Hola, Playwright (I) - Conceptos e2e y configuración inicial.

e2e, testing, tutorial, playwright
Image
Jak Wybrać Idealne Meble Sypialniane: Przewodnik po Komfortowym Śnie
Image
Tools and Libraries Roundup: Supercharge Your Development Workflow 🚀

webdev, softwaredevelopment, programming, vscode
Image
best forex trading platforms india
Image
NURS FPX 4020 Assessment 4: Enhancing Patient Safety and Quality of Care

tutors, academy
Image
Rendering Strategies in Next.js

webdev, javascript, programming, nextjs
Image
Judi Online Win-Rate Paling tinggi di Indonesia
Image
AI Tools Every Mobile Developer Needs to Use

ai, mobile, webdev, beginners
Image
SECURE YOUR LOST INVESTMENT ASSET WITH CRYPTO RECOVERY CONSULTANT

bitcoinrecoveryexpert, cryptocurrency, ethereum, usdtexpert
Image
top forex brokers in india
Image
Guidelines for Charging Electric batteries Safely
Image
NURS FPX 4060 Assessment 2: A Practical Guide to Effective Patient Care

tutorrs, academy
Image
🌥️ Kickstart Your Cloud Journey: Creating Your First EC2 Instance ☁️

aws, cloudskills, beginners, tutorial
Image
forextrading for beginners online
Image
Tapczan - Królewskie Miejsce Relaksu w Twoim Salonku!
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .