I am so excited to tell you all about the code that inspired this tweet...

I'm mostly excited because this affects pretty much all the users of our community in a positive way and unlocks a lot of possibilities for future development approaches and saves incredible amounts of data that would otherwise be shipped across the wire.

Demo time

To best demonstrate this feature, reload this page.

Unless the demo gods are frowning upon us, you should experience a shockingly fast response.

To further demonstrate this feature, head into the network tab in your browser's dev tools and throttle down your performance, perhaps to "slow 3G".

You should experience a page which immediately loads your top navigation and displays some loading text.

What's happening in either case is that the first part of the web request is being stored locally via Service Workers.

This demo may break if you're accessing this site via Twitter's in-app iOS browser or other edge cases I'm not aware of yet. Hence the above tweet.

The magic of Service Workers

The concept here is that Service Workers can act as a reverse proxy and execute code on behalf of a website before sending a page request. We've now leveraged this to store the "top" part of DEV, which was already established as the same for every page across the site.

Our approach is akin to the "App Shell Model" wherein a basic page exoskeleton is shipped to the browser and then the rest of the page is sent over via JSON in order to be filled in with frontend code. This approach dramatically adds to the efficiency of each request. However, given that our site is driven by cacheable documents meant for reading, and the fact that our team and tech stack leans more towards traditional backend templating through Ruby on Rails, I wanted to go in a different direction.

In experimenting with app-shell ideas it became clear that in most cases it actually takes longer to render useful content via the app shell model because there is more waiting around for code to execute at different stages, and there is no ability to leverage "streaming". It would also have forced us to re-architect a lot of what we do, and I mostly wanted to make this change invisible to our developers as long as they understand the basic constraints and possible gotchas in place.

Streams are a technology as old as time as far as the web is concerned. It's what enables the browser to progressively render a web page as the bits and bytes make their way across the universe and into your living room.

We use the ReadableStream class in order to piece together a page as its parts become available. The first "part" in our case is the top.

Our top is captured upon installation of the Service Workers in your browser, alongside the rest of the cacheable assets.

From our serviceworker.js file...

  self.addEventListener('install', event => {
    self.skipWaiting();

    // Populate initial serviceworker cache.
    event.waitUntil(
      caches.open(staticCacheName)
        .then(cache => cache.addAll([
          "/shell_top", // head, top bar, inline styles
          "/shell_bottom", // footer
          "/async_info/shell_version", // For comparing changes in the shell. Should be incremented with style changes.
          "/404.html", // Not found page
          "/500.html", // Error page
          "/offline.html" //Offline page
        ]))
    );
  });

Even though we're not using the App Shell Model proper, shell still seemed like a good term for what's going on.

The top and bottoms are basically partials of the full page delivered as standalone HTML snippets with an endpoint. They are cached static via our CDN so this request doesn't hit our servers or waste a lot of download time. In the shell top we basically load everything in for styling and rendering that first part of the site. The shell bottom is our footer and any code that needs to execute there.

/async_info/shell_version is an endpoint designed to ensure the shell is kept in sync and updated when we make changes.

This is the meat of what's going on...

  function createPageStream(request) {
    const stream = new ReadableStream({
      start(controller) {
        if (!caches.match('/shell_top') || !caches.match('/shell_bottom')) { //return if shell isn't cached.
          return
        }

        // the body url is the request url plus 'include'
        const url = new URL(request.url);
        url.searchParams.set('i', 'i'); // Adds ?i=i or &i=i, which is our indicator for "internal" partial page
        const startFetch = caches.match('/shell_top');
        const endFetch = caches.match('/shell_bottom');
        const middleFetch = fetch(url).then(response => {
          if (!response.ok && response.status === 404) {
            return caches.match('/404.html');
          }
          if (!response.ok && response.status != 404) {
            return caches.match('/500.html');
          }
          return response;
        }).catch(err => caches.match('/offline.html'));

        function pushStream(stream) {
          const reader = stream.getReader();
          return reader.read().then(function process(result) {
            if (result.done) return;
            controller.enqueue(result.value);
            return reader.read().then(process);
          });
        }
        startFetch
          .then(response => pushStream(response.body))
          .then(() => middleFetch)
          .then(response => pushStream(response.body))
          .then(() => endFetch)
          .then(response => pushStream(response.body))
          .then(() => controller.close());
      }
    });

    return new Response(stream, {
      headers: {'Content-Type': 'text/html; charset=utf-8'}
    });
  }

?i=i is how we indicate that a page is part of "internal" navigation, a concept that already existed within our app which set us up to implement this change without much business logic on the backend. Basically this is how someone requests a page on this site that does not include the top or bottom parts.

The crux of what’s going on here is that we take the top and bottom from a cache store and get to work rendering the page. First comes the already available top, as we get to work streaming in the rest of the page, and then finishing off with the bottom part.

This approach lets us generally ship many fewer bytes while also controlling the user experience with more precision. I would like to add more stored snippets for use in areas of the site that can most make use of them. I especially want to do so on the home page. I think we can store more of the home page this way and ultimately render a better experience more quickly in a way that feels native in the browser.

We have configurations such as custom fonts in user settings and I think this can be incorporated smartly into Service Workers for the best overall experience.

There was a period of edge case discovery and bugs that needed to be ironed out once this was deployed. It was hard to catch everything upfront, especially the parts which are inherently inconsistent between environments. Conceptually, things are about the same as they were before for our developers, but there were a few pages here and there which didn't work as intended, and we had some cached content which didn't immediately play well. But things have been mostly ironed out.

Early returns indicate perhaps tens of milliseconds are being saved on requests to our core server which would have otherwise had to whip up our header and footer and send it all across the wire.

There is still a bug that makes this not quite work properly in the Twitter in-app browser for iOS. This is the biggest head scratcher for me, if anybody can track this down, that would be helpful. iOS, in general, is the platform that is least friendly to Service Workers, but the basic Safari browser seems to work fine.

Of course, all the work that went into this is open source...

Further Reading

Stream Your Way to Immediate Responses
2016 - the year of web streams

Happy coding ❤️