Offline-Capable Progressive Web Apps for the Modern Web

Native applications are great – they allow a user to easily discover a brand through an app store they are familiar with, are super easy to manage and provide some quantum of functionality when connectivity is sparse or non-existent, as well can be extremely performant even when the network is slow. While I advocate for native apps when the time is right, sometimes they can be a bit heavy-handed; between developer fees, software licensing, and on-going maintenance, sometimes a native app just isn’t the right fit. However, the year is 2022 and we can bring many native app advantages to the modern web with just a little know-how (and the correct support, of course).

Enter the Progressive Web Application

Progressive Web Applications (PWAs) are a great way to bring app-like features to your website for users to enjoy, with some caveats. First and foremost; not all browsers support the core APIs needed to make a PWA, and furthermore, not all browsers that do support some features may not support all features of available to a PWA. Keep that in mind as you build; unless you have a reasonable expectation that your users will be on a subset of modern browsers, design the features of your PWA around being entirely optional at best.

For the purposes of this exercise, let’s assume a few things:

  1. We want to add a PWA to an existing React.js web app
  2. Our website is backed by a headless CMS that will serve the content and pages we want to view offline or semi-offline
  3. We want our PWA to served cached content first, but fetch any updated content in the background to serve next time the user visits
  4. We want our website to be “installable” on supported browsers. That is to say, we want the PWA to be pinned to a home-screen or menu bar if a user opts-in to offline-capabilities (for an example of this, got to pokedex.org and select the prompt in the upper-right to install for supported browsers)

Before we Begin…

As a reminder, many of the features of PWAs are still experimental. For best results while following along here, I recommend using a recent version of Google Chrome (or another chromium-based web browser). Personally, I will be using the Chromium version of Microsoft Edge. That said, let’s dive in.

I Work Offline Sometimes

We’ll need a few things to get started. First and foremost, let’s add a manifest to the root of our application to let browsers know that we have a PWA with optional features a visitor can take advantage of. I like to add mine to the root of my project, but it can be placed anywhere as long as it ends up in the root directory of your deployed website. The last part is critical – we will be using the root directory of our website folder structure a lot to take full advantage of what our PWA has to offer (more on manifests here).

Consider the following manifest:

{
  "name": "I Write Code Sometimes Example PWA",
  "icons": [
    {
      "src": "favicon.ico",
      "sizes": "64x64 32x32 24x24 16x16",
      "type": "image/x-icon"
    },
    {
      "src": "logo192.png",
      "type": "image/png",
      "sizes": "192x192"
    },
    {
      "src": "logo512.png",
      "type": "image/png",
      "sizes": "512x512"
    }
  ],
  "start_url": "/home",
  "display": "standalone",
  "theme_color": "#490e6f",
  "background_color": "#ffffff"
}

The existence of the manifest file suggests to a browser that there may be PWA features to take advantage of. After that, the manifest largely exists to define how our PWA will appear while it is being used while in an installed state. The official documentation elaborates further:

The web app manifest provides information about a web application in a JSON text file, necessary for the web app to be downloaded and be presented to the user similarly to a native app (e.g., be installed on the homescreen of a device, providing users with quicker access and a richer experience). PWA manifests include its name, author, icon(s), version, description, and list of all the necessary resources (among other things)

https://developer.mozilla.org/en-US/docs/Web/Manifest

Be sure to also reference the manifest in your HTML header, so that visitors know about it:

<!DOCTYPE html>
<html @Html.RenderLangAttribute()>
<head>
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
    <title></title>

    <!-- Service Worker Manifest -->    
    <link rel="manifest" href="~/manifest.json" />

Once the manifest is settled, it can be worthwhile to test a deployment to make sure it is placed properly. Open the dev tools in your browser of choice and locate the “Application Tab” – this tool will be extremely useful in debugging and testing our PWA.

The application pane

Here, we can see if everything is working properly. In my case, I have some image assets that need cleaning up (inferred as the .ico file I had automagically generated). Read through any warning (or errors) and determine if any must be resolved before proceeding. In my case, I’ll correct the cosmetic warning later and continue on for now. Since we have not yet done anything to register our PWA, you will also likely see the following warning, which we’ll work to resolve shortly:

No service worker installed

Registration and Proof of Service Worker, Please

To resolve the instability message above, we’ll need to register what’s called a Service Worker. A service worker acts as proxy between web servers and browsers to regulate content going to and from the client. Before we can register and activate a service worker, we’ll need to check if it is supported using a simple js call like this one:

Checking for service worker support in the console

This is just plain old javascript, so you can run it right in your dev tools console if you’re curious. Let’s get to registering our service worker by creating a new react component, which will look like the following:

The administration page for servicing offline mode

Note; in some cases, you may want to register your service worker automatically, which you can totally do! Just have the methods we’re about to discuss called on page load (or similar) to get started. In this example, however, I’m anticipating north of 500MB of content, which can be slow to cache, so I’d like my visitors to explicitly opt-in to our PWA experience.

function registerValidSW(swUrl: string) {
  navigator.serviceWorker
    .register(swUrl)
    .then(registration => {
      // do something meaningful here if needed
      };
    })
    .catch(error => {
      console.error('Error during service worker registration:', error);
    });
}

So, if we assign this function as the effect of a button click we can now manually opt-in to register our service worker. In my use case, I want to immediately start fetching content to save offline, so I’ll rpelace the // do something meaningful here line above with some meaningful code, like so:

      registration.onupdatefound = () => {
        const installingWorker = registration.installing;
        if (installingWorker == null) {
          return;
        }
        installingWorker.onstatechange = () => {
          if (installingWorker.state === 'installed') {
            if (navigator.serviceWorker.controller) {
              // At this point, the updated pre-cached content has been fetched,
              // but the previous service worker will still serve the older
              // content until all client tabs are closed.
              console.log(
                'New content is available and will be used when all ' +
                  'tabs for this page are closed. See https://cra.link/PWA.'
              );

              // Execute callback
              if (config && config.onUpdate) {
                config.onUpdate(registration);
              }
            } else {
              // Service Worker is installed and we should request the initial sync
              navigator.serviceWorker.ready.then(function (swRegistration) {
                if (swRegistration.sync) return swRegistration.sync.register('');
                else {
                  console.warn("Browser does not support background sync! Pages will still cache when visited")
                }
              });

              // Execute callback
              if (config && config.onSuccess) {
                config.onSuccess(registration);
              }
            }
          }
        };
      };

The most important part here is that when we detect the service worker is “ready”, we can kick off the initial sync process, described in the file that actually contains our service worker.

The Sync Process

Since we want to cache content in advance, the sync process has its work cut out. As a caveat – sync isn’t supported by all browsers yet, so make sure you understand the implications of that on how your users might interact with your PWA before proceeding to use it. Let’s take a look at the following code:

// A List of endpoints we expect to get content from
const endpoints = ["/api/assets/", "/api/pages/"];

let failedRequests = [];
let successfulRequests = [];
let retryRequests = []

let sumRequests = 0;
let totalRequests = 0;
let totalRetry = 0;

self.addEventListener("sync", (event) => {
  self.skipWaiting();

  event.waitUntil(
    caches.open(CACHE_NAME).then(async function (cache) {

      let newContent = [];

      failedRequests = [];
      successfulRequests = [];
      retryRequests = []

      sumRequests = 0;
      totalRequests = 0;
      totalRetry = 0;

      // Fetch content, which should result in an array of "pages" and "assets" known to the site
      // plus important content we may need to cache locally
      for(let index = 0; index < endpoints.length; index++) {
        let r = endpoints[index];

        await fetch(r)
          .then((response) => response.json())
          .then(async (content) => {
            console.info(`Endpoint ${r} yielded ${content.length} items for cache`);
          
            totalRequests += content.length;

            newContent.push(...content);
          });
      }

      // Try to get individual items one by one
      for(let r = 0; r < newContent.length; r++) {
        let c = newContent[r];
        await fetchItem(cache, c);
      }

      // Retry boundary
      for (totalRetry + 1; totalRetry < 10; totalRetry++) {
        retryRequests = []; // Reset retry requests

        // Any failed requests?
        if (failedRequests.length > 0) {
          retryRequests.push(...failedRequests); // Copy Failed requests into Retry
          failedRequests = [];

          // Retry to get individual items one by one
          for(let j = 0; j < retryRequests.length; j++) {
            let rr = retryRequests[j];
            let t = rr.split(location.origin).pop();

            console.warn(`RETRY fetch for ${t}`)
            await fetchItem(cache, t);
          }

        } else {
          console.info("Content sync complete.");
          return;
        }
      }
    })
  );
});

For the major beats here, we need to fetch a list of pages and website assets from two known endpoints provided by our CMS. As we attempt to fetch and store content and assets, we also want to keep track of progress so we can use the message protocol (more on that later) to update the UI and show end-users progress of the download, rather than rely on the developer tools.

Other Service Worker Events

While there are a variety of service worker events you may want to leverage, note that not all are widely supported. That said, here are some of the events we’ll need for this example, along with some caveats of each. Check out the documentation for full details about each:

register

This is the most basic Service Worker function. Calling this activates the service worker and allows for other functions to take place. This action is widely supported, and could be a good spot to download content… if your content is small enough! If activate runs too long while waiting for content to be cached, it may be terminated and won’t complete, causing any other event requests to fail .

install

This function is widely supported, but recently deprecated. Install is a good place to check for updates to the service worker itself (which will then need to be “activated” when the website is next visited) or download additional content for offline use.

sync

This function is not widely supported, and serves as a supplemental install function. Use this to compare content that is cached to what might exist on a remote source and determine if local assets need to be updated.

fetch

This function is widely supported, and is triggered whenever a request is made to a supported subdomain the service worker may want to cache. Use this function to determine your service worker strategy: should you serve from the cache first, or serve from the network first? Add logic here to make that determination and enforce any business rules about content sync

message

This function is widely supported, and sends content to connected message listeners. See below for more details.

Messaging Connected Clients

If you’ve been monitoring your PWA from the developer tools, you’ll likely notice progress as it happens either from the cache section of the Application tab or from strategically placed console.log lines. However, we can make use of a special type of service worker event to affect the UI if desired. Enter the “message” event:

self.addEventListener("message", function (event) {

  if (!self.clients) return;

  // Get all the connected clients and forward the message along.
  var promise = self.clients.matchAll().then(function (clients) {
    var senderID = event.source.id; // contains the ID of the sender of the message.

    clients.forEach(function (client) {
      // Skip sending the message to the client that sent it.
      if (client.id === senderID) {
        return;
      }

      client.postMessage({
        client: senderID,
        message: event.data,
      });
    });
  });

  if (event.waitUntil) {
    event.waitUntil(promise);
  }
});

To listen for these events, we can set up a simple hook in the React.js component we want to update, like so:

 const [manifest, setManifest] = useState<AdministrationMessage>({
    failed: 0,
    succeed: 0,
    total: 0,
    expected: 0,
    retry: 0,
    error: [],
  });  

useEffect(() => {
    function handleMessageEvent(event: any) {
      if (!event.data.message) return;

      setManifest(event.data.message);
    }

    navigator.serviceWorker.addEventListener('message', handleMessageEvent);
    window.addEventListener('message', handleMessageEvent);

    if (manifest.total > 0 && manifest.expected > 0)
      setProgress((manifest.total / manifest.expected) * 100);

    return () => {
      navigator.serviceWorker.removeEventListener(
        'message',
        handleMessageEvent
      );
      window.removeEventListener('message', handleMessageEvent);
    };
  }, [manifest, swState]);

So, now while we execute a sync function in the service worker, we can update all connected clients about the progress. It’s important to note that “connected clients” in this case is only browsers that has the service worker installed and a tab open where our react component is being rendered. So, if you have three tabs open, all three will update in sync! But since all clients are “local”, if you have multiple devices, each with a tab open that has a message event listener, each will only be aware of it’s respective, local service worker instance – that is to say, this operation will not be able to message all clients across the network.

The Progressive Web App Experience

With our message listeners in place we’re ready to look at the final experience for users. Assuming our clients are using the correct browser, we can design an admin experience any way we like that will:

  • Allow the user to opt-in content download for offline use, caching the entire website on-demand
  • Allow the user to opt-out, removing content cached offline
  • Allow the user to monitor content sync progress or force a fresh content sync
  • Automatically update content after a page is visited if an update exists on the CMS

With a little UI magic, that might looks something like the following:

The Service Worker in action, actively caching content from the CMS

Plus, once the Service Worker is registered, supported browsers will prompt the user to add the site as a standalone app. You can even try this WordPress, which registers a service worker automatically when visited!

WordPress prompting a user to install the website as a PWA. Clicking Install allows some content to be available offline

The (Reduced) Progressive Web App Experience

Now, not everyone may be using a browser that supports all the features we made use of. Your use case may very, but for the example used in this document, here’s a list of what features a user may expect if they are using, say, Safari on a Mac.

  • Allow the user to opt-in, caching the content of each page after it is visited
  • Allow the user to opt-out, removing content cached offline
  • Automatically update content after a page is visited if an update exists on the CMS

…which is still a pretty good experience! Especially if the CMS is serving rich media content that may take time to fetch, any users who opt-in to our PWA experience can expect each successive visit to a page to render nearly instantly, without affecting the render time of first visit. Pretty neat!

Pokedex PWA – a simple PWA demo with useful code samples

Create React App + PWA – how to start a CRA project with PWA support out-of-the-box

Service Worker Spec on MDM – manual on all the supported Service Worker APIs

Service Worker Cookbook – a repository of a variety of common Service Worker templates

2 thoughts on “Offline-Capable Progressive Web Apps for the Modern Web

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s