Speculative Module Fetching

Qwik is able to load page and become interactive extremely fast due to its ability to startup without JavaScript. In addition to this, Speculative Module Fetching is a powerful feature that allows Qwik to pre-populate the browser's cache in a background thread.

The goal of Qwik is not to pre-populate the cache with the entire application, but to have already cached what's possible at that time. When the Qwik optimizer breaks apart the application, it's able to understand possible user interactions. And from this, it's just as important that it's able to understand what's not possible from user interaction, and to avoid loading those bundles.

• Pre-populating the Cache with a Service Worker
• Caching Request and Response Pairs
• Parallelizing Network Requests

Pre-populating the Cache Per Page

Each page load will pre-populate the cache with bundles that could be executed by the user, at that moment, on the page. For example, let's say that the page has a click listener on a button. When the page loads, the very first thing the service worker does is ensure the bundle for that click listener is already waiting in the cache. When the user clicks the button, and Qwik makes a request to the event listener's bundle, the goal is that the bundle is already sitting in the browser's cache ready to execute.

Pre-populating the Per Interaction

You can think of the page load as the first user interaction, which pre-populates the cache with what the next user interaction could be. When a follow-up interaction happens, such as opening a modal or menu, then Qwik will again emit another event with additional bundles that could be used since the last interaction happened. Pre-populating the cache not only happens on page load, but continuously as users interact with the application.

Pre-populate Cache Event

The recommended strategy is to use a service worker to populate the browser's cache. The Qwik framework itself should use the prefetchEvent implementation, which is already the default.

Pre-populating the Cache with a Service Worker

Traditionally, a service worker is used to cache most, or all, of the bundles that an application uses. Service workers are commonly seen only as a way to make an application work offline.

Qwik City, however, uses service workers quite differently to provide a powerful caching strategy. Instead, the goal is not to download the entire application, but rather to use the service worker to dynamically pre-populate the cache with what's possible to execute. By not downloading the entire application, resources are freed up, enabling the user to only request the necessary parts of the app they could use to complete their current task on the screen.

Additionally, the service worker will automatically add listeners for these events emitted from Qwik.

Background Task

An advantage of using a service worker is that it's also an extension of a worker, which runs in a background thread.

Web Workers makes it possible to run a script operation in a background thread separate from the main execution thread of a web application. The advantage of this is that laborious processing can be performed in a separate thread, allowing the main (usually the UI) thread to run without being blocked/slowed down.

By pre-populating the cache from within a service worker (which is a worker), we're able to essentially run the code in a background task, in order to not interfere with the main UI thread. By not interfering with the main UI we're able to help improve the performance of the Qwik application for users.

Interactively Pre-populating the Cache

Qwik itself should be configured to use the prefetchEvent implementation (which is also Qwik's default). When Qwik emits the event, the service worker registration actively forwards the event data to the installed and active service worker.

The service worker (which runs in a background thread) then fetches the modules and adds them to the browser's cache. “The main thread only needs to emit data about the required bundles, while the service worker’s sole focus is to cache those bundles. To achieve this, the service worker pre-populates the browser’s Cache.

1. If the browser already has it cached? Great, do nothing!
2. If the browser hasn't already cached this bundle, then let's kick off the fetch request.

Read more about Caching Request and Response Pairs.

Additionally, the service worker ensures that multiple requests for the same bundle do not happen at the same time. More about this in the Parallelizing Network Requests documentation.

Caching Request and Response Pairs

In many traditional frameworks, the preferred strategy is to use <link> with a rel attribute of prefetch, preload or modulepreload. However, there are enough known issues that Qwik has preferred to not make link the default prefetching strategy (though it still can be configured).

Instead, Qwik prefers to use a newer approach that takes full advantage of the browser's Cache API, which is also better supported compared to modulepreload.

Cache API

The Cache API, often associated with service workers, is a way to store request and response pairs in order for an application to work offline. In addition to enabling applications to work without connectivity, the same Cache API provides an extremely powerful caching mechanism available to Qwik.

Using the installed and activated service worker to intercept requests, Qwik is able to handle specific requests for known bundles. In contrast to the common way service workers are used, the default does not attempt to handle all requests, but rather, only known bundles generated by Qwik. The site's installed service worker can still be customized by each site.

An advantage of Qwik's optimizer is it also generates a q-manifest.json file. The manifest provides a detailed module graph of not only how bundles are associated, but also which symbols are within each bundle. This same module graph data is provided to the service worker which allows for every network request for known bundles to be handled by the cache.

Dynamic Imports and Caching

When Qwik requests a module it uses a dynamic import(). For example, let's say a user interaction happened, requiring Qwik to execute a dynamic import for /build/q-abc.js. The code to do so would look something like this:

const module = await import('/build/q-abc.js');

What's important here is that Qwik itself has no knowledge of a prefetching or caching strategy. It's simply making a request for a URL. However, because we've installed a service worker, and the service worker is intercepting requests, it's able to inspect the URL and say, "look, this is a request for /build/q-abc.js! This is one of our bundles! Let's first check to see if we already have this in the cache before we do an actual network request."

This is where the power of the service worker and Cache API comes in! Qwik first pre-populates the cache for modules the user may soon request within another thread. And better yet, if it's already cached, then there's no need for the browser to do anything.

Other benefits include Parallelizing Network Requests.

Parallelizing Network Requests

In the Caching Request and Response Pairs docs we explained the powerful combination of the Cache and Service Worker APIs. However, we can take it even one step further by ensuring duplicate requests are not created for the same bundle, and we can prevent network waterfalls, all from within the background thread.

Avoiding Duplicate Requests

As an example, let's say an end-user currently has a slow 3G connection. When they first request the landing page, as fast as this slow network allows, the device downloads the HTML and renders the content (an area where Qwik really shines). On this slow connection, it'd be a shame if they'd have to also download a few more hundred kilobytes just to make their app work and become interactive.

However, because the app was built with Qwik, the end-user doesn't need to download the entire application for it to become interactive. Instead, the end-user already downloaded the SSR rendered HTML app, and any interactive parts, such as an "Add to cart" button, can be prefetched immediately.

Note that we're only prefetching the actual listener code, and not the entire stack of the component tree render functions.

In this extremely common real-world example of a device with a slow connection, the device immediately starts to pre-populate the cache for the possible interactions that are visible by the end-user. However, due to their slow connection, even though we started to cache the modules as soon as possible in a background thread, the fetch request itself could still be in flight.

For demo purposes, let's say the fetching for this bundle takes two seconds. However, after one second of viewing the page, the user clicks on the button. In a traditional framework, there's a good chance that absolutely nothing would happen! If the framework hasn't finished downloading, hydrating and re-rendering, the event listener can't be added to the button yet. Which in turn means that the user's interaction would just be lost.

However, with Qwik's caching, if the user clicked the button, and we already started a request one second ago, and it has one second left until it's fully received, then the end-user only has to wait for one second. Remember, they're on a slow 3G connection in this demo. Luckily the user already received the full rendered landing page, so they're already looking at a completed page. Next, they're only pre-populating the cache with the bits of the app they could interact with, and their slow connection is dedicated to just those bundle(s). This is in contrast to their slow connection downloading all of the app, just to execute one listener.

Qwik is able to intercept requests for known bundles, and if a fetch in a background thread is already in flight, and then a user requests the same bundle, it'll ensure that the second request is able to re-use the initial one, which may already be done downloading. Trying to perform any of this with the link also shows why Qwik preferred to not make it the default, but instead uses the caching API and intercepts requests with a service worker.

Reducing Network Waterfalls

A network waterfall is when numerous requests happen one after another, like steps downstairs, rather than in parallel. A waterfall of network requests will usually hurt performance because it increases the time until all modules are downloaded, rather than each module starting to download at the same time.

Below is an example with three modules: A, B and C. Module A imports B, and B imports C. The HTML document is what starts the waterfall by first requesting Module A.

import './b.js';
console.log('Module A');

import './c.js';
console.log('Module B');

console.log('Module C');

<script type="module" src="./a.js"></script>

In this example, when Module A is first requested, the browser has no idea that it should also start requesting Module B and C. It doesn't even know it needs to start requesting Module B, until AFTER Module A has finished downloading. It's a common problem in that the browser doesn't know ahead of time what it should start to request, until after each module has finished downloading.

However, because our service worker contains a module graph generated from the manifest, we do know all of the modules which will be requested next. So when either user interaction or a prefetch for a bundle happens, the browser initiates the request for all of the bundles that will be requested. This allows us to drastically reduce the time it takes to request all bundles.

User Service Worker Code

The default service worker that is installed by Qwik City can still be controlled entirely by the application. For example, the source file src/routes/service-worker.ts becomes /service-worker.js, which is the script requested by the browser. Notice how its place within src/routes still follows the same directory-based routing pattern.

Below is an example of a default src/routes/service-worker.ts source file:

import { setupServiceWorker } from '@builder.io/qwik-city/service-worker';
 
setupServiceWorker();
 
addEventListener('install', () => self.skipWaiting());
 
addEventListener('activate', (ev) => ev.waitUntil(self.clients.claim()));

The source code for src/routes/service-worker.ts can be modified by the developer however they'd like. This includes opting-in, or opting-out, of setting up Qwik City's service worker.

Notice that the setupServiceWorker() function is imported from @builder.io/qwik-city/service-worker and executed at the top of the source file. If, and where, this function is called can be modified by the developer. For example, the developer may want to handle the fetch requests first, in which case they'd add their own fetch listener above setupServiceWorker(). Or if they didn't want to use Qwik City's service worker at all, they would just have to remove setupServiceWorker() from the file.

Additionally, the default src/routes/service-worker.ts file comes with an install and activate event listeners, each added at the bottom of the file. The callbacks provided are the recommended callbacks. However, the developer can modify these callbacks depending on their own app's requirements.

Another important note is that Qwik City's request intercepting is only for Qwik bundles, it does not attempt to handle any requests which are not a part of its build.

So while Qwik City does provide a way to help prefetch and cache bundles, it does not take full control of the app's service worker. This still allows developers to add their service worker logic without conflicting with Qwik.

Disabled During Development and Preview

One gotcha during development and using Vite's preview mode is that the service worker is disabled which also disables speculative module fetching. During development we want to always ensure the latest development code is being used, rather than previous code.

Speculative module fetching only kicks on a production build. To see speculative module fetching in action you'll need to run the production build on a server other than development or preview.

HTTP Cache vs. Service Worker Cache

Speculative module fetching may appear to not be working partly due to the various levels of caching. For example, the browser itself may cache requests in its HTTP cache, and the service worker may cache requests in the Cache API. Just emptying one of the caches may not be enough to see the effects of speculative module fetching.

Misleading Empty Cache and Hard Reload

When developers run Empty Cache and Hard Reload, it's a bit misleading because it actually only empties the browser's HTTP cache. It's not, however, emptying the service worker's cache. Even though the browser's HTTP cache is empty, the service worker still has the previous cached requests.

Additionally, when "Empty Cache and Hard Reload" is used, the browser sends a no-cache cache-control header in the request to the server. Because the request has a no-cache cache-control header, the service worker purposely does not use its own cache, and instead the browser performs the usual HTTP fetch again.

Emptying the Service Worker Cache

The recommended way to test specualtive module fetching is to:

• Unregister the service worker: In Chrome DevTools, go to the Application tab, and under Service Workers, click the "Unregister" link for the for your site's service worker.
• Delete the "QwikBuild" Cache Storage: In Chrome DevTools, go to the Application tab, and under Cache Storage on the left side, right click "Delete" on the "QwikBuild" cache storage.
• Do not hard reload: Instead of hard reloading, which would send a no-cache cache-control to the service worker, just click the URL bar and hit enter. This will send a normal request as if you were a first time visitor.

Note that this process is only for testing the speculative module fetching, and is not required for new builds. Each build will create a new service worker, and the old service worker will be automatically unregistered.