wicg / background-fetch Goto Github PK

Some background:

The backgroundfetched event fires in the service worker once the background fetch operation is complete, as in we can provide a full response for every request within the background fetch operation.

The event name is a little weird and past-tense because I felt like this may be too similar to the fetch event, which fires early on in fetch. I also considered backgroundfetchcomplete (which felt really long) and backgroundfetchend (which felt a little clunky).

@annevk is there any prior art you're aware of for the above?

The event constructor is currently BackgroundFetchEndEvent, which is wrong, as it should mirror the name. I'll fix that now.

Does it fire as soon as possible, and discard any in-progress fetches? Or does it wait for all fetched to settle?

On one hand, failing fast is nice, because it's fast, but you might end up throwing away a 99% downloaded movie because a piece of metadata 404'd.

It feels like we should reject on "manual" redirect modes to avoid dealing with opaque resources.

However, what about "follow"? If we hit a redirect, then need to resume the download, does the request go to the destination url or the original target? I suspect it's the target, but need to test what downloads do today.

https://wicg.github.io/background-fetch/

https://jakearchibald.github.io/background-fetch

The current proposal allows the browser to turn a single request into multiple range requests, allowing for pause/resume. However, what should we do if the request the user passes contains a range header?

We should probably prevent the browser making its own range requests for that resource in this case.

cc @paullewis

I wonder if we can reuse fetch() somehow. That way we don't have all the features of the Fetch API in two places. That already causes issues with the Cache API.

Perhaps with the controller/observer design it would be possible to hand those off to the background process somehow?

Restricting to CORS only makes it easier to deal with 404s etc. It also avoids adding another API where opaque responses use quota.

This could be entirely separate to the cache API and be called "background-fetch". The "bgfetchcomplete" event could hold a map of requests to responses, and it's down to the developer to put them wherever they want.

Allow devs to signal total size so progress bars can be displayed before headers have arrived for all resources.

const registration = await navigator.serviceWorker.ready;
const bgFetchJob = await registration.backgroundFetch.get(tag);

Here, bgFetchJob.fetches has properties:

• request - a Request.
• responseReady - a promise for a Response. Rejects if the fetch has terminally failed.

The response needs to be behind a promise, as we may not have headers yet. I tried having response which may be undefined, but that turns the object into a kind of snapshot, which doesn't work since the other parts are pretty active.

addEventListener('backgroundfetched', bgFetchEvent => {
  // …
});

Here, bgFetchEvent.fetches has properties:

• request - a Request.
• response - a Response.

We don't need a promise here, as we definitely have a response. However, the difference between this and the other .fetches is weird.

They could become the same object, and just live with an extra await to get the actual response even though it isn't needed, or the objects could be changed a bit more so this difference makes sense.

Something meaningful to display in the notification, eg the name of the movie/podcast.

If the user interacts with the notification should there be a separate event for that?

While the fetch is in progress it could take the user to a "more details" page. When the fetch is complete it could take them to the thing they downloaded/uploaded.

This should clarify what should happen if the browser profile is cleared too, as in incognito in Chrome.

Background fetch should be bound by the same rules as fetch().

Given that @igrigorik and I are planning on merging sendBeacon() into fetch() and this offers similar functionality, but for the response scenario rather than the request scenario, it might be nice to explore the overlap a bit.

(See w3c/beacon#27 (comment) onwards for context.)

The backgroundfetched and backgroundfetchfail events have init dictionaries that require a sequence of BackgroundFetchSettledFetches objects. (Note that it's not a sequence right now either.)

However, BackgroundFetchSettledFetches is not constructable. It should be.

Feels like there should be a BackgroundFetchSettledEvent that the fail event uses, which BackgroundFetchedEvent extends to add updateUI

Just the part that does the internal book-keeping and returns a new registration object.

https://wicg.github.io/background-fetch/#background-fetch-registration

A BackgroundFetchRegistration has a sequence of active fetches, whereas two events have a sequence of settled fetches. Each entry in such a sequence, of type BackgroundFetchFetches, contains a single request/response(Ready) pair, while the name implies its multiple.

Could we consider renaming them as follows?

One concern here is that they describe and contain a request, as opposed to being an actual request. We can stick Info in the name too, but that makes the types even longer. WDYT?

We made a big mistake with cache.addAll by considering 404 etc successful. We changed it so 404 is considered a failure. However, this means it can't be used for opaque resources, until we decide that response.ok already leaks for opaque responses by other means (<object>), and expose it.

Personally I'm flip-flopping on this, and think it could be dependent on the outcome of #3.

Excuse me bringing that in. Probably this topic is well thought through, but I couldn't find anything related in this repository.

If I wanted to abuse the background-fetch I'd do the following:

1. When someone visits my website I would
2. install a service worker and
3. start a dozen (or just one) of small file downloads (uploads?).
4. On the server side I'd abort the stream in the middle.
5. That would trigger backgroundfetchfail in the browser, then
6. I'd do my malicious actions (work as a bot in the DDOS botnet?) and
7. (re)start another file download. GoTo # 4.

What am I missing?

Background-fetching 1gb of data then adding it to the cache API should only use 1gb of quota.

This should work since responses can only be read once, but the spec should be explicit about it.

Currently, calling backgroundFetch.fetch(tag, requests) will reject if there's already an in-progress job with that tag name.

Should it be possible to add to an in-progress job. This may great weirdness in UI.

Whilst the APIs implicitly require a secure context since they hang off ServiceWorkerRegistration, the interfaces themselves are still exposed as window.BackgroundFetchManager etc in non-secure contexts.

It would be neater to hide them there by adding [SecureContext]. This matches the consensus in w3c/ServiceWorker#941 which added [SecureContext] to interfaces in the service worker spec (though I notice we haven't yet updated Blink's IDL to match).

Probably not for "v1".

Apps might want to have buttons to pause/resume an upload/download job. At least we can have it now for downloading jobs. Also, these buttons can be part of the browser UI (e.g. in the Android drop down notification).

Maybe also add backgroundFetchPause and backgroundFetchResume events. And allow browser to pause jobs. Reasons to pause a job:

• Not enough space (can happen after the download started)
• User tap to pause via the Android drop down notification or via app UI
• Device goes offline
• etc

Some background on the feature necessity.

We are developing an app to report incidents which can and often happen in rural no-coverage areas. The connectivity might be actually present occasionally or the coverage can be weak/slow (Australian thing). The incident media (photos, videos) should be uploaded with fewer possible retries.

Pausing and resuming upload sounds like a dream feature. I wish there was a standard to resume uploads. But as a workaround we can upload a large file contents in smaller chunks.

Hi,

I currently work for Kinderly and am scoping out a way to rewrite our current Flash/Air mobile app as a progressive web application. One of the key requirements is the ability to schedule fire-and-forget uploads of large files (photos and videos in addition to other data) when the user is offline, for automatic assumption and resumption when the user returns online. The use case is as follows:

Picture a nursery with a large garden, with one crappy BT Hub from 5 years ago. The manager there bought a bunch of cheap tablets¹ with just WiFi on them and told the staff that they need to use these to take EYFS observations and attach photos and videos to them. Now, in the far end of the garden, the WiFi is nonexistent so the app absolutely must work offline - so far that's not too much of a problem as service workers are already here. Then the staff return the devices to the office, where it can connect to WiFi again. What should happen at this point is that background sync should be initiated, and all those observations with media attached should start being uploaded to the server.

My question is, how much of this use case is covered by the current service worker spec as well as this proposal? On first reading of this example it seems like it may actually be quite difficult to pull off, requiring a request to be fired immediately and then put on hold by the cache. In my scenario, I'd like to be able to store all the media in say IndexedDB, and initiate uploads in response to some event that informs the service worker that it has network again (unsure if I can listen to online/offline events in a service worker, probably not).

¹ For now I'm assuming Android tablets which can run latest Chrome; I have no illusions that a shim Cordova app for iOS won't be necessary

Since only a single registration for a {service worker registration, tag} pair may be active at a time, having abort() return a void Promise will enable developers to avoid a race condition when having to re-register a background fetch with the same tag.

The API is intended for Service Workers and most of the time will expect that the download/upload completes a significant time after it's started. In that case, most consumers will be implementing event listeners to deal with success/failure.

If event listeners are going to be the expected use case for notification on status, I'd suggest you remove the "done" promise just to keep the API clean.

Currently Background Fetch supports any URI scheme supported by Fetch, with the added restriction that Background Fetch is only exposed in Secure Contexts since it hangs off ServiceWorkerRegistration, and so Mixed Content §should-block-fetch requires request urls to be a priori authenticated. This is defined to be either a data: URI or anything considered "Potentially Trustworthy" by Secure Contexts §is-url-trustworthy.

So in practice requests are currently limited to: https:, [loopback] http:, data:, wss:, file:, chrome-extension: (or similar), about:blank, or about:srcdoc.

I'd like to propose restricting to only https: and [loopback] http:, since:

• wss: doesn't seem to allow fetching resources.
• data:, file:, chrome-extension:, and about: are local requests that can be satisfied almost immediately, so the ordinary Fetch API should be preferred for these.

(one case that's debatable is file: URLs to network shares, for which a background fetch might make more sense; however since that's a rather niche usecase I'd rather start without it, and we can always add support later)

The actual fetching of resources. I guess this is the big one.

We need to think about which cases are "retryable", and which indicate terminal failure. Eg, a POST that results in a network failure may be "retryable", but a POST that results in a 403 may not. We also need to define how often a "retryable" request can be retried before failing, and any kind of delay before retrying.

Range requests probably don't make sense for non-GET requests.

It doesn't really make sense to use foreign fetch with background fetch, it should be skipped.

A common use case will be to display the transfer progress of a Background Fetch on a page. Right now that'd require the developer to (a) know the total download size, and (b) iterate over all activeFetches to read the length of the bodies of those that have completed. That's super expensive and complicated.

There's not much we can do about (a), but we can make (b) far more convenient:

interface BackgroundFetchRegistration : EventTarget {
    // Exposes the number of bytes that have been downloaded so far.
    readonly attribute long totalDownloadedSize;

    // Will be fired _at some frequency_ when progress happens.
    [Exposed=Window] attribute EventHandler onprogress;
}

(On a tangent, do we need something similar for totalUploadedSize?)

I guess the event type could just be a simple event without properties, as the developer can re-read totalDownloadSize and totalDownloadedSize from the BackgroundFetchRegistration instance. It also implies that we need to continue to return the same instances from e.g. BackgroundFetchManager.get() instead of creating new ones every time, and keep them updated. How would we define that? Would we be expected to keep updating totalDownloadedSize without event listeners?

Since you give access to the Request object, what if someone drains it? Or is it a clone of sorts? Wouldn't that be rather wasteful?

Seems like a nice privacy feature, and gives the user something to interact with afterwards.

Worth looking at what browsers do right now with video range requests.

They're almost identical, and could be replaced by a single event that has a property to indicate success/failure.

I tried this in an earlier draft and it felt a bit funky.

There's a small benefit of having them as different events: If the user doesn't handle backgroundfetchfailed explicitly, the browser is in a better position to change the download notification itself.

With just RequestInfo you don't get very far, right?

A bg fetch could be 10,000+ requests & responses (think of a chunked movie). We could be creating ourselves a performance problem we can't later work around.

bgFetchJob.activeFetches doesn't feel like a common thing to use. The only place I use it in the examples is to play a podcast that's in-progress of downloading. Notably I had to create my own matching function that found the single entry I was interested in.

In success events, it's likely that you'll want to add all of bgFetchEvent.fetches to a cache.

In error events, you may wish to inspect which items succeeded & failed, or you might just show a message, so bgFetchFailEvent.fetches isn't always needed.

partial interface BackgroundFetchRegistration {
  readonly attribute BackgroundFetchActiveFetches activeFetches;
}

interface BackgroundFetchActiveFetches {
  // .match works like the cache API
  Promise<BackgroundFetchActiveFetch> match(RequestInfo request, optional CacheQueryOptions options)
  iterable<Promise<BackgroundFetchActiveFetch>>;
}

partial interface BackgroundFetchedEvent : BackgroundFetchEvent {
  readonly attribute BackgroundFetchFulfilledFetches fetches;
}

interface BackgroundFetchFailEvent : BackgroundFetchedEvent {
  readonly attribute BackgroundFetchSettledFetches fetches;
};

interface BackgroundFetchSettledFetches {
  // .match works like the cache API
  Promise<BackgroundFetchSettledFetch> match(RequestInfo request, optional CacheQueryOptions options)
  iterable<Promise<BackgroundFetchSettledFetch>>;
}

interface BackgroundFetchFulfilledFetches : BackgroundFetchSettledFetches {
  // We might not include this for v1
  Promise<void> putAllInCache(Cache cache);
}

I'm a little concerned about using .fetches for two slightly different objects.

Playing a podcast as it's background-fetching

addEventListener('fetch', event => {
  if (isPodcastAudioRequest(event.request)) {
    const podcastId = getPodcastId(event.request);

    event.respondWith(async function() {
      const bgFetchJob = await self.registration.backgroundFetch.get(`podcast-${podcastId}`);

      if (bgFetchJob) {
        // Look for response in fetches
        const activeFetch = await bgFetchJob.activeFetches.match(event.request);
        return activeFetch.responseReady;
      }

      // Else fall back to cache or network
      const response = await caches.match(event.request);
      return response || fetch(event.request);
    }());
  }
});

This example got quite a bit shorter thanks to .match.

Caching a podcast after background fetching

addEventListener('backgroundfetched', event => {
  if (event.id.startsWith('podcast-')) {
    event.waitUntil(async function() {
      // Get podcast by ID
      const podcast = await getPodcast(/podcast-(.*)$/.exec(event.id)[0]);

      // Cache podcasts
      const cache = await caches.open(event.id);
      const promises = [...event.fetches].map(async p => {
        const {request, response} = await p;
        return cache.put(request, response);
      });

      await Promise.all(promises);
      event.updateUI(`Downloaded ${podcast.showName} - ${podcast.episodeName}`);
    }());
  }
});

This got slightly more complicated, but not by much. If add putAllInCache:

addEventListener('backgroundfetched', event => {
  if (event.id.startsWith('podcast-')) {
    event.waitUntil(async function() {
      // Get podcast by ID
      const podcast = await getPodcast(/podcast-(.*)$/.exec(event.id)[0]);

      await event.fetches.putAllInCache(await caches.open(event.id));
      event.updateUI(`Downloaded ${podcast.showName} - ${podcast.episodeName}`);
    }());
  }
});

…much simpler.

Summary: I suspect we need a way to minimize the number of events fired into the SW if the site is downloading files in chunks. I suspect we also want to be able to defer the events for some time.

Observations:

• Often these downloads/uploads will be occurring in the background on low memory devices
• On such devices it may be difficult to wake up a SW in the background, or keep a renderer alive
• Some sites may want to download many files (e.g. songs for an album) or chunk files and download them individually (e.g. parts of a movie).

I'm presuming that the current design coalesces all files downloaded in one call to the API into one event fired in the SW, is that correct? If not I think we need to support some form of coalescing.

Do you see any issues with a browser deferring the events until it is next opened in the foreground to maximize the chance of success on low memory devices?

Thanks

We should define that a tag gets released whenever the backgroundfetch* events fire.

Filed against Chrome here: https://bugs.chromium.org/p/chromium/issues/detail?id=709921

Cover that removing an item from background fetch active fetches doesn't mean the data is removed from disk.

Removing from disk happens when references to those requests/responses, along with the methods to retrieve them, such as BackgroundFetchSettledFetches, are GC'd.

Maybe just merge a version of the spec with it.

It feels like something we shouldn't do by default, as making the same "POST" request twice may have side effects.

Perhaps we should provide an opt-in to this? Something that says "My server is coded in a way that prevents double-posting. Please retry POSTs".

+@beverloo @johnmellor

I had been thinking that the backgroundfetchfail could return a set of results, only some of which failed and some of which succeeded. However, the attribute on the event is called "failedFetches". On the backgroundfetched event, the list of result/response pairs is called "completedFetches".

Proposal: Use "completedFetches" for the attribute on both BackgroundFetchedEvent and BackgroundFetchFailedEvent. Then embed the success or failure of each request in the response?

This may be a moot point depending on the outcome of #23.

I'm in two minds about this.

We could add these options to background sync, then developers could trigger the background-cache from the sync event. This seems like a nice lower-level way of doing it.

Alternatively, we could add the feature directly to background-cache. This would be higher-level, but would allow the OS to pause the download if the connection changes.

Use FrozenArray<>.