feat: customizeable content type parsing in @helia/verified-fetch about helia HOT 5 CLOSED

Providing a way to pass custom content type sniffer sounds sensible, but will be a very niche feature request if your default is something comprehenbsive like file-type with magic bytes sniffing.

I think the question we could ask is when is content-type relevant:

• If we use verified-fetch in JS the same way as fetch, the content-type header won't matter.
  End user will use .json(), .text(), .blob() etc themselves.
• If we use verified-fetch in service worker for web gateway implementation, then we pass response to browser renderer directly, and returned content-type matters. In this case hard-coding a few content types won't be enough anyway, and the user wants something more future-proof, like file-type.

That is to say, I think it is sensible to either:

• go with file-type everywhere (avoid maintaining "minimal list of types we support", ~5% bundle size increase does not sound like a lot when compared to UX/DX of content-type being taken care of)
• OR skip setting content-type by default entierely, and only use it gateway contexts, in which you use contentTypeParser to pass file-type that does the comprehensive magic bytes sniffing.

from helia.

My feeling is that if we don't need to do content type sniffing then let's not do it.

If we need to do it, we should do the minimum required (e.g. just support detecting a small subset of content types) and provide an extension mechanism for more comprehensive detection.

Given that we're billing this as fetch-like, most people will just do .json(), .blob(), etc and get on with things which suggests that we don't need to detect content types - we just try to process the data as the requested type and fail loudly if we can't.

There are valid use-cases for content detection though (e.g. service worker gateway) so allowing users to configure a mime type sniffer if they need it seems like a good compromise.

from helia.

FYI that file-type is fairly small compared to the entirety of @helia/verified-fetch (currently totals 560.2kb) at only 26.7kb:

from helia.

Mostly agree with @lidel and @achingbrain, though I don't have a strong inclination either way.

If we don't include magic-byte sniffing by default, it should be as easy as possible to configure so it works smoothly in service workers.

from helia.

Sounds good. Ill get a PR out today that will not do content-type unless passed a function for it

from helia.