Address a TODO in the code to validate meta CSP elements to ensure that they don't include forbidden directives: sandbox, report-uri, and frame-ancestors.

Enable capo extensibility by publishing the capo ruleset to npm.

• Publish to npm
• Write consumer-facing docs

It could be useful to have a permanent link to specific capo.js results. I really like how the Lighthouse viewer uses GitHub gists as a storage layer, which completely absolves the LH developers of any data storage concerns (ownership, costs, legal).

There are a few use cases:

• An extension user wants to take a "snapshot" of the dynamic capo.js results to share with their development team.
• I want to showcase a few different validation features of capo.js by using preset HTML snippets
• A user wants to get a live look at the same URL periodically (URL persistence already exists)

If a gist ID is passed into the demo page, we should be able to read from the gist without an API. The contents could be added to the raw HTML field.

TODO: need to think through what exactly we'd be validating but it seems useful to include HTTP response headers in the scope of the tool. Only the HTML document should be validated.

According to Stack Overflow and this blog post shrink-to-fit is a non-standard directive briefly supported by older versions of Safari (9.0–9.2 circa 2018).

Capo will warn if it sees this directive, flagging it as invalid:

❌ Invalid viewport directive "shrink-to-fit".

Here's an example of it in the wild: https://rviscomi.github.io/capo.js/user/demo/?url=https%3A%2F%2Fwww.cnn.com%2F (cnn.com)

<meta name="viewport" content="width=device-width,initial-scale=1,shrink-to-fit=no">

We could provide better messaging for developers who believe that this directive is valid and they still need it, similar to the IE-specific obsoletion warnings.

On https://www.youtube.com/ there's an origin trial registered to https://youtube.com with isSubdomain set to true:

{
    "origin": "https://youtube.com:443",
    "feature": "PrivacySandboxAdsAPIs",
    "expiry": "2023-09-19T23:59:59.000Z",
    "isSubdomain": true
}

capo.js warns that the origin is invalid. There shouldn't be any validation warnings for this origin trial because the isSubdomain check permits it to be used by the www subdomain.

The blanket rule for all meta[http-equiv] tags is to put them first, but there's more nuance than that. Some origin trials alter the behavior or feature support of the page and their meta tags really would benefit from coming first. Other origin trials are less consequential.

To help differentiate between them and for developers to get a better sense of urgency if a super important OT is too low in the head, include some metadata logging.

@samdutton created this excellent OT debug tool, which includes a decoder function for the OT token:

function decodeToken(token) {
    const buf = base64decode(token);
    const view = new DataView(buf.buffer)
    const version = view.getUint8()
    const signature = buf.slice(1, 65)
    const length = view.getUint32(65, false)
    const payload = JSON.parse((new TextDecoder()).decode(buf.slice(69, 69 + length)))
    return {payload, version, length, signature}
}

It doesn't look like there would be any licensing incompatibilities if we integrated that into the tool with proper attribution.

Some features like static head evaluation seem to have buggy edge cases (#31). Other features like validation might not be useful to all users. Create a way for users to enable/disable features or customize the script in some ways.

Features:

• static/dynamic head
• validation
• automatically log on page load (extension)
• validation error count badge (extension)

Customizations:

• color palette

Maybe I missed them but can't seem to find type definition for this lib.

Would be nice to be able to use this in node.js environment where html is passed as argument instead of script interacting with the page directly in the browser. Also for the results to be JSON rather than console.log

Is this something you're thinking of?

Can we have a chrome extension for capo.js? Great work btw.

In HTTPArchive/custom-metrics#12 (comment) @jroakes shared a screencast of some code for a document to parse its own body. Taking inspiration from that, I've modified the code a bit to produce a snippet that yields the entire contents of the static (server-rendered) <head>:

async function parse_own_body() {
    const url = document.location.href;
    let response = await fetch(url);
    let responseText = await response.text();
    return responseText;
}

let html = await parse_own_body();
html = html.replace(/(<\/?)(head)/ig, '$1static-head');
const staticDoc = document.implementation.createHTMLDocument("New Document");
staticDoc.documentElement.innerHTML = html;
const staticHead = staticDoc.querySelector('static-head');
staticHead;

The insight I had was that by renaming the head element to anything else, we can circumvent the HTML parser's behavior to truncate on invalid elements. Combined with fetching the raw HTML from the server, this should give us a pristine copy of the original head to use for capo analysis.

This screenshot shows it working on the NYT site.

It should be possible to drop this approach into capo.js and fall back to the dynamic head as needed—I'd imagine some CSP rules blocking this use of fetch.

There might be sandbox limitations of doing something like this in a Chrome extension. I'll investigate.

Content model:
If the document is an iframe srcdoc document or if title information is available from a higher-level protocol: Zero or more elements of metadata content, of which no more than one is a title element and no more than one is a base element.
Otherwise: One or more elements of metadata content, of which exactly one is a title element and no more than one is a base element.

https://html.spec.whatwg.org/multipage/semantics.html#the-head-element

Metadata content is content that sets up the presentation or behavior of the rest of the content, or that sets up the relationship of the document with other documents, or that conveys other "out of band" information.

• base
• link
• meta
• noscript
• script
• style
• template
• title

https://html.spec.whatwg.org/multipage/dom.html#metadata-content-2

Ensure there is:

• exactly one <title>
• no more than one <base>
• only the listed valid elements

In general, the WHATWG supports a limited set of keywords that are valid attribute values for http-equiv:

• content-language
• content-type
• default-style
• refresh
• set-cookie
• x-ua-compatible
• content-security-policy

Notable omissions include:

• origin-trial
• etag
• x-* (besides x-ua-compatible
• cache-control
• expires
• pragma
• accept-ch
• content-style-type
• content-script-type

These are all http-equiv attribute values used by over 100k pages, according to HTTP Archive, in descending order of popularity.

WITH meta AS (
  SELECT
    page,
    LOWER(JSON_VALUE(meta, '$.http-equiv')) AS http_equiv
  FROM
    `httparchive.all.pages`,
    UNNEST(JSON_QUERY_ARRAY(custom_metrics, '$.almanac.meta-nodes.nodes')) AS meta
  WHERE
    date = '2023-06-01' AND
    client = 'mobile' AND
    is_root_page
)


SELECT
  http_equiv,
  COUNT(DISTINCT page) AS pages
FROM
  meta
WHERE
  http_equiv IS NOT NULL
GROUP BY
  http_equiv
ORDER BY
  pages DESC

The biggest one that jumps out to me is origin-trial, which is used on ~375k pages. Given that it is explicitly supported and endorsed by Chrome, Edge, and Firefox (Safari doesn't support origin trials) I've left a comment on the WHATWG issue recommending its standardization.

I don't think capo.js should complain about spec validity for these keywords as long as browsers support them. But there are some specific usages worth validating.

http-equiv=content-type

According to the W3C spec, the content attribute of a meta[http-equiv=content-type] tag must be set to a "specially formatted string providing a character encoding name... in exactly the following order":

1. The literal string "text/html;".
2. Optionally, one or more space characters.
3. The literal string "charset=".
4. One of the following:
   • For documents in the HTML syntax: A character encoding name.
   • For documents in the XML syntax: Any case-insensitive match for the string "UTF-8".

The WHATWG further requires that the character encoding name be exactly utf-8 and that:

A document must not contain both a meta element with an http-equiv attribute in the Encoding declaration state and a meta element with the charset attribute present.

capo.js should validate that HTML5 pages set a charset to utf-8 and don't have redundant meta tags. Not sure about HTTP header vs meta tag redundancy, but that's also worth exploring (related #59).

According to the spec:

The Content-Security-Policy-Report-Only header is not supported inside a meta element.

Add a validation check to ensure that this directive isn't included in a <meta http-equiv> element.

Make sure stuff works

Make it easier for users to tell what version their snippet is, ie if it's out of date.

Consider using __CAPO__.VERSION

I just learned about capo and think it's very cool. I applied it to my own site and found that I could fix some ordering, but due to the way that Astro injects CSS and scripts at the end of the <head>, I wasn't able to entirely make capo happy. I noticed that the docs site is built using Astro's starlight, so has the same issue.

I wondered if capo could be used to automatically re-order the items in the <head> so that it could be run during the build process of a site. That way you can write the elements out in whatever order makes sense to you, and frameworks can inject parts in whatever order works for them, and then a capo plugin could come in and set the order straight.

Is this possible with how capo works today? And if not, is it something that capo could be extended to do?

1. Run the extension on https://web.dev/
2. Click the red element that sticks out at the end
3. Check the console log

There are a couple of discrepancies that I noticed between the actual element and what's getting logged:

• The token is missing 0= at the end

Ah3H7DwyoUUsaRQdSySa1hMCS/JFQn/VVmrQODVDnRJGH9mU/uG6G0Uhh+4atnFGAoiEwDq+r9TzCyBi7f7wRw4AAABfeyJvcmlnaW4iOiJodHRwczovL3dlYi5kZXY6NDQzIiwiZmVhdHVyZSI6IlNwZWN1bGF0aW9uUnVsZXNQcmVmZXRjaEZ1dHVyZSIsImV4cGlyeSI6MTY5NDEzMTE5OX

• Expiry is missing

I'd expect the metadata to match DevTools:

We don't have this bug when logging the full list of elements:

Seems like some kind of serialization issue, or getting the custom validations specifically for the popup.

In 1.5.2 viewport-fit= cover is flagged as invalid. While it's indeed not mentioned on mdn, I think it should not be flagged as an issue because both iOS and Android implement it.

For example, https://rviscomi.github.io/capo.js/user/demo/?url=https%3A%2F%2Frviscomi.dev%2F:

The origin trial is not invalid, but because it's being validated on rviscomi.github.io, capo.js incorrectly assumes that's the actual 1P origin, rather than the page being tested, rviscomi.dev in this case.

To fix, capo.js needs to explicitly take the test origin into consideration while running the validation.

Hey Rick, I hope you're well. I just wanted to play with the new extension and it's crashing for my site. 🫣

Might be because there's too much inline styles? 😅

Capo positions scripts before meta data.

There is an issue with that though. If a script in the head tag introduces a body tag - example with a Google Tam Manager script and advertiser could add in an <iframe> tag as part of their advert script for tracking, then when the page is read by Google (and probably other search engines) it will read the body tag and will assume it has reached the body and will from that point ignore any further head tags. This can lead to meta descriptions and canonicals being missed completely. My advice, because of this, has been to always put scripts last in the head unless you know what is in them - tag managers mean you will probably not.

But perhaps this is beyond the scope of this very good project!

Hey! Just wanted to mention that this tool is awesome!
Although I am having one issue with expanding the visual output from the script into a detailed view.

I ran the script on the citi.com website and this is what I see:

When logging the sorted head, omit any elements that have been previously flagged as invalid/discouraged.

Related to #70, I think capo.js should go a step farther and de-weight http-equiv elements that are invalid.

For example, on https://github.com/:

None of these http-equiv=x-* elements are meaningful and there's no compelling reason why they must be loaded before other more important elements (title, preconnect, style, etc).

In cases like these where http-equiv is known with high confidence to be ignored by the browser, assign a weight of 1 (lowest).

Effectively this is the list of keywords provided by the HTML spec plus a few extras that browsers care about—origin-trial being one, but there are others.

The core capo logic is effectively duplicated across the snippet, WPT custom metric, BQ function, and now the Chrome extension.

Devise a way to have all of the core logic defined in one place and integrated into each surface. This will probably require a build process.

The Chrome Web Store gives me some usage stats. There might be some buggy behavior in non-Mac OSes worth investigating.

41% of installs are on Windows, 9% on Linux

51% of uninstalls are on Windows, 17% on Linux

If anyone has Windows or Linux and is willing to file any bugs with the extension, please do!

Current behavior:

I see this on https://www.amazon.com/ when I'm logged in using a desktop device. When I emulate a mobile viewport, they adaptively serve a meta viewport, albeit with validation warnings of its own:

Given that meta viewports are really only useful for mobile devices, it's not necessarily bad for a desktop site to omit one. The thing is that it's hard for Capo to know whether we're looking at the desktop version of a site or whether it adaptively serves a meta viewport for the mobile version.

I'd be interested to hear feedback on whether it's generally a good idea to always have one, in which case we should keep the validation warning, or if there are too many use cases where omitting it is valid and removing the warning would be better.

In the BigQuery README, httparchive.fn.CAPO has a broken link.

To analyze pages in HTTP Archive, pass the HTML response body to the httparchive.fn.CAPO function:

Getting this exception when testing on https://www.linkedin.com/feed/:

capo.js:1 Uncaught (in promise) DOMException: Failed to execute 'querySelectorAll' on 'Element': 'LINK[id="ui-theme-dark"][rel="stylesheet"][href="https://static.licdn.com/sc/h/f2zpda4bn629123l4t5564326"][%theme_dark_disabled%=""]' is not a valid selector.

From @ebizindia: https://twitter.com/ebizindia/status/1683146602113044486

I agree that it doesn't make much sense to preload after the image already starts loading, so capo.js isn't necessarily right here. But I still think the priorities are correct. So rather than find a way to preload first, the issue here seems to be that it's not needed at all.

Add a validation warning against unnecessary preloads. Check if the href of the preload matches the href/src of another resource in the head.

There's a UX issue that a service worker might be able to help with:

When there's high latency to load the HTML resource, it takes a noticeably long time from clicking the extension to seeing the color bars. It appears blank for several seconds.

The same thing happens with the DevTools snippet: there's a delay from running the script to seeing the logs.

The extension is able to use a service worker to run in the background before the icon is clicked. We could use this to preload the HTML resource, so that when the icon is clicked, the response is immediately available. Unfortunately, this doesn't make sense for the snippet.

The same service worker could be used for a feature mentioned in #44, showing a badge with the number of validation warnings.

Consider whether capo.js should expand its validation beyond the head, if elements that are only expected to exist in the head are found outside of it.

According to https://rviscomi.github.io/capo.js/user/rules/#-8-asynchronous-scripts

<script async src>

The following scripts were included in the Asynchronous scripts order:

<script type="text/javascript"> document.documentElement.className = 'js'; </script>
<script type="text/javascript" src=""></script>
<script type="text/javascript" src="" id="jquery-migrate-js"></script>

I thought maybe they should have been included in Synchronous scripts?

I'm consistently getting this warning on https://web.dev/ when the "prefer static head" option is enabled.

On one hand, this page is sending HTML without any <head> element:

Maybe this is some kind of extreme minification setting. But because capo can't extract the static head from the markup, it seems ok to fall back.

On the other hand, the browser is forgiving enough to make everything between html and body the head, so maybe capo could be more flexible and try falling back to that.

So one idea would be to check for the static head, fall back to the implicit static head, and if all else fails, fall back to the dynamic head. If we ever get into this middle state like on web.dev, there should be a different warning message about the lack of any discernible head in the static markup. It's not invalid but it's sketchy.

Would be good to understand other notable examples of this warning in the wild. (please comment if you have one)

There's a TODO to support external stylesheets in the isImportStyles matcher. The code is mostly implemented but commented out due to async complexity in an earlier version of the script. Now that we have #29, which also relies on async code, I think it'll be easier to fit all the async plumbing together.

Right now the demo writes capo.js results to the console. It's not totally natural to be using the demo web page and have to open DevTools to see the output so I think it'd be nicer if the results appeared in the page itself in a sort of emulated console UI.

One of the biggest challenges will be rendering HTML elements. Things like expand/collapse and syntax highlighting are free in DevTools but non-trivial to reimplement.

Maybe a tradeoff is to only print the color bars with tooltips, similar to the extension popup.

When the extension is slow to fetch the static head, users see a blank color bar:

There isn't any indication that work is being done, so it's unclear whether there's an error or whether to expect results soon.

Design some kind of loading animation to suggest progress being made in the background. Consider also including some kind of loading message.

One design idea is to play on the rectangle shapes of the populated color bar:

The colors can be desaturated so it's obvious that it's not the final result:

And maybe some kind of pulsing animation on random "elements"/rectangles to mesmerize the user while they wait :)

Side note: the desaturated demo image uses the pink palette in place of the default theme because it uses a linear luminance scale, which preserves the visual "sort order" of the results. Only important if the bottom row is actually sorted, otherwise we can randomize both rows.

The dynamic mode shouldn't need a loading animation because it can ~immediately process the document.head without any network latency. The snippet is also affected by the static loading delay, but we're limited in how much we can do in the console.

Would you consider adding some documentation on the best practices concerning the order of the base element? I feel it's good to put it somewhat early because it may apply to subsequent elements but the script makes me feel like I'm doing something wrong (I might be, but would like to know why).

I imagine this extension is fairly easily ported to Firefox. Would this be something you'ld consider?

Print stylesheets have the same weight as all other synchronous styles (5).

Given that they're not necessary for rendering UI to screens, and browsers still attempt to load them anyway (at a lower priority), consider lowering their weight to 0 accordingly.

Ensure that the viewport meta:

• permits zooming (a11y)
• avoids double tap to zoom delay (perf)

For example: <meta name="viewport" content="width=device-width, initial-scale=1">

The default behavior of capo.js is to output logs to the console. The extension does that plus visualizing the color bar in HTML.

Create a new client type that outputs both the color bar and logs in HTML. We can embed that in the documentation site to enable users to interactively paste static HTML snippets and see how it would be evaluated. With some HTML presets, we can also use it to demo some of the script's capabilities.

We can add a footer to add info/documentation and link to this repo

Code

<footer class="capo-footer">
  <ul>
    <li>Click in the rectangles to see the element in the console</li>
    <li><a href="https://github.com/rviscomi/capo.js" target="_blank">GitHub</a></li>
  </ul>
</footer>

:root {
  --text-colot: #353535;
}

body {
  color: var(--text-colot);
  padding: 0;
  margin: 0;
}

.capo-footer {
  padding: 1ch;

  & ul {
    display: flex;
    justify-content: space-between;
    list-style: none;
    margin: 0;
    padding: 0;
    width: 100%;
  }

  & a {
    color: var(--text-colot);
  }
}

Screenshot