ref #26

In https://travis-ci.org/GoogleChrome/web.dev/builds/604263335#L634 i was hoping to see the new github status i just added the token for
so i hit "restart" but it errored because the hash was already uploaded. :(

but maybe instead of erroring we just overwrite the previous data with the newest one?
last one wins seems slightly more reasonable than first one wins.

(I don't know how codecov, etc handle this case. Probably something well considered.)

i'm sure in the future we'll have a more complicated solution, but wdyt about this for now?

I swapped URLs I was testing, and once the new URL was included it started to appear as 0 for all previous runs, even though there are no LHRs for that URL.

Notice red line exists prior to 100, even though there is no data:

Should be no red line prior to the first LHR. Also the blue drops to 0 when there are no more LHRs, should just disappear.

lol rounding is hard.

https://lhci-canary.herokuapp.com/app/projects/d819a1f6-2630-4ee8-a451-95e369fec825/builds/a204633f-ab48-4c26-930a-813c4e05c8d0

i suppose this is related to #28

looks like sometimes these deltas are in seconds.. othertimes in milliseconds

Currently I feel like it should report in milliseconds... always?
-0.1s just feels pretty obtuse. and there's already the % delta threshold filter.

but i don't know for sure.. just an initial impression.

Minimal repro:

{
  "ci": {
    "assert": {
      "preset": "lighthouse:recommended",
      "assertions": {
        "critical-request-chains": ["warn", {"minScore": 1}]
      }
    }
}

This leads to:

  ⚠️  critical-request-chains warning for minScore assertion
     Minimize Critical Requests Depth
     Documentation: https://web.dev/critical-request-chains

        expected: >=1
           found: 0
      all values: 0, 0, 0

Hi,

I've setup a base PR with Lighthouse CI on GitHub Action. Result.

But I'm laking some features:

1. The diff against master branch.
   I though it was built-in and I'll be able to compare by current PR to the master branch for eg. (from this repo screenshots)
   My bet is that I can have this diff only with the Lighthouse Server behind. Do you think we could have a free-budget style for Lighthouse Server on OSS projects?
2. Having a budget setup. I've setup the lighthouserc.json as follow:

{
    "ci": {
        "budgetsFile": "./lighthouse/budget.json",
        "collect": {
            "staticDistDir": "./build"
        },
        "upload": {
            "target": "temporary-public-storage"
        }
    }
}

but so far nothing change.

I'm having difficulties following the documentation overall. There a lot of option but it lack of full example like the cli doc but for the lighthouserc.json.

Currently every build needs a .hash but we don't necessarily need one. We could switch to a .projectVersion and support timestamps instead?

• dashboard view, no paper for top
• add date to build hash selector
• calibrated/overview of metrics
• new pill design for item deltas

• fontcolor of gauge text and ring should be 444
• pill height should be 26
• pill avatar 20x20
• roboto mono 15px not 16px
• body font size 16px with anti-aliasing
• larger arrow for material transition, 20px
• audit group line height 36px
• marginbottom / 2 on audit group header
• numericDiff bar 24px height
• numericDiff indicator height match height of bar
• numericDiff edge label width 38px
• compare color in table details should use darker text
• table headers in details view
• confetti on project dashboard
• 48px loading spinner
• project dashboard fixed width hash
• project dashboard fixed height of commits, vertical scroll, load 100
• remove branch label from dashboard dropdown,
• fixed width branch dropdown on dashboard
• wrap many urls legend in project dashboard
• transparent background when hash selector is open
• just icon for open report on small viewports (circle white background, no rectangle
• add different spacing for Paper component (16px)
• unused css/js should be using code icon
• sidebar 260px
• remove underline on hash links "view build" "github"
• replace dashboard link with info circele
• replace hamburger with back arrow on diff view
• only indent URL label on hover of report
• hide x axis of graphs on project dashbaord
• move y axis labels to right

On mac the chrome/system tooltip (that shows [title] takes like 1 second to display. (iirc, it's twice as slow as windows/linux for no good reason)

regardless, i think we want a JS tooltip component so that the values behind [title] are a lot more accessible. we hide some good stuff in there.

i found these without looking very hard

• https://react-hint.js.org/ doesn't feel right to me, personally.
• https://ajainvivek.github.io/preact-fluid/#tooltip would be nice without the animation
• https://react-tooltip.netlify.com/ i dig this style. i love the mouse tracking thing (tho i'm not 100% that we want it here.. but maybe?)
• https://codepen.io/fuggfuggfugg/pen/mqdBH what can you say. the man's got taste

The no differences page CSS is sad.

The CLI apparently runs Chrome headless (and headfull optionally), but could it run with Puppeteer?

$ lhci wizard
? Which wizard do you want to run? new-project
? Which server would you like to use? http://localhost:9001/
? What would you like to name the project? v8.dev
? Where is the project's code hosted? https://github.com/v8/v8.dev
FetchError: request to http://localhost:9001/v1/projects failed, reason: connect ECONNREFUSED 127.0.0.1:9001
    at ClientRequest.<anonymous> (~/.nvm/versions/node/v12.6.0/lib/node_modules/@lhci/cli/node_modules/node-fetch/index.js:133:11)
    at ClientRequest.emit (events.js:203:13)
    at Socket.socketErrorListener (_http_client.js:402:9)
    at Socket.emit (events.js:203:13)
    at emitErrorNT (internal/streams/destroy.js:91:8)
    at emitErrorAndCloseNT (internal/streams/destroy.js:59:3)
    at processTicksAndRejections (internal/process/task_queues.js:77:11)

Why does it attempt to fetch /v1/projects? My web app does not do anything special for this endpoint, so the wizard exits at this point. For http://localhost:9001/v1/projects I get the above output.

If I enter https://v8.dev/ instead, it fetches https://v8.dev/v1/projects which 404s, resulting in:

$ lhci wizard
? Which wizard do you want to run? new-project
? Which server would you like to use? https://v8.dev/
? What would you like to name the project? v8.dev
? Where is the project's code hosted? https://github.com/v8/v8.dev
Error: Unexpected status code 404
    at ApiClient._convertFetchResponseToReturnValue (~/.nvm/versions/node/v12.6.0/lib/node_modules/@lhci/cli/node_modules/@lhci/utils/src/api-client.js:54:21)
    at processTicksAndRejections (internal/process/task_queues.js:85:5)
    at async runNewProjectWizard (~/.nvm/versions/node/v12.6.0/lib/node_modules/@lhci/cli/src/wizard/wizard.js:48:19)
    at async Object.runCommand (~/.nvm/versions/node/v12.6.0/lib/node_modules/@lhci/cli/src/wizard/wizard.js:75:7)
    at async run (~/.nvm/versions/node/v12.6.0/lib/node_modules/@lhci/cli/src/cli.js:82:7)

Either way the wizard exits at this point. Am I misunderstanding what I should enter?

I fear this may be too strict for real production sites. The PWA stuff being included makes it pretty opinionated too.

(I'd like paulirish.com to pass our recommended preset but a service worker doesnt make sense there. ) Right now I think there's only but a few showcase PWA sites that will actually pass this preset. :/

The strictness here matters a lot more since assertion with recommended is on by default in autorun. I think it's rough that most folks that try autorun for the first time will have failures to deal with before they can see a report or a diff.

In summary: I think any default for autorun should be MUCH more forgiving (if we're recommending new users start with autorun). no PWA checks, allowing color contrast failures etc. And probably this will have to be called loose or forgiving, etc instead of recommended. :)

also only slightly related but what's happening here?

seems like its not using any assertions if they're being explicitly provided? i probably am not reading this right :)

If I'm doing TPS (temp public storage) I kinda want to see that report more than I want assertions.

I feel like upload should precede assert in autorun, but I see the docs specifically say to upload after assert..... why? :)

In a few places in the Getting Started docs, there are some checks for Travis Node version, but the text echoed doesn't make any sense. I guess it's a copy/paste error?

Also condititions => conditions

# example if only running lighthouse on node 10
if [[ "$TRAVIS_NODE_VERSION" != "10" ]]; then
  echo "Only run Lighthouse CI once per build, condititions did not match.";
  exit 0;
fi

Found at https://github.com/GoogleChrome/lighthouse-ci/blob/master/docs/getting-started.md#complete-setup
and https://github.com/GoogleChrome/lighthouse-ci/blob/master/docs/getting-started.md#create-a-run-lhci-script

Having the ability to specify / pass cookies would be really helpful for testing pages that are behind auth or interact with authed endpoints.

Could we do a once-over all on the webserver URLs?
And also review the rest API?

I remember thinking that the compare view should have 'compare' in it, etc. But I figure we should do this sooner than later since it's very breaking changey.

@patrickhulce what's the best way to review all these?

existing work

typescript doesn't have named a CLI option for setting the config file, but their config is named tsconfig.json

eslint uses --config, although it does have --no-eslintrc. allowed config names are many:

babel doesn't have any way to set a config from the CLI, but it does have --no-babelrc. allowed config names are:

thoughts

sorted by order of how strongly i feel

--rc-file should be --config

It seems to just be a config, so shouldn't it be called that? I didn't find any tools that name their config option like this.

remove ability to customize config path

I think babel does this to simplify compiling across project boundaries. That doesn't apply much here, but I wonder if there is any purpose to have multiple config files for LHCI? We could consider removing any option to configure, and add it in later if demand is there. If we do this, I'd recommend the file be called .lhciconfig.json (or .lhcirc.json)

allow .js

right now just .json is allowed. this makes sense for now, and we should definitely defer this. just calling it out now

configure via package.json

ditto above section

nit on rc

rc stands for run commands*, and so I'd think files called rc should be something you must run (shell, node, etc). I wouldn't consider a json file to be a executable file. I see that eslint uses eslintrc.js and eslintrc.json (ditto for babel), so that doesn't jive with my understanding, and it's a popular enough tool that I concede any argument for using .lhcirc.js and .lhciconfig.json.

* at least, it did initially. it has morphed into run configuration over time, which of course kills my argument. but I did some light research on this and can't find a satisfactory source on this definition. from what I currently know, I consider this morph to be a mistake.

We need a way of dealing with changes to table schemas between version of LHCI, no promises made until initial published version but migration should be smooth after that

Ideas:

• Use https://github.com/sequelize/umzug for table schema changes
• Run migrations on every startup
• Store a version tag for each precomputed stat record
• On each query for computed stats, check for current version tag and compute on the fly

Questions:

• How do we handle future alternate storage mechanisms like bigquery?
• How do we handle potentially expensive API queries the first time after an update? Should we precompute the stat records?

External build ID was a nice idea, but it has issues when a project moves CI providers/external build locations. It'd be better to save the entire build URL for easy reference and avoid complex reconstruction logic

Hi. I tried the lighthouse-ci on Releases inside Azure DevOps, but returns a strange error:

2019-06-28T21:53:23.0440309Z ##[error]- Running Lighthouse on https://URL ...
2019-06-28T21:53:23.0450647Z 
2019-06-28T21:53:35.3907204Z performance: 31
2019-06-28T21:53:35.3910085Z accessibility: 67
2019-06-28T21:53:35.3913014Z best-practices: 86
2019-06-28T21:53:35.3913111Z seo: 92
2019-06-28T21:53:35.3913485Z pwa: 30
2019-06-28T21:53:35.3913834Z All checks are passing. 🎉
2019-06-28T21:53:35.3914397Z 
2019-06-28T21:53:35.4122090Z ##[error]Command failed with errors on remote machine.
2019-06-28T21:53:35.4237341Z ##[section]Finishing: Task

I'm using the Task Command Line Script of Azure DevOps to connect on server and execute the command: lighthouse-ci https://URL --score=20 --performance=40 --seo=90 --accessibility=60 --best-practices=80 (the values are smaller to test the funcionality)

I tried to run the command: lighthouse-ci https://URL, without parameters, but not work.

But when I execute the command inside the server, directly, works fine.

Regards.

thanks to node 10 and node 12...
https://travis-ci.org/paulirish/mojibar-web/builds/601044688

we got two LHRs from the same hash
https://lhci-canary.herokuapp.com/app/projects/d819a1f6-2630-4ee8-a451-95e369fec825/builds/59f0de64-68ab-41b4-ae0c-430bb3028f2c

kinda confusing

Lots of options out there, lots of functionality we need.

At a glance we want...

• Compatible license
• Actively maintained
• Support for line graphs with error bars/confidence ranges/candlestick chart
• Play well with preact/react (wrapper already available)

Options

https://lhci-canary.herokuapp.com/app/projects/d819a1f6-2630-4ee8-a451-95e369fec825/builds/59f0de64-68ab-41b4-ae0c-430bb3028f2c

Hello, first of all congratulations for the amazing work, this looks promising.

I would like to know the next if you considering the possibility of create a github action for this?

Does it make sense?

Thank you

was developing on a git branch branch/new-branch. Yes just like that :)

Uploaded some data to canary server here https://github.com/exterkamp/lighthouse-ci-action/commit/21869d5a3234f0d69d8a30027f4455c8057669e5/checks?check_suite_id=299822077

but you can see it was identified as branch HEAD
https://lhci-canary.herokuapp.com/app/projects/3e1ad4e7-689b-4340-83f4-409c84308f89

i notice something weird about this checkout and git branch is odd.. and rev-parse doesnt seem to like it.

i guess the currentBranch command just isn't getting it due to the clone/checkout style?? dunno.

but i tried the git command that is used in my bash/fish prompt and it succeeds

git symbolic-ref --quiet --short HEAD 2> /dev/null || \
git describe --all --exact-match HEAD 2> /dev/null || \
git rev-parse --short HEAD 2> /dev/null || \
echo '(unknown)')

in my bash prompt it uses that terrible thing and then seds out 's|^refs/heads/||' but i guess it could do the same with remotes/origin/? :)

anyway GIT IS GREAT! amirite

cc @exterkamp

npm install -g @lhci/[email protected] currently fails, because no non-preview 0.3.x release has been npm published yet:

$ npm install -g @lhci/[email protected]
npm ERR! code ETARGET
npm ERR! notarget No matching version found for @lhci/[email protected]
npm ERR! notarget In most cases you or one of your dependencies are requesting
npm ERR! notarget a package version that doesn't exist.

npm ERR! A complete log of this run can be found in:

Note that this exact command occurs in the README and in the docs. Let’s release v0.3.0!

In order to quickly render the dashboard UI, the API will need to store and readily provide computed summary statistics on the LHRs stored in the runs table.

These statistics will likely depend on the specific configuration and subject to frequent change based on UI needs, so current thinking is to follow a simple schema and compute these values on the fly at query time (which are stored/cached for future loads). Recomputation could follow the process in #4

Example Rows

Currently master is hardcoded in a number of places, making this fully configurable requires changes in a couple places

• upload configurable ancestor branch
• project model property for configurable base branch
• loading project model in multiple components to use the configured branch

while setting up lhci on https://github.com/paulirish/mojibar-web i managed to run into the "you dont have access to secret env vars on PRs from clones" bug like 3 times. partly because i'm dumb, admittedly.

so by default, the LHCI_TOKEN and LHCI_GITHUB_TOKEN would be expected to be there. but since they're not present on contributor PRs, then both uploading and status checks are disabled. and that sucks quite a bit.

i wonder if we have other options?

Is it possible to run lhci collect along with a plugin?

Hmmmm is it possible via something like

lighthouse-ci collect --numberOfRuns=5 --url=https://example.com --settings="--plugins=foo-plugin"

err nope, thats not it.
anyway i guess this is a matter of documentation. :p

Burndown list created for items outstanding as of 2019-10-01 or later.

🎯 ✅ Gotta happen

UI

• PWA Gauge (c881617)
• numeric audit tweaks for overflow text (6eaf47d)
• ellipsis overflow text on audit name (6eaf47d)
• use SVG for the border triangle icon (48a30ef)
• multiple URL graphs on project dashboard (e24d46f)
• support node details type in table (f655599)
• markdown description support (2e9a282)
• pass/fail/icon indicator in audit detail view (2e9a282)
• hide gauges when detail is expanded (ccf61b6)
• simplified deltas when detail is expanded (ccf61b6)
• numeric diff in detail view (b142e82)
• move numeric diff delta label to right side when it goes to 0 (2d4f0db)
• only metrics get numeric diff in list view (98b0ca2)
• use closest previous build time when exact ancestor is not known (2ceebda)
• double render preact bug fix (81569a6)
• close remaining details table gaps
  • main-thread work items (68dfdac)
  • enormous network payload items (68dfdac)
  • fall back to display value when no diff available (0e0df55)
  • link details type (844110a)
• bug fix for different URLs in UI (f7e797e)
• diff logic improvements
  • key by things other than URL (i.e. for budget categories) (5329b62)
  • handle hash components of a URL (dee2613)
  • sort within item groups (#31) (2a8d6ad)
• limit builds in hash selector (4f5476a)
• limit statistics fetched for dashboard (4f5476a)
• fix same branch compare/base (91a4c7a)
• label only "added"/"removed" and move to before first numeric cell (5bf7b48)
• add percentages everywhere (2f0fb8e)
• smaller font size of upper/lower limits (ead96e7)
• use ms everywhere for deltas (7c1e5ff)
• branch picker on project dashboard (ec95bc8)
• move avatar inside pill (5e3e71e)
• extend hover color to entire build picker row (4bcaa4e)
• revert back to pill background instead of header (4bcaa4e)
• tweak base/compare pill colors in hash picker (4bcaa4e)
• run through all spec colors/sizes in figma
• consolidation of error states on "no diff" view (be6a263)
• legit loading spinner (f22bb8c)
• faded out base/compare chip hash selector (4bcaa4e)
• optional merge base (#36)
• no diff available fixes for opportunities (c4a2c40) (eaffaf2) (f87124a)
• handle image thumbnail failure to load (14666ac)
• favicon (2dc0f90)
• project pane on scroll isn't full viewport width (0f9ff47)
• fix base+compare on same branch (1b4619f)
• friendly URLs (696a95e)
• synthesize group for best practices audits (4fcf7af)

CLI

• update notifier (7bcd7f2)
• rename to staticDistDir (6244102)
• assert on median build, not just median value (70aef52)
• upload URL replacements for <port>, <uuid>, etc (ccb528b)
• generate lighthouserc from budgets.json (00cf1a4)
• lighthouserc granular budget assertions (3d29976)
• assert allow assertion on multiple urls/patterns as keys in matrix
• update the lighthouse:recommended preset (e78b25a)
• rename mergeMethod to aggregationMethod (c0329f1)
• support file-wide default aggregationMethod (6c098cd)
• collect mode with a server for local files (ebbe030)
• automegadoeverything command (5e42e21)
• print audit titles and learn more link (261fd0a)
• healthcheck that fails/warns when things aren't configured correctly (#38)
• list item details on failure

API

• create friendly slug for project (40de6e7) (26024ff)
• support partial hash search (e43d0e7)
• create migration files for schema changes (d27e181)
• route for closest build to target ancestor (e19d9cd)
• use a real token distinct from project ID (4e814d7)
• committedAt (#35)
• add statistic version to statistic records (#4) (c2dc676) (0f4dd6b)

Tests

• multiple URL comparison test in build view (914dbd7)
• setup LHCI on itself for homepage, project dashboard, build view (13c9bfd)
• setup assert flow in dogfood (0cc0178)

Perf

• spin up a test server of 50 LHRs/build, 1000 builds (n1-standard-1 handled 3700 LHRs with 75 builds easily, only ~2GB sqlite file, ~20ms response times for common queries, 300ms response times for lifecycle/statistic computation)

Docs

• document versioning strategy for breaking changes
• document healthcheck and common mistakes (git depth, external contributions preventing githubToken, creating empty commits to rerun, etc)
• document assertion format with basic examples (9e10393)
• getting started guide (6babedc)
• update docs for optionality of fallback server (5943427)
• docs on automegadoeverything command (5943427)

Design Feedback

• ordering of audits (bc476b6)

Misc

• publish docker image with 0.3.0
• update all examples with install of 0.3.x

🌆 ☑️ Nice to have

UI

• warning when using a different build from exact ancestor (47f968f)
• LH report link next to legend (1188120)
• minimum diff threshold slider on build view (1188120)
• clicking off build hash picker closes it (0652474)
• redesign of "no diff" view (47f968f)
• new pill design for item deltas
• recent activity pagination
• git tree visualization on build-view hash picker
• nicer tooltips (#30)
• --start-server-command (#37) (#46)
• audit-specific icons (c7934e6)
• add color to the gauges when there is no diff at all
• calibrated/overview metrics
• dates on hash picker commits
• nicer design of empty states (4d268a7) (04126b8) (3913e43)
• closeable warning boxes
• diff logic improvements
  • extract numbers from displayValue and use those for ignorable thresholds
  • ignore pass -> pass variance (68a906d)

API

• pagination on builds
• GH app and proxy service
  • create the actual GH app/logo/copy/etc
  • route to create a GH token for you
  • route to validate the GH token against the repo and post the status to GH

CLI

• upload target of transient cloud storage service (cbad14d)
• upload sets a GH status check bit (97d2639)
• automatically find a local .lighthouserc (bd7d44c)
• assert against category scores (13c687e)
• open command view HTML reports (df13eb4)

Docs

• document travis CI setup for basic assert flow
• document installation of CI server
• document different ways to use budgets.json with lighthouse ci

🏈 🏖 Punt till after CDS

• e2e puppeteer UI tests for all mocked design areas
• serve thumbnails of images from bespoke thumbnails entry in LHR
• configurable base branch #82
• allow last build to be overriden/deleted by a new build
• embed thumbnails of images that are on localhost into LHR
• Errored/missing audits #83
• List of no diff/hidden audits
• Project management #86
• Authentication/user management #85
• bespoke non-platform dropdowns

CDS Feedback

• Docs and CircleCI tests #90
• Windows tests #67
• --puppeteer-script #89
• Docs on auth testing
• Docs on jenkins setup #91
• GitHub enterprise URL configuration #92

We don't want to compute statistics until the build has all the runs uploaded, we have two main options.

1. Eliminate the individual saving of runs and force a build to be saved in a single API call. i.e. POST /v1/projects/<id>/builds accepts something more like

{
  "build": {"hash": "afe15fcb..."},
  "runs": [{"lhr": "<first lhr>"}, {"lhr": "<second lhr>"}]
}

This is more straightforward on our side and eliminates the possibility of a hanging, unfinished build. However, these payloads will get insanely large, like on the order of ~30MB for pages with many runs/pages. Additionally this eliminates the possibility of supporting parallel travis builds that would be uploading runs from different environments

2. Add a PUT /v1/projects/<id>/builds/<id>/sealed route or something that must be called after all runs have been uploaded to signal that there are no more coming and the stats can be finalized. This feels like the proper solution to support all the sensible use cases and prevent massive, unsustainable payloads. Just a slightly bigger pain to implement up front.

Firstly: this is a great project and I'm really excited to use it. Thanks for all the hard work!

In https://github.com/GoogleChrome/lighthouse-ci/blob/master/docs/cli.md#overview it says "Install the CLI globally to try it out locally", but on Windows the run ends with an spawn UNKNOWN error:

✅  .lighthouseci/ directory writable
⚠️   Configuration file found
Healthcheck passed!

Automatically determined ./dist as `staticDistDir`.
Set it explicitly in lighthouserc.json if incorrect.

Started a web server on port 6146...
Running Lighthouse 3 time(s) on http://localhost:6146/404.html
Run #1...failed!
Error: spawn UNKNOWN
    at ChildProcess.spawn (internal/child_process.js:394:11)
    at Object.spawn (child_process.js:540:9)
    at LighthouseRunner.run (C:\Users\Rhian\AppData\Roaming\npm\node_modules\@lhci\cli\src\collect\lighthouse-runner.js:83:34)

This looks to be because childProcess.spawn() in lighthouse-runner.js is called without a third argument { shell: true }, which is required on Windows.

Since this is a CI tool and was probably never intended to be run on Windows, perhaps the only change needed is to mention this in the CLI documentation that I linked above. It would be useful to be able to do dry runs locally, though.

I believe this means upstreaming the work done here:

• https://github.com/GoogleChromeLabs/gulliver/tree/master/lighthouse_machine
• https://github.com/ebidel/lighthouse-ci/tree/master/builder

The lighthouse service within should be easily accessible over HTTP.
I kind of preferred the basic REST api within lighthouse_machine, but perhaps we should chat more about the API.

@ebidel wanna take a first whack at this?

300 vs 0.8. Implied ms and s. Confusing since there is no unit.

Hey,

After upgrading @lhci/cli to 3.1.0 (from 0.3.0-alpha.0) I can no longer use a custom lighthouserc config.

Here is my lighthouserc.json:

{
  "ci": {
    "assert": {
      "preset": "lighthouse:recommended",
      "assertions": {
        "categories.pwa": "off"
      }
    }
  }
}

and logs:

Run lhci autorun --config=./lighthouserc.json
✅  .lighthouseci/ directory writable
✅  Configuration file found
Healthcheck passed!

Automatically determined ./public as `staticDistDir`.
Set it explicitly in lighthouserc.json if incorrect.

Started a web server on port 38257...
Running Lighthouse 3 time(s) on http://localhost:38257/index.html
Run #1...done.
Run #2...done.
Run #3...done.
Done running Lighthouse!

TypeError: normalizeAssertion is not a function or its return value is not iterable
    at getAllFilteredAssertionResults (/opt/hostedtoolcache/node/10.17.0/x64/lib/node_modules/@lhci/cli/node_modules/@lhci/utils/src/assertions.js:390:39)
    at getAllAssertionResults (/opt/hostedtoolcache/node/10.17.0/x64/lib/node_modules/@lhci/cli/node_modules/@lhci/utils/src/assertions.js:440:23)
    at Object.runCommand (/opt/hostedtoolcache/node/10.17.0/x64/lib/node_modules/@lhci/cli/src/assert/assert.js:54:22)
    at run (/opt/hostedtoolcache/node/10.17.0/x64/lib/node_modules/@lhci/cli/src/cli.js:87:23)
    at Object.<anonymous> (/opt/hostedtoolcache/node/10.17.0/x64/lib/node_modules/@lhci/cli/src/cli.js:118:1)
    at Module._compile (internal/modules/cjs/loader.js:778:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)
    at Module.load (internal/modules/cjs/loader.js:653:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:593:12)

    at Function.Module._load (internal/modules/cjs/loader.js:585:3)assert command failed. Exiting with status code 1.
##[error]Process completed with exit code 1.

I'm not sure if I missed something during the update or is it a bug?

carrying over from https://github.com/GoogleChrome/web.dev/pull/1757/files#r339844263 .....

zeit used to allow you to run your own webserver (they're now just static site and serverless functions).. but at the time they'd look for a now-start npm script first.. and if not they'd use a start npm script.

that's one option here too.. we look for start:lhci followed by start. if there's neither we log an error.

One downside here in doing all this automagically is that we don't really know exactly how long to wait after starting the server before we can begin collecting. i guess some sites have complicated servers that don't kick off instantly?

Is there any way to find the LHCI_GITHUB_APP_TOKEN once you close the authorization confirmation page? 😄

I can't find it in the documentation.
This would be a good question to add to the FAQ.

Same as the cyan bar being too low and overlapping, the red overlaps the other way...

If I run Travis without Lighthouse it passes all the tests.
When I add Lighthouse the test fails because my app is not optimized yet.
But when Lighthouse fails Travis also fails.

I only use Travis for checking syntax style and linting, not for running tests.
So, in my case, the ideal behavior would be that when Lighthouse finishes its tests just updates the GitHub status, but not making Travis fail.

Can this be added as a CLI Command or as an option in lighthouserc.json?

Thanks,

btw: I love this tool, you are doing a great job

project-graphs.jsx:71 Uncaught (in promise) TypeError: Cannot read property 'map' of undefined
    at render (project-graphs.jsx:71)
    at y.o [as constructor] (async-loader.jsx:19)
    at y.L [as render] (index.js:337)
    at U (index.js:119)
    at children.js:85
    at C (children.js:194)
    at C (children.js:190)
    at b (children.js:52)
    at U (index.js:131)
    at children.js:85

on https://lhci-canary.herokuapp.com/app/projects/d1e4b15c-e644-4552-b136-e975f486a2ce

dunno which of these we'd consider post-launch but just jotting them all down for now

• "our server recipe." link in docs 404s
• I think a couple screenshots in the initial readme would be nice :)

from GoogleChrome/web.dev#1757

https://travis-ci.org/GoogleChrome/web.dev/builds/604191888#L634

1. should it only run on the PR build?
2. since this is fatal i wonder if it should check this before starting collect? :/

also I think the /build/ command is also posted by the github status

But we (thanks to me) changed the url to /compare/ and so it 404s and the router falls back to the homepage.

Handling the redirect would be cool, but mostly we should just fix the original links.

example: https://lhci-canary.herokuapp.com/app/projects/d1e4b15c-e644-4552-b136-e975f486a2ce/builds/0a732a7a-1702-4aec-afff-8a4b7a735876

The link in the readme for 'Lighthouse' directs to a 404 page.

https://lhci-canary.herokuapp.com/app/projects/d819a1f6-2630-4ee8-a451-95e369fec825/builds/80872396-61cc-46b0-ae98-ddeb705ea990?baseHash=006657567ad7fdd6d71f8041c3f9b9baeca50225

how can it have no change if there is a change? :)

also.. are we currently sorting within the groups? seems like we're not sorted within 'better' here, right?

Hey,

I use Github Actions as CI. I set up a Github Token through secrets, everything works well except setting status, it fails with Invalid repo slug "undefined", skipping.

lgci autorun logs:

✅  .lighthouseci/ directory writable
✅  Ancestor hash determinable
Healthcheck passed!

Automatically determined ./public as `staticDistDir`.
Set it explicitly in lighthouserc.json if incorrect.

Started a web server on port 33093...
Running Lighthouse 3 time(s) on http://localhost:33093/index.html
Run #1...done.
Run #2...done.
Run #3...done.
Done running Lighthouse!

Uploading median LHR of http://localhost:33093/index.html...success!
Open the report at https://storage.googleapis.com/lighthouse-infrastructure.appspot.com/reports/1573513709814-41442.report.html
GitHub token found, attempting to set status...
Invalid repo slug "undefined", skipping.
Done running autorun.