googlechromelabs / autowebperf Goto Github PK

Hello everyone! First of all, thank you for creating this framework, you have done amazing work!

I was trying to use the framework with different gatherers and it looks great for me so I started thinking about deploying it on the production to make it running regularly.

Is this framework already exist on the NPM? It would be so great to be able to install it globally where I needed without cloning the repository itself.

I am happy to help and create PR if it makes sense.
@jonchenn

Run completed for 1 tests with errors: [ [psi] Unable to assign json.lighthouseResult.audits["service-worker"].score to metrics: Cannot read properties of undefined (reading 'score') ] TypeError [ERR_INVALID_ARG_TYPE]: The "code" argument must be of type number. Received an instance of Array at process.set [as exitCode] (node:internal/bootstrap/node:123:9) at exit (node:internal/process/per_thread:187:24) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) { code: 'ERR_INVALID_ARG_TYPE' }

My last measurement using the sheets connector was on 2024-04-01

TypeError: doc.useServiceAccountAuth is not a function
    at SheetsConnector.getSheet (AutoWebPerf/src/connectors/sheets-connector.js:47:15)
    at SheetsConnector.getTestsSheet (AutoWebPerf/src/connectors/sheets-connector.js:56:32)
    at SheetsConnector.getTestList (AutoWebPerf/src/connectors/sheets-connector.js:236:53)
    at AutoWebPerf.run (AutoWebPerf/src/awp-core.js:315:38)
    at Object.begin (AutoWebPerf/src/cli.js:193:17)
    at Object.<anonymous> (AutoWebPerf/awp:8:7)
    at Module._compile (node:internal/modules/cjs/loader:1376:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1435:10)
    at Module.load (node:internal/modules/cjs/loader:1207:32)
    at Module._load (node:internal/modules/cjs/loader:1023:12)

Apparently, it's now required to instantiate the auth separately.
https://stackoverflow.com/a/76981493

Add a gatherer that can run local lighthouse with CLI and parse metrics.

Hi,
When I run script I get some results but I have error
[psi] Unable to assign json.lighthouseResult.audits["works-offline"].score to metrics: Cannot read property 'score' of undefined
Has anyone have similar problem ?

When the locale set to "de-de" in the PSI gatherer, the following fields return #NUM! instead of values:

psi.metrics.lighthouse.Images
psi.metrics.lighthouse.Fonts
psi.metrics.lighthouse.Javascript
psi.metrics.lighthouse.ThirdParty
psi.metrics.lighthouse.HTML

While running the sample test, I've gotten error below:

[ec2-user@ AutoWebPerf]$ ./awp run url:https://www.thinkwithgoogle.com/ json:output/results.json
Run with 1 test(s)
Run completed for 1 tests with errors:
[
[psi] Unable to assign json.lighthouseResult.audits["uses-webp-images"].details.overallSavingsBytes to metrics: Cannot read property 'details' of undefined
]

Anything can be done?

Add a CSVConnector to be able to read/write in CSV format.

I like the idea of this project, but am confused about how to start using it with my private instance of webpagetest.

The docs/webpagetest.md file notes:

## Required Environmental Variables

- `GCP_PROJECT_ID` - The project unique ID in the Google Cloud Platform. 
- `SERVICE_ACCOUNT_CREDENTIALS` - The path to the Service Account json file.

... where I would've expected to see maybe optional env vars for WEBPAGETEST_SERVER or WPT_API_KEY or similar.
Is there really a dependency on GCP and a "Service Account json file"?

Hi, love the tool.

I have a list of somewhere around ~5k URLs to fetch from the CrUX API per day. When I run AWP on full blast, I blow through the 150 req/minute quota for the CrUX API and get Quota exceeded errors for the majority of requests. Is there any way to throttle the request rate down?

• Support PSI first as connector to use.
• Input should be a JSON via CLI mode.
• Output should be a Spreadsheet ID and Tab to send data via a defined API Key.

It would super awesome to enable generating reports using the current Lighthouse version as well as previewing upcoming scores from the Lighthouse master branch so you can get an idea of what your scores will be in the next release and take action accordingly.

When there's no enough traffic in CrUX for a specific URL, PSI falls back to origin-level CrUX data. As for now, the PSI gatherer couldn't tell the difference.

When excute command ./awp --help
./awp run --config=examples/awp-config.json command shown in the below.

I think this is not workiing because of not giving mandatory arguments.

This example should be revised.

It would be great to be able to run AWP from the CLI by just passing a single URL via the CLI like the lighthouse-ci, rather than a JSON file.

Ex.

./awp run url

The test would be run with default values (mimicking the examples/tests.json file parameters)

https://developers.google.com/web/tools/chrome-user-experience-report/api/reference/rest/v1/records/queryRecord

I have try this command
WPT_APIKEY=SAMPLE_KEY ./awp run examples/tests-wpt.json output/results.json
And i got this message webpagetest] invalid Location, pls try submitting your test again
So what needs to be done with this?

As title, looking into this now.

Hi,

Is there a way we can leverage this tool to get performance metrics for web components present in an android app?
Our app primarily consists of PWA pages and some web views.

Any suggestions on the same would be greatly helpful! @jonchenn

Thanks,
Chatur