formidablelabs / trace-pkg Goto Github PK

• Add JSON output of report. Maybe --report=json? (And default --report to --report=text
• Add docs about doing --dry-run to not output zips and just do JSON report for further programmatic usage.

Found this:

// // Read package.json version from disk
// const getVersion = async (filePath = "") => {
//     if
// };

in https://github.com/FormidableLabs/trace-pkg/blob/main/lib/util/package.js#L11-L14

We should delete it :)

Done:

• concurrency
• package.include: Straight inclusion without tracing.
• trace.include (Array<string>)
• trace.ignores (Array<string>)
• trace.allowMissing (Object.<string, Array<string>>)
• trace.dynamic.resolutions (Object.<string, Array<string>>)
• trace.dynamic.bail (Boolean)

To do:

• collapsed.bail (Captured in #3 )

Maybe:

• package.exclude: (Could collapse or rename with above).

... with all the options.

Since e2e tests are run in a separate proc, we don't need them in coverage, which is slowing things down and getting timeouts.

Tasks:

• Separate yarn test into something like yarn test:spec and yarn test:e2e
• Set e2e timeout to like 5 seconds (maybe a new mocha options or something).
• Only do coverage on test:spec
• Wrap up the test runners for normal test and check-ci|test-ci|test-cov as makes sense.

• concurrency
• ... figure out other things.

Jetpack has more verbose information when reporting conflicts. trace-pkg just does the bare minimum of packages.

Some enhancements to consider:

• Package version of dependency duplicates.
• Individual dependency package summaries

Jetpack example:

Serverless: [serverless-jetpack] WARNING: Found 1 collapsed dependencies in .serverless/my-function.zip! Please fix, with hints at: https://npm.im/serverless-jetpack#packaging-files-outside-cwd
Serverless: [serverless-jetpack] .serverless/FN_NAME.zip collapsed dependencies:
- lodash (Packages: 2, Files: 108 unique, 216 total): [node_modules/[email protected], ../node_modules/[email protected]]`

Our current dynamic.resolutions configuration allows for specifying new JS imports to match dynamic import misses. A slightly different problem is when a library does something like const style = fs.readFileSync("./a-style.css") or something. We won't get dynamic import miss, and we can fix the situation with a normal non-parsed include but that needs the relative to CWD path to the file...

Instead, it would be great to have a more portable option that was relative to a file to include something that isn't parsed. Could look something like:

dynamic:
  resolutions:
    "a-pkg": ["./real-js.js"] # This file _is_ parsed
  resolutionsInclude:
    "a-pkg": ["./a-style.css"] # This file is _not_ parsed

Such a feature doesn't have to be part of dynamic.* at all because it's not part of the dynamic import misses. So maybe a top-level field or something instead?

• Relative or absolute build directory to cwd to serve as base of all output.

• Use https://www.npmjs.com/package/jest-worker

Background

Some of our deps (globby) are ESM-only. We'll eventually need to upgrade.

Packages

• globby
• chalk
• execa

Options

Possible pattern for async dynamic import() and then cache the module: https://github.com/FormidableLabs/trace-pkg/pull/new/experiment/esm-pkgs-support Note that we'll need Node 14+ as mainline for dynamic import.

Unfortunately, this approach isn't compatible with early Node 12, which is in maintenance mode until 2022-04-30. Here are some notes about 12 versions:

• v12.16.3: ESM won't work without experimental flag.
• v12.17.0: Removed experimental flag for ESM loading
• v12.20.0: Remove ESM experimental warning.

In Jetpack, package.include|exclude can be used to later exclude files found during tracing. In trace-pkg .trace is completely separate from .include and thus .include globs can't affect/remove files found during tracing.

Task:

• Consider either having .include do glob matching of some sort on .trace or having a .traceInclude or something that just applies patterns over .trace results
• Update docs

Docs sections to update:

This would have otherwise been ported over from Jetpack for the "handling collapsed files" section:

• Use packages.<PKG_NAME>.include: You can manually adjust packaging by excluding files that would be collapsed and then allowing the other ones to come into play. In the two lodashs example above, a negative package.include for !node_modules/lodash/** would solve our problem in a semver-acceptable way by leaving only root-level lodash.

We'll probably be pretty close on configuration with serverless.yml just for ease, but document differences.

See: FormidableLabs/serverless-jetpack#142

• Basic tests for how symlinks are resolved (probably in a monorepo with a lib/package1|2 and functions/packageA|B|C, etc.

• Package a single (or comma separated list or something) of packages instead of all of them.

• Example failure: https://github.com/FormidableLabs/trace-pkg/runs/2751558479?check_suite_focus=true
• Upstream issue: tschaub/mock-fs#332

Presently .include and .trace options are only on package.<PGK_NAME>. We could add global options. fields and smart concat them in...

• Implement collapsed.bail feature in options and packages.
• Add README section on what collapsing means and how you can avoid it by adjusting cwd to project root and then adjusting handler paths deep into project.
• Add log link to README section.

The source map inclusion feature can include discovered source maps in node_modules. Typically, we would only want our application source maps.

Task:

• Add a filter on source maps to only include (and include misses) for things matching a pattern so you can e.g., just do source map inclusion for application source files and not things in node_modules.
• OR do something like add a glob/pattern matcher for files that is separate and just a pattern matcher so that you can exclude any type of files and then we could add that for like !**/node_modules/**/*.js.map as an implementation of #21