My solution to the NPM package-lock.json problem about isolate-package HOT 11 CLOSED

I'm hoping to revisit this in the coming months but I've been really busy with work. Personally, I've been just excluding the lock files. I consider my project dependencies stable enough, but of course, it could go wrong. It's a tradeoff I'm willing to live with. I'd rather do that than chasing and excluding packages at the moment.

In the meantime, I've also been working on a monorepo setup example that showcases isolate-package, because a number of people have been asking me for help on that, and it's not trivial. I'll probably wrap that up first before I get to this...

from isolate-package.

Posting our approach, as it might be useful. Note that we've only tested in a small test project, and that we bundle the shared dependencies instead of using isolate-package.

We generate a npm lockfile specifically for the target package, by using @npmcli/arborist.
We do so in the monorepo root postinstall script, in order for the target package lockfile to be in sync with the monorepo root lockfile.

Ref: npm/rfcs#554 (comment)

from isolate-package.

Sure, from the target package directory:

// Create a tree of the dependencies for this workspace.
const arborist = new Arborist({ path: __dirname });
const { meta } = await arborist.buildIdealTree({ rm: DEPS_LOCAL });
meta?.commit();

// Write `package-lock.json` file in the `dist/` directory.
await fs.promises.mkdir(path.join(__dirname, "dist"), { recursive: true });
await fs.promises.writeFile(
  path.join(__dirname, "dist", "package-lock.json"),
  String(meta)
);

{ rm: DEPS_LOCAL } removes shared deps from the package-lock.json, since they are bundled.

from isolate-package.

@giacomoran Thanks for sharing! 👍

In the meantime, I'm also happy to report that it looks like I've almost solved the PNPM challenge, generating a lockfile specific for the isolated package based on the existing monorepo root lockfile.

After that I will likely give Arborist another try.

from isolate-package.

Hi @BenJackGill thanks for looking into this! I hope to give the lockfile situation a closer look soon too. It's good to know that individual packages might be causing this.

I think I might also have the functions test package as a dependency while not using it...

from isolate-package.

I came up against this error again.

This time the package causing the error was type-fest and after removing it and running npm install from the monorepo root the error went away.

Here is an example of the full error I was seeing. You can see that it calls out type-fest as the offending package.

Build failed with status: FAILURE and message: npm ERR! code EUSAGE
npm ERR! 
npm ERR! `npm ci` can only install packages when your package.json and package-lock.json or npm-shrinkwrap.json are in sync. Please update your lock file with `npm install` before continuing.
npm ERR! 
npm ERR! Invalid: lock file's [email protected] does not satisfy [email protected]
npm ERR! Missing: [email protected] from lock file
npm ERR! 
npm ERR! Clean install a project
npm ERR! 
npm ERR! Usage:
npm ERR! npm ci
npm ERR! 
npm ERR! Options:
npm ERR! [--install-strategy <hoisted|nested|shallow|linked>] [--legacy-bundling]
npm ERR! [--global-style] [--omit <dev|optional|peer> [--omit <dev|optional|peer> ...]]
npm ERR! [--strict-peer-deps] [--no-package-lock] [--foreground-scripts]
npm ERR! [--ignore-scripts] [--no-audit] [--no-bin-links] [--no-fund] [--dry-run]
npm ERR! [-w|--workspace <workspace-name> [-w|--workspace <workspace-name> ...]]
npm ERR! [-ws|--workspaces] [--include-workspace-root] [--install-links]
npm ERR! 
npm ERR! aliases: clean-install, ic, install-clean, isntall-clean
npm ERR! 
npm ERR! Run "npm help ci" for more info

npm ERR! A complete log of this run can be found in: /www-data-home/.npm/_logs/debug-0.log. For more details see the logs at https://console.cloud.google.com/cloud-build/builda.

While this worked, it is not a great a solution for me because I do need to use type-fest.

I am still looking for a final solution.

from isolate-package.

@giacomoran Interesting. Care to share the commands you use for generating that lockfile specifically for the target package? From my shallow reading of the docs, it's not clear to me yet.

from isolate-package.

@giacomoran I've been spending a lot of time on the lockfile situation recently. I'm mostly focusing on PNPM now, also because NPM was giving me some unhelpful errors in my test monorepo that were driving me nuts, but I would like to give it another try soon...

I was wondering in your case how you are bundling your shared deps. Is it only your own shared code that gets included in the bundle, or are you also bundling code from any external NPM packages that the shared code depends on?

If you are not bundling the dependencies of the shared dependencies, then I think Arborist is doing something that I'm still failing to implement with PNPM, because I managed to generate a valid PNPM lockfile, but the deps of my shared deps are not getting installed unfortunately...

from isolate-package.

I'm bundling the dependencies of the shared dependencies, to keep things simple.
My bad, I overlooked this difference between bundling and isolate-package.

If you want to take a look, I've created a small demo project showcasing our approach: https://github.com/giacomoran/demo-npm-bundle-local-deps

from isolate-package.

Happy to announce that NPM lockfile support has been added, based on Arborist. Thanks for bringing it under my attention @giacomoran

I have also forked firebase tools to integrate isolate-package, so you can run the local emulators with live code updates. For more info see this thread firebase/firebase-tools#653 (comment)

from isolate-package.

That's awesome! Thank you for your work.

from isolate-package.