Is there a way to force compilation during install-time? With node-pre-gyp you can pass --build-from-source as a flag to npm.

The problem in my case is that the Linux prebuilt binary is libc based, and thus doesn't work on Alpine Linux which uses MUSL.

Environment

• Node.js Version: 18, 20
• Platform: Windows x86

Description
The build process fails with an EINVAL error during the spawn operation in node-gyp-build. This issue seems to affect only Windows x86 configurations. A similar problem has been reported in prebuildify: prebuild/prebuildify#83.

Steps to Reproduce

1. Run the node-gyp-build in a Windows x86 environment using Node.js versions 18 or 20.
2. Observe the failure in the build process.

Expected Behavior
The build should complete without errors.

Actual Behavior
The build process fails and outputs the following error:

> node-gyp-build

node:internal/child_process:421
    throw new ErrnoException(err, 'spawn');
    ^

Error: spawn EINVAL
    at ChildProcess.spawn (node:internal/child_process:421:[11](https://github.com/NeuraLegion/node-raw-socket/actions/runs/8686906219/job/23819333405#step:4:12))
    at Object.spawn (node:child_process:761:9)
    at build (D:\a\node-raw-socket\node-raw-socket\node_modules\node-gyp-build\bin.js:30:8)
    at preinstall (D:\a\node-raw-socket\node-raw-socket\node_modules\node-gyp-build\bin.js:39:32)
    at D:\a\node-raw-socket\node-raw-socket\node_modules\node-gyp-build\bin.js:11:7
    at ChildProcess.exithandler (node:child_process:430:5)
    at ChildProcess.emit (node:events:518:28)
    at maybeClose (node:internal/child_process:1105:[16](https://github.com/NeuraLegion/node-raw-socket/actions/runs/8686906219/job/23819333405#step:4:17))
    at ChildProcess._handle.onexit (node:internal/child_process:305:5) {
  errno: -4071,
  code: 'EINVAL',
  syscall: 'spawn'
}

Using node-gyp-build 4.1.0 I'm sometimes seeing node-gyp-build stall forever when the package has already been built.

It seems to work ok with node 10, but it reliably hangs with node 12. I can reliably reproduce this by installing my node foundationdb bindings:

$ nvm use 12
Now using node v12.1.0 (npm v6.9.0)
$ npm i [email protected]

> [email protected] install /Users/josephg/test/fd/node_modules/foundationdb
> node-gyp-build

<stalls forever>

It might have something to do with the fact that the binary is a napi module.

I've started bundling my code with esbuild, but some of my dependencies use node-gyp-build. These two don't seem to play well together at the moment. I can successfully bundle the code with esbuild, but things break at runtime.

At runtime, the code that uses node-gyp-build throws an error that says: Error: No native build was found for platform=linux arch=x64 runtime=node abi=93 uv=1 libc=glibc node=16.16.0. It looks like that message comes from here.

The error stack also references dependencies including sodium-native, sodium-universal and others. But I believe the problem stems from node-gyp-build.

Is this something that can be addressed here? I don't understand the actual issue, and can't find much help online for it. So any clarity on this would be wonderful.

esbuild settings (if that's helpful):

• platform: node
• target: node16
• bundle: true

When using a dependency that uses node-gyp-build prebuilt binaries, next does not seem to pick up those binaries.

The directory it searches for binaries is ./.next/server instead of ./node_modules/zeromq in my case. If I change this and hardcode it to /absolute/path/to/node_modules/zeromq it works because it sees that prebuilds.

As you might have noticed I am using a fork that was made on 4.5.0, because that's what was done for https://github.com/zeromq/zeromq.js

I wrote https://github.com/juliangruber/prebuildify-load as a replacement for this module which isn't build system dependant (my module uses cmake-js). I propose to advice to use prebuildify-load as the main module.

This issue goes hand in hand with prebuild/prebuildify#7.

When using Webpack to bundle a Node application into a single .js file, node-gyp-build cannot unambiguously load the correct .node file.

In a bundled application, Webpack can provide as __dirname either "/" which is the default, the input file's path relative to the build context, or Node's regular behaviour

To support bindings based native modules, you can copy the built .node files into build/Release relative to the bundle file, because bindings is passed the name of the .node file. When node-gyp-build tries to load build files from that folder, if it contains multiple .node files from multiple modules, it will grab the first one in it for every module that uses node-gyp-build.

E.g. if my project uses argon2 and leveldown and builds them from source, I will have the bundled output structure:

foo
|-- index.js
|-- build
    |-- Release
        |-- argon2.node
        |-- leveldown.node

but either library trying to load its bindings file will just end up loading argon2.node.

For similar reasons prebuilds are completely impossible to use, because all flavours of prebuild for a given runtime and ABI will have the same filename (such as node-napi.node), and since there's only one bundle file its containing directory won't disambiguate between them.

Right now, if install script is node-gyp-build and user do not have required tools / libraries on machine npm install will fail with exit code 1. But it's can be ok if we have pure js replacement of native bindings (for example cryptocoinjs/secp256k1-node, cryptocoinjs/keccak).

What if we add flag --ignore-build-fail with node-gyp-build will exit with 0 for any result of node-gyp rebuild?

I am not sure if this is the right place, but I got this error on my M2 MacBook. Any Other machine is working fine. Do you have any idea what the problem is?

I am using bun but npm run dev gives me the same error.

$ bun run dev                                                                                                               [20:51:28]
$ parcel src/index.html --open
Error: No native build was found for platform=darwin arch=x64 runtime=node abi=108 uv=1 libc=glibc node=18.7.0
    loaded from: /Users/frankmayer/Documents/Git/WorldArchitect/WebGLPreview/node_modules/lmdb and package: @lmdb/lmdb-darwin-x64

    at load.path (/Users/frankmayer/Documents/Git/WorldArchitect/WebGLPreview/node_modules/node-gyp-build-optional-packages/index.js:64:9)
    at Object.load [as default] (/Users/frankmayer/Documents/Git/WorldArchitect/WebGLPreview/node_modules/node-gyp-build-optional-packages/index.js:20:30)
    at Object.<anonymous> (/Users/frankmayer/Documents/Git/WorldArchitect/WebGLPreview/node_modules/lmdb/dist/index.cjs:47:47)
    at Module._compile (/Users/frankmayer/Documents/Git/WorldArchitect/WebGLPreview/node_modules/v8-compile-cache/v8-compile-cache.js:192:30)
    at Module._extensions..js (node:internal/modules/cjs/loader:1174:10)
    at Module.load (node:internal/modules/cjs/loader:998:32)
    at Module._load (node:internal/modules/cjs/loader:839:12)
    at Module.require (node:internal/modules/cjs/loader:1022:19)
    at require (/Users/frankmayer/Documents/Git/WorldArchitect/WebGLPreview/node_modules/v8-compile-cache/v8-compile-cache.js:159:20)
    at _lmdb (/Users/frankmayer/Documents/Git/WorldArchitect/WebGLPreview/node_modules/@parcel/cache/lib/LMDBCache.js:61:39)
Script error "dev" exited with 1 status

Hi,

I'm using a few packages that use node-gyp-build, and I'm having a hard time figuring out if I'm using native versions of those or their js fallback. Is there a way I can distinguish between them?

Hi Mathias,

I've followed the instructions in your lovely article on Nearform to use prebuilds with N-API to create a new module for reading files from devices using MTP: https://github.com/tidepool-org/node-mtp

Now, I've set up a Travis CI build for the module successfully that creates prebuilds for both macOS and Linux, and I can see the prebuilds in my NPM. I've used the module in an Electron app, that works fine on my local machine, but doesn't build on a Circle CI machine on macOS:

/Users/distiller/tidepool-org/chrome-uploader/node_modules/node-mtp: Command failed.
Exit code: 1
Command: node-gyp-build
Arguments: 
Directory: /Users/distiller/tidepool-org/chrome-uploader/node_modules/node-mtp
Output:
gyp info it worked if it ends with ok
gyp info using [email protected]
gyp info using [email protected] | darwin | x64
gyp http GET https://nodejs.org/download/release/v10.1.0/node-v10.1.0-headers.tar.gz
gyp http 200 https://nodejs.org/download/release/v10.1.0/node-v10.1.0-headers.tar.gz
gyp http GET https://nodejs.org/download/release/v10.1.0/SHASUMS256.txt
gyp http 200 https://nodejs.org/download/release/v10.1.0/SHASUMS256.txt
gyp info spawn /usr/local/bin/python2
gyp info spawn args [ '/Users/distiller/.nvm/versions/node/v10.1.0/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'make',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/distiller/tidepool-org/chrome-uploader/node_modules/node-mtp/build/config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/distiller/.nvm/versions/node/v10.1.0/lib/node_modules/npm/node_modules/node-gyp/addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/distiller/.node-gyp/10.1.0/include/node/common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=/Users/distiller/.node-gyp/10.1.0',
gyp info spawn args   '-Dnode_gyp_dir=/Users/distiller/.nvm/versions/node/v10.1.0/lib/node_modules/npm/node_modules/node-gyp',
gyp info spawn args   '-Dnode_lib_file=/Users/distiller/.node-gyp/10.1.0/<(target_arch)/node.lib',
gyp info spawn args   '-Dmodule_root_dir=/Users/distiller/tidepool-org/chrome-uploader/node_modules/node-mtp',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'build',
gyp info spawn args   '-Goutput_dir=.' ]
/bin/sh: pkg-config: command not found

Now, if understood correctly from the article, if I have a darwin-x64 prebuild the CI server shouldn't have to rebuild the module for my Electron app. If this is the case, why is it still rebuilding? Please let me know what other info I can provide to troubleshoot further?

Could it be because I'm linking to a shared library? In which case, should I include the shared library in the module itself as a static library?

Rough idea: prebuildify would write a JSON file containing build metadata (platform, arch, abi, etc). A secondary tool (serving a similar role as prebuildify-ci) would collect those files and generate a binding.js to be included in your npm package. E.g.:

if (platform === 'linux' && arch === 'x64') {
  module.exports = require('./prebuilds/linux-x64/example.node')
} else {
  throw new Error('No native build was found for ...')
}

Removing the need to discover files at runtime.

Currently, errors are silent, which makes it problematic to know what went wrong when loading an addon failed.

node-gyp-build-the-executable currently detects whether there's a build or prebuild available by running a script that loads node-gyp-build-the-library, which (reasonably) throws an error if the build/prebuild is missing. It also only shows stderr output from this script in "verbose" mode, which is also reasonable. However, this means that yarn install --verbose of a library using node-gyp-build can end up with an "error" in the log output, even though it's just how node-gyp-build is detecting whether there's a build or not. This can lead to confusion, especially if there's an actual problem later on (see signalapp/libsignal#456).

I personally think the best solution is to separate the error-throwing from the search. This would allow other errors to still be displayed, as well as distinguishing between "the build is missing" and "the build is present but cannot be loaded" (rebuilding may not help in the latter case).

Should be using linux, osx and windows.

Bug report

• I confirm this is a bug with Supabase, not with my own application.
• I confirm I have searched the Docs, GitHub Discussions, and Discord.

Describe the bug

I'm receiving this warning (wasn't receiving previous to reinstalling from package.json):

./node_modules/node-gyp-build/index.js
Critical dependency: require function is used in a way in which dependencies cannot be statically extracted

Import trace for requested module:
./node_modules/node-gyp-build/index.js
./node_modules/bufferutil/index.js
./node_modules/websocket/lib/WebSocketFrame.js
./node_modules/websocket/lib/websocket.js
./node_modules/websocket/index.js
./node_modules/@supabase/realtime-js/dist/main/RealtimeClient.js
./node_modules/@supabase/realtime-js/dist/main/index.js
./node_modules/@supabase/supabase-js/dist/main/index.js
./app/api/supabase/createClient.ts
./app/api/supabase/index.ts
./app/api/index.ts
./app/decks/ai/create/[id]/page.tsx

When I uninstall "@supabase/supabase-js": "^2.33.1", from the app and remove the client code:

import { SupabaseClient, createClient } from "@supabase/supabase-js";


const supabaseUrl = process.env.SUPABASE_URL;
const supabaseKey = process.env.SUPABASE_ANON_KEY;

export const supabase = createClient<SupabaseClient<any, "public", any>>(
    `${supabaseUrl}`, 
    `${supabaseKey}`,
    {
        auth: {
            persistSession: false
        }
    }
);

The warning goes away.

To Reproduce

1. Install the package in a nextjs app
2. run next dev

Expected behavior

I expect there to be no warning as nothing seems to be wrong with the code in node-gyp (however I could be wrong) which is added by websocket:

if (typeof require.addon === 'function') { // if the platform supports native resolving prefer that
  module.exports = require.addon.bind(require)
} else { // else use the runtime version here
  module.exports = require('./node-gyp-build.js')
}

System information

• MacOS Sonoma
• next dev cli
• Version of supabase-js: "@supabase/supabase-js": "^2.33.1",
• Version of Node.js: happens in both 16 and 18

Additional context

Let me know if you have any insights on how to remedy or turn off the warning

This is related to an issue I have open with @parcel/watcher.

When building on my local Ubuntu 20.04 machine, @parcel/watcher installs successfully because node-gyp-build is able to find the linux-x64 prebuild supplied with the package. However, when building in our Gitlab CI environment, the prebuild is not found. While this does end up exposing an apparent build error in @parcel/watcher, I can't figure out why node-gyp-build is failing to detect the prebuild.

I've attached npm ci --loglevel silly logs below. Here is the tree of @parcel/watcher when it is successfully installed on my local machine.

.
├── prebuilds
│   ├── darwin-arm64
│   │   └── node.napi.glibc.node
│   ├── darwin-x64
│   │   └── node.napi.glibc.node
│   ├── linux-x64
│   │   ├── node.napi.glibc.node
│   │   └── node.napi.musl.node
│   └── win32-x64
│       └── node.napi.glibc.node
└── src
     ├── ...

If a fix can't be found, could we add some logging that might help debugging in the future?

node-gyp-build_noPrebuildFound.txt

When using node-gyp-build to build a Node.js native addon:
is the node-gyp package required in the project too, or does node-gyp-build have the same capabilities?

In my Node.js project, I have a binding.gyp file, C and C++ code. I use node-gyp-build and also prebuildify to create prebuilds to speed up local development.

Hi All,
Facing an Issue while Installing Hardhat using yarn add --dev hardhat, there is an error:

error ./node_modules/classic-level: Command failed.
Exit code: 127
Command: node-gyp-build

Even tried using npm, Got same error.
npm Version of Error:

npm ERR! code 127
npm ERR! path ./node_modules/classic-level
npm ERR! command failed
npm ERR! command sh -c node-gyp-build
npm ERR! sh: 1: node-gyp-build: not found

My Node version: v18.14.0

I'm trying to install a library for an electron app I am writing

In the library they use

const lib = require('node-gyp-build')(__dirname);

to load the module.

This seems correct based on the README here. However, my project uses npm workspaces and this is causing a problem:

__dirname evaluates to myproject/packages/mypackage even though the library is actually installed to myproject/node_modules/library

The end result being an error:

 No native build was found for platform=win32 arch=x64 runtime=node abi=93 uv=1 libc=glibc node=16.18.1

I'm not sure how to proceed -- ultimately I could patch the package to force it to use the correct path but I'm curious if there is a more reasonable path forward for supporting npm workspaces using this tool

See prebuild/prebuild-install#99 for an example.

It looks like prebuildify will try to strip binary if option PREBUILD_STRIP specified.
will this feature be ported to node-gyp-build?
see:
https://github.com/prebuild/prebuildify/blob/5b2c60620daae74c8d1deb6e838d1ac7cceeac16/index.js#L242

Currently, it is quite hard to understand why a prebuild is not picked up, especially when the log comes from the end user. I think adding a console.log that includes some information like the host's platform, architecture, libc, etc. will help debug the issues.

My Electron project's target platform is Windows x64, but it's built on a Linux system. I already have a native module which uses "prebuild". To install dependecies I run:

CI=1 npm_config_platform=win32 npm_config_arch=x64 npm_config_target=9.1.1 npm_config_runtime=electron npm ci

This has been working correctly.

Now I need to add another native module: ffi-napi. They use prebuildify according to this pull request. It seems, it doesn't work the same way here.

I get this error message:

[email protected] install /.../myapp/node_modules/ffi-napi
node-gyp-build
gyp WARN install got an error, rolling back install
gyp ERR! configure error
gyp ERR! stack Error: 404 response downloading https://nodejs.org/dist/v9.1.1/node-v9.1.1-headers.tar.gz

I see two major problems here:

1. It tries to download headers for NodeJS v9.1.1, but "9.1.1" is the version of Electron. I guess there is no NodeJS 9.1.1 and that's probably the cause for the 404 status code.
2. Downloading headers means it's trying to build instead of using the prebuild - but I'm not sure about that.

It's interesting that ff-napi also has another native dependencies ref-napi and that it seems there are no problems with that:

[email protected] install /.../myapp/node_modules/ref-napi
node-gyp-build
[email protected] install /.../myapp/node_modules/ffi-napi
gyp WARN install got an error, rolling back install
(see above)

Hi, I have an issue with this line

I wrapped libuiohook using N-API in uiohook-napi.
libuiohook uses some compiler specific code, we can speak about it as side effects on require().

So basically when I try to install my package on linux without desktop environment (CI builds), I got this

npm info lifecycle [email protected]~install: [email protected]

> [email protected] install /home/runner/work/test-ci-bug/test-ci-bug/node_modules/uiohook-napi
> node-gyp-build

on_library_load [441]: XOpenDisplay failure!
settings_thread_proc [121]: XOpenDisplay failure!
load_input_helper [1797]: XkbGetKeyboard failed to locate a valid keyboard!
Segmentation fault (core dumped)

Because obviously there is no X11, node-gyp-build-test process crashes and node-gyp rebuild is triggered.

My proposal is to add some flag like --naive, that will just check *.node file existence, not actually require-ing it.

Hi Team,

Docker build is failing with node build error as below

[email protected] install /opt/atlassian/pipelines/agent/build/node_modules/bufferutil
node-gyp-build
make: Entering directory `/opt/atlassian/pipelines/agent/build/node_modules/bufferutil/build'
CC(target) Release/obj.target/bufferutil/src/bufferutil.o
../src/bufferutil.c:3:22: fatal error: node_api.h: No such file or directory
#include <node_api.h>

Attached is the compile error file . I am using Bitbucket pipeline.[
pipelineLog-264.txt
](url)

Packages that use node-gyp-build fail to install on node v20 when using yarn in the case where they don't bundle prebuilds for the platform already (i.e. when building is needed).

Using leveldown as an example (but the same happens with sodium-native, rocksdb, node-lmdb, etc):

$ docker run -it --rm -w /root node:20 /bin/bash
root@78779b01eaaf:~# npm_config_build_from_source=true yarn add leveldown
yarn add v1.22.19
info No lockfile found.
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
error /root/node_modules/leveldown: Command failed.
Exit code: 1
Command: node-gyp-build
Arguments: 
Directory: /root/node_modules/leveldown
Output:
node:events:492
      throw er; // Unhandled 'error' event
      ^

Error: spawn node-gyp ENOENT
    at ChildProcess._handle.onexit (node:internal/child_process:286:19)
    at onErrorNT (node:internal/child_process:484:16)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21)
Emitted 'error' event on ChildProcess instance at:
    at ChildProcess._handle.onexit (node:internal/child_process:292:12)
    at onErrorNT (node:internal/child_process:484:16)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
  errno: -2,
  code: 'ENOENT',
  syscall: 'spawn node-gyp',
  path: 'node-gyp',
  spawnargs: [ 'rebuild' ]
}

Node.js v20.5.1
info Visit https://yarnpkg.com/en/docs/cli/add for documentation about this command.
root@78779b01eaaf:~#

Here the npm_config_build_from_source env variable is set to true to emulate the behaviour of --build-from-source on NPM.

When no building is needed, everything works as expected:

$ docker run -it --rm -w /root node:20 /bin/bash
root@c7c921d3c316:~# yarn add leveldown
yarn add v1.22.19
info No lockfile found.
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
success Saved lockfile.
success Saved 12 new dependencies.
info Direct dependencies
└─ [email protected]
info All dependencies
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
└─ [email protected]
Done in 3.72s.
root@c7c921d3c316:~#

node-webkit / nwjs is supported by node-abi, yet, is unusable in node-gyp-build due to a bug in runtime negotiation.

Find related PR : #43

• allow injecting a prebuild using an env var
• support prebuild.test for test script setup
• preinstall and postinstall arguments

WARNING Compiled with 2 warnings 11:32:56

warning in ./node_modules/node-gyp-build/index.js

13:9-32 Critical dependency: the request of a dependency is an expression

warning in ./node_modules/node-gyp-build/index.js

20:15-54 Critical dependency: the request of a dependency is an expression