staltz / ssb-keys-neon Goto Github PK
View Code? Open in Web Editor NEWA drop-in replacement of ssb-keys, implemented in Rust and delivered as a native module in Node.js
License: GNU Lesser General Public License v3.0
A drop-in replacement of ssb-keys, implemented in Rust and delivered as a native module in Node.js
License: GNU Lesser General Public License v3.0
Installing ssb-keys-neon with npm on OpenBSD amd64 produces an error after the rust compilation which seems to happens correctly.
kongroo$ npm install ssb-keys-neon
> [email protected] install /home/test/node_modules/ssb-keys-neon
> neon-load-or-build
/home/test/node_modules/neon-load-or-build/index.js:62
throw new Error('No native build was found for ' + target + '\n loaded from: ' + dir + '\n')
^
Error: No native build was found for platform=openbsd arch=x64 runtime=node abi=72 uv=1 libc=glibc node=12.16.1
loaded from: /home/test/node_modules/ssb-keys-neon
at Function.load.path (/home/test/node_modules/neon-load-or-build/index.js:62:9)
at load (/home/test/node_modules/neon-load-or-build/index.js:27:30)
at Object.<anonymous> (/home/test/node_modules/neon-load-or-build/build-test.js:19:19)
at Module._compile (internal/modules/cjs/loader.js:1158:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1178:10)
at Module.load (internal/modules/cjs/loader.js:1002:32)
at Function.Module._load (internal/modules/cjs/loader.js:901:14)
at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:74:12)
at internal/main/run_main_module.js:18:47
neon info forcing rebuild for new build settings
neon info running cargo
Updating crates.io index
Downloaded stream-cipher v0.4.1
Downloaded aho-corasick v0.7.6
Downloaded cc v1.0.50
Downloaded opaque-debug v0.3.0
Downloaded semver v0.9.0
Downloaded curve25519-dalek v3.0.0
Downloaded semver-parser v0.7.0
Downloaded serde v1.0.116
Downloaded serde_derive v1.0.116
Downloaded quote v1.0.2
Downloaded thiserror v1.0.20
Downloaded typenum v1.12.0
Downloaded universal-hash v0.4.0
Downloaded zerocopy v0.3.0
Downloaded signature v1.2.2
Downloaded zeroize v1.1.1
Downloaded zeroize_derive v1.0.1
Downloaded aead v0.3.2
Downloaded cslice v0.2.0
Downloaded base64 v0.12.3
Downloaded cpuid-bool v0.1.2
Downloaded crypto-mac v0.9.1
Downloaded block-buffer v0.9.0
Downloaded digest v0.9.0
Downloaded generic-array v0.14.4
Downloaded hmac v0.9.0
Downloaded itoa v0.4.5
Downloaded poly1305 v0.6.0
Downloaded neon-runtime v0.5.1
Downloaded ppv-lite86 v0.2.9
Downloaded ed25519 v1.0.2
Downloaded proc-macro2 v1.0.21
Downloaded rand_chacha v0.2.2
Downloaded ed25519-dalek v1.0.1
Downloaded rand_core v0.5.1
Downloaded neon-build v0.5.1
Downloaded subtle v2.3.0
Downloaded sha2 v0.9.1
Downloaded synstructure v0.12.4
Downloaded thiserror-impl v1.0.20
Downloaded arrayvec v0.5.1
Downloaded version_check v0.9.2
Downloaded base64 v0.11.0
Downloaded cfg-if v0.1.10
Downloaded salsa20 v0.5.2
Downloaded byteorder v1.3.4
Downloaded ryu v1.0.2
Downloaded unicode-xid v0.2.0
Downloaded smallvec v1.4.2
Downloaded xsalsa20poly1305 v0.4.2
Downloaded thread_local v1.0.1
Downloaded getrandom v0.1.15
Downloaded lazy_static v1.4.0
Downloaded memchr v2.3.0
Downloaded neon-sys v0.5.1
Downloaded ssb-crypto v0.2.1
Downloaded ssb-keyfile v0.4.0
Downloaded private-box v0.6.0
Downloaded serde_json v1.0.45
Downloaded rand v0.7.3
Downloaded x25519-dalek v1.1.0
Downloaded neon v0.5.1
Downloaded regex-syntax v0.6.13
Downloaded syn v1.0.41
Downloaded regex v1.3.3
Downloaded libc v0.2.66
Downloaded zerocopy-derive v0.2.0
Downloaded 67 crates (3.9 MB) in 4.63s
Compiling proc-macro2 v1.0.21
Compiling unicode-xid v0.2.0
Compiling version_check v0.9.2
Compiling typenum v1.12.0
Compiling syn v1.0.41
Compiling cfg-if v0.1.10
Compiling memchr v2.3.0
Compiling lazy_static v1.4.0
Compiling libc v0.2.66
Compiling regex-syntax v0.6.13
Compiling subtle v2.3.0
Compiling cc v1.0.50
Compiling getrandom v0.1.15
Compiling byteorder v1.3.4
Compiling ppv-lite86 v0.2.9
Compiling serde_derive v1.0.116
Compiling signature v1.2.2
Compiling cpuid-bool v0.1.2
Compiling opaque-debug v0.3.0
Compiling ryu v1.0.2
Compiling serde v1.0.116
Compiling base64 v0.12.3
Compiling semver-parser v0.7.0
Compiling itoa v0.4.5
Compiling cslice v0.2.0
Compiling smallvec v1.4.2
Compiling base64 v0.11.0
Compiling arrayvec v0.5.1
Compiling neon-build v0.5.1
Compiling thread_local v1.0.1
Compiling generic-array v0.14.4
Compiling ed25519 v1.0.2
Compiling semver v0.9.0
Compiling quote v1.0.2
Compiling aho-corasick v0.7.6
Compiling regex v1.3.3
Compiling rand_core v0.5.1
Compiling digest v0.9.0
Compiling stream-cipher v0.4.1
Compiling block-buffer v0.9.0
Compiling universal-hash v0.4.0
Compiling crypto-mac v0.9.1
Compiling aead v0.3.2
Compiling synstructure v0.12.4
Compiling rand_chacha v0.2.2
Compiling sha2 v0.9.1
Compiling poly1305 v0.6.0
Compiling hmac v0.9.0
Compiling rand v0.7.3
Compiling neon-sys v0.5.1
Compiling zeroize_derive v1.0.1
Compiling zerocopy-derive v0.2.0
Compiling thiserror-impl v1.0.20
Compiling zeroize v1.1.1
Compiling neon-runtime v0.5.1
Compiling curve25519-dalek v3.0.0
Compiling salsa20 v0.5.2
Compiling neon v0.5.1
Compiling ssb-keys-neon v8.0.0 (/home/test/node_modules/ssb-keys-neon/native)
Compiling ed25519-dalek v1.0.1
Compiling x25519-dalek v1.1.0
Compiling zerocopy v0.3.0
Compiling xsalsa20poly1305 v0.4.2
Compiling thiserror v1.0.20
Compiling ssb-crypto v0.2.1
Compiling private-box v0.6.0
Compiling serde_json v1.0.45
Compiling ssb-keyfile v0.4.0
Finished release [optimized] target(s) in 4m 31s
neon info generating native/index.node
neon ERR! ENOENT: no such file or directory, copyfile '/home/test/node_modules/ssb-keys-neon/native/target/release/undefinedssb_neon_keysundefined' -> '/home/test/node_modules/ssb-keys-neon/native/index.node'
Error: ENOENT: no such file or directory, copyfile '/home/test/node_modules/ssb-keys-neon/native/target/release/undefinedssb_neon_keysundefined' -> '/home/test/node_modules/ssb-keys-neon/native/index.node'
npm WARN enoent ENOENT: no such file or directory, open '/home/test/package.json'
npm WARN [email protected] requires a peer of secret-stack@>=6.2.0 but none is installed. You must install peer dependencies yourself.
npm WARN test No description
npm WARN test No repository field.
npm WARN test No README data
npm WARN test No license field.
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] install: `neon-load-or-build`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR! /home/test/.npm/_logs/2020-11-06T07_51_13_664Z-debug.log
ssb-neon
repossbSecretKeyToPrivateBoxSecret
Looks good, eh? https://docs.travis-ci.com/user/multi-cpu-architectures/
Read this thread: prebuild/prebuildify#25
Check out leveldown's CI configs: https://github.com/Level/leveldown
We should make it possible for people to download this package to replace ssb-keys
, like this in package.json
"dependencies": {
- "ssb-keys": "7.2.0",
+ "ssb-keys": "staltz/ssb-neon-keys#replace-7.2.0",
The package.json > name
should be ssb-keys
because that's necessary to make the replacement above work when npm install
happens, but because of the install script that runs node-pre-gyp
, we have to maintain some metadata in order to fetch the .tar files from GitHub releases.
replacer
branch to test if this worksreplace-X.Y.Z
branchreplace-X.Y.Z
create and load presigil-legacy async
{"curve":"ed25519","id":"xPvwSVYnIaRiAbQWndTeHq5GUmzhEX49hkurrAiJ0mI=.ed25519","private":"TW/2MuiUL2MK08qdiHsk7j/FfoE6msWRHFgtapMDD/nE+/BJVichpGIBtBad1N4erkZSbOERfj2GS6usCInSYg==.ed25519","public":"xPvwSVYnIaRiAbQWndTeHq5GUmzhEX49hkurrAiJ0mI=.ed25519"}
./ssb-neon-keys/test-extra/index.js:11
var k2 = ssbkeys.loadSync(path);
^
Error: Failed to deserialize secret file: Invalid multikey: Invalid sigil at line 1 column 78
Perhaps ssb-multiformats(-rs) and ssb-keyfile(-rs) don't support presigil-legacy id
field?
TODO
Hey @sbillig, after merging your PR I also added benchmarks (see the ./benchmarks
directory) from an older branch. I remember the results being consistently ssb-neon-keys lasting 95%
of the time that ssb-keys lasts, but with the new changes it's now:
➜ ssb-neon-keys git:(master) node benchmarks/test.js
ssb-keys ran in 3143ms
ssb-neon-keys ran in 26469ms (842.2%)
➜ ssb-neon-keys git:(master) node benchmarks/test.js
ssb-keys ran in 3319ms
ssb-neon-keys ran in 29489ms (888.5%)
Maybe this is due to Dalek? From your PR I didn't see any significant changes other than libsodium=>dalek.
Slightly different to #23 because it doesn't mention rebuilding it? Electron seems to launch correctly though.
Getting this in manyverse desktop when I tried to update to electron 11
> [email protected] electron /home/me/prj/ssb/manyverse/desktop/nodejs-project
> electron .
App threw an error during load
Error: No native build was found for platform=linux arch=x64 runtime=electron abi=85 uv=1 libc=glibc node=12.18.3 electron=11.2.1
loaded from: /home/me/prj/ssb/manyverse/desktop/nodejs-project/node_modules/ssb-keys-neon
at Function.SeIAA+v1biYg+xx+VMFvALJXYDOtGjIMIN+wmwqcARc=.load.path (/home/me/prj/ssb/manyverse/desktop/nodejs-project/index.js:47747:9)
at load (/home/me/prj/ssb/manyverse/desktop/nodejs-project/index.js:47712:30)
at Object.SA2A364X82V7Hc9sltQAMSM5AJ/s1AXFK9wKY8Lmu/Q= (/home/me/prj/ssb/manyverse/desktop/nodejs-project/index.js:46626:47)
at load (/home/me/prj/ssb/manyverse/desktop/nodejs-project/index.js:14:8)
at /home/me/prj/ssb/manyverse/desktop/nodejs-project/index.js:17:29
at Object.ods4p4vuydn9swzsvmQ6C/RgGtgTL9B2E8ySD4OmAdc= (/home/me/prj/ssb/manyverse/desktop/nodejs-project/index.js:73084:17)
at load (/home/me/prj/ssb/manyverse/desktop/nodejs-project/index.js:14:8)
at /home/me/prj/ssb/manyverse/desktop/nodejs-project/index.js:17:29
at Object.ceDFZ1vphrnsjkSFXVorvao4nv1qtzyPZ7Um5/ddwuA= (/home/me/prj/ssb/manyverse/desktop/nodejs-project/index.js:58766:5)
at load (/home/me/prj/ssb/manyverse/desktop/nodejs-project/index.js:14:8)
A JavaScript error occurred in the main process
Uncaught Exception:
Error: No native build was found for platform=linux arch=x64 runtime=electron abi=85 uv=1 libc=glibc node=12.18.3 electron=11.2.1
loaded from: /home/me/prj/ssb/manyverse/desktop/nodejs-project/node_modules/ssb-keys-neon
at Function.SeIAA+v1biYg+xx+VMFvALJXYDOtGjIMIN+wmwqcARc=.load.path (/home/me/prj/ssb/manyverse/desktop/nodejs-project/index.js:47747:9)
at load (/home/me/prj/ssb/manyverse/desktop/nodejs-project/index.js:47712:30)
at Object.SA2A364X82V7Hc9sltQAMSM5AJ/s1AXFK9wKY8Lmu/Q= (/home/me/prj/ssb/manyverse/desktop/nodejs-project/index.js:46626:47)
at load (/home/me/prj/ssb/manyverse/desktop/nodejs-project/index.js:14:8)
at /home/me/prj/ssb/manyverse/desktop/nodejs-project/index.js:17:29
at Object.ods4p4vuydn9swzsvmQ6C/RgGtgTL9B2E8ySD4OmAdc= (/home/me/prj/ssb/manyverse/desktop/nodejs-project/index.js:73084:17)
at load (/home/me/prj/ssb/manyverse/desktop/nodejs-project/index.js:14:8)
at /home/me/prj/ssb/manyverse/desktop/nodejs-project/index.js:17:29
at Object.ceDFZ1vphrnsjkSFXVorvao4nv1qtzyPZ7Um5/ddwuA= (/home/me/prj/ssb/manyverse/desktop/nodejs-project/index.js:58766:5)
at load (/home/me/prj/ssb/manyverse/desktop/nodejs-project/index.js:14:8)
Used here:
ssb-keys-neon/.github/workflows/main.yml
Line 206 in e250744
I stumbled over this while playing with my own branch, and I see you also have a failed workflow for the same reason.
https://github.blog/changelog/2020-10-01-github-actions-deprecating-set-env-and-add-path-commands
Not sure how much work this will be
Dalek is really cool, and I would prefer it for performance, and for keeping things in one language (Rust), but I heard crypto libraries have to be triple checked for security, and Dalek being a young project, I don't see it being audited sufficiently.
ssb-crypto(-rs) allows switching to Libsodium as a togglable feature. We could switch back to Dalek once we have enough confidence and security audits.
That said, audits of Dalek's security:
electron-build-env
node-pre-gyp
with prebuildify
to ship .node
for all variants in the npm package, no GitHub releasesnode-pre-gyp install
should be node-pre-gyp install --runtime=electron --target=X.Y.Z
node-gyp-build
that looks for various binaries in ./prebuilds
./prebuilds
npm publish
from CInpm publish
from CIos.tmpdir()
The setup part with git checkout, Node installation, Rust installation, could be generalized into one action file in .github/actions/setup.yml
. This action would need some inputs: (1) node version, (2) os version (see #4), etc.
When installing the software with npm on a system without a native build, the display shows an error and continue, with a fallback to build the rust program.
I think this shouldn't be reported as an error because it's not blocking the installation process.
$ npm install ssb-keys-neon
> [email protected] install /home/test/node_modules/ssb-keys-neon
> neon-load-or-build
/home/test/node_modules/neon-load-or-build/index.js:62
throw new Error('No native build was found for ' + target + '\n loaded from: ' + dir + '\n')
^
Error: No native build was found for platform=openbsd arch=x64 runtime=node abi=72 uv=1 libc=glibc node=12.16.1
loaded from: /home/test/node_modules/ssb-keys-neon
at Function.load.path (/home/test/node_modules/neon-load-or-build/index.js:62:9)
at load (/home/test/node_modules/neon-load-or-build/index.js:27:30)
at Object.<anonymous> (/home/test/node_modules/neon-load-or-build/build-test.js:19:19)
at Module._compile (internal/modules/cjs/loader.js:1158:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1178:10)
at Module.load (internal/modules/cjs/loader.js:1002:32)
at Function.Module._load (internal/modules/cjs/loader.js:901:14)
at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:74:12)
at internal/main/run_main_module.js:18:47
neon info forcing rebuild for new build settings
neon info running cargo
Updating crates.io index
Downloaded stream-cipher v0.4.1
[...]
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.