Code Monkey home page Code Monkey logo

fastcrypto's Introduction

fastcrypto

crate Docs Build status Update the fastcrypto pointer Sui Apache2/MIT licensed Rust Version codecov

fastcrypto is a common cryptography library used in software at Mysten Labs. It contains three individual crates: fastcrypto, fastcrypto-zkp and fastcrypto-cli. They are published as independent crates to encourage reusability across different applications and domains.

fastcrypto is a wrapper library around several carefully selected crates with the following considerations:

  • Security: Whether the libraries are vulnerable to known attack vectors or possible misuses.
  • Performance: Whether the crate performs cryptographic operations with speed after extensive benchmarking. This is critical for the Sui Network to be performant when signing and verifying large amounts of transactions and certificates.
  • Determinism: Whether the signature is non-malleable.
  • Popularity: Whether the library is used by other consensus critical systems.

Furthermore, we extend the selected libraries with additional features:

  • Robust testing framework: Wycheproof tests and prop tests are added when possible to protect against arbitrary inputs and crafted edge cases.
  • Zeroization: Sensitive private key materials are cleared from memory when it goes out of scope using the zeroize trait. Note that this is best effort and does not guarantee that all sensitive data is cleared from memory as data may be copied or moved around by the compiler, FFI, etc.
  • Serialization: Effective and standardized serialization are required.

This library will be continuously updated with more schemes and faster implementations based on benchmarking results, RFC updates, new research and auditor inputs.

The fastcrypto crate contains:

  • Traits that should be implemented by concrete types representing digital cryptographic materials.

    • [SigningKey]: Trait implemented by the private key with associated types of its public key and signature.
    • [VerifyingKey]: Trait implemented by the public key with associated types of its private key and signature. It also includes a default implementation of batch verification that fails on empty batch verification.
    • [Authenticator]: Trait implemented by the signature with associated types of its public key and private key.
    • [AggregateAuthenticator]: Trait implemented by the aggregated signature, which allows adding signatures to the aggregated signature and verifying against the public keys with the corresponding messages.
    • [KeyPair]: Trait that represents a public/private keypair, which includes the common get priv/pub key functions and a keypair generation function with seeded randomness.
    • [ToFromBytes]: Trait that aims to minimize the number of steps involved in obtaining a serializable key.
    • [EncodeDecodeBase64]: Trait that extends ToFromBytes for immediate conversion to/from Base64 strings. This is the format in which cryptographic materials are stored.
  • Concrete signature schemes of type that implement the recommended traits required for cryptographic agility.

    • Ed25519: Backed by ed25519-consensus crate. Compliant to ZIP-215 that defines the signature validity that is lacking from RFC8032 but critical for consensus algorithms. ed25519-dalek is fully deprecated due to the recently discovered Chalkias double pub-key api vulnerability.
    • Secp256k1: ECDSA signatures over the secp256k1 curve. Backed by Secp256k1 FFI wrapper that binds to C library and provides performance faster than the native Rust implementation k256 library by ~30% on verification. Produces either a standard ECDSA signature or a 65-byte recoverable signature of shape [r, s, v] where v can be 0 or 1 representing the recovery Id. Produces deterministic signatures using the pseudo-random deterministic nonce generation according to RFC6979, without the strong requirement to generate randomness for nonce protection. Uses sha256 as the default hash function for sign and verify. An interface for verify_hashed is provided to accept a pre-hashed message and its signature for verification. Supports public key recovery by providing the Secp256k1 recoverable signature with the corresponding pre-hashed message. An accepted signature must have its s in the lower half of the curve order. If s is too high, normalize s to order - s where curve order is 0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFEBAAEDCE6AF48A03BBFD25E8CD0364141. See more at BIP-0062.
    • Secp256r1: ECDSA signatures over the secp256r1 curve backed by the p256 crate which is a pure rust implementation of the Secp256r1 (aka NIST P-256 and prime256v1) curve. The functionality from p256 is extended such that, besides standard ECDSA signatures, our implementation can also produce and verify 65 byte recoverable signatures of the form [r, s, v] where v is the recoveryID. Signatures are produced deterministically using the pseudo-random deterministic nonce generation according to RFC6979, without the strong requirement to generate randomness for nonce protection. Uses sha256 as the default hash function for sign and verify. Supports public key recovery by providing the Secp256r1 ECDSA recoverable signature with the corresponding pre-hashed message. An accepted signature must have its s in the lower half of the curve order. If s is too high, normalize s to order - s where curve order is 0xFFFFFFFF00000000FFFFFFFFFFFFFFFFBCE6FAADA7179E84F3B9CAC2FC632551 defined here.
    • BLS12-381: Backed by blst crate written in Assembly and C that optimizes for performance and security. G1 and G2 points are serialized following ZCash specification in compressed format. Provides functions for verifying signatures in the G1 group against public keys in the G2 group (min-sig) or vice versa (min-pk). Provides functions for aggregating signatures and fast verifying aggregated signatures, where public keys are assumed to be verified for proof of possession.
    • RSA: Backed by crate rsa. Provides functions to sign and verify RSA signature according to RFC 8017
  • Utility functions on cryptographic primitives. Some of them serve as the Rust implementation of the Move smart contract API in Sui.

    • HKDF: An HMAC-based key derivation function based on RFC-5869, to derive keypairs with a salt and an optional domain for the given keypair. This requires choosing an HMAC function that expands precisely to the byte length of a private key for the chosen KeyPair parameter.
    • Pedersen Commitment: Function to create a Pedersen commitment with a value and a blinding factor. Add or subtract Ristretto points that represent Pedersen commitments.
    • Bulletproofs Range Proof: Function to prove that a committed value is an unsigned integer that is within the range [0, 2^bits). Function to verify that the commitment is a Pedersen commitment of some value with an unsigned bit length, a value is an integer within the range [0, 2^bits).
    • Elliptic Curve VRF (ECVRF): A verifiable random function implementation using the Ristretto255 group. Function to create a proof based on a given input and verify a proof for a given output, based on specification in draft-irtf-cfrg-vrf-15.
  • Encoding: Base64 and Hex are defined with an encoding trait with its customized serialization and validations, backed by base64ct and hex. Notably, the base64ct crate has been chosen instead of the most popular base64 Rust crate, because (a) it is constant time and (b) mangled encodings are explicitly rejected to prevent malleability attacks when decoding, see paper on in-depth analysis.

  • Hash functions wrappers: SHA2_256 with 256 bit digests, SHA3_256 with 256 bit digests, SHA2_512 with 512 bit digests, SHA3_512 with 512 bit digests, KECCAK with 256 bit digests, BLAKE2-256 with 256 bit digests.

  • Multiset Hash: A hash function where the output of the hash function is a point on the elliptic curve. It also allows for efficient computation for the hash of the union of two multiset.

  • A asynchronous signature service is provided for testing and benchmarking.

The fastcryto-zkp crate contains APIs to verify a Groth16 proof along with its prepared verifying key and public inputs. BN254 and BLS12381 curves are supported. The verifier is backed Arkworks and blst libraries.

The fastcrypto-cli crate includes CLI tools available for debugging. See usages with -h flag.

$ cargo build --bin encode-cli
$ target/debug/encode-cli -h
$ cargo build --bin sigs-cli
$ target/debug/sigs-cli -h
$ cargo build --bin ecvrf-cli
$ target/debug/ecvrf-cli -h

Tests

There exist unit tests for all primitives in all three crates, which can be run by:

$ cargo test

Benchmarks

In fastcrypto, one can compare all currently implemented signature schemes for sign, verify, verify_batch and key-generation by running:

$ cargo bench

A report of the benchmarks is generated for each release, allowing easy comparison of the performance of the different cryptographic primitives and schemes available in fastcrypto. As an example, we get these timings for signing messages and verifying the signature for the different schemes in fastcrypto as of revision dd5adb:

Signature schemes comparison.

Below is another plot made using data from the benchmark report, showing benchmarks for batched signature verification where all signatures are on the same message:

Batched signature verification with all signatures on same message.

In fastcrypto-zkp, benchmarks can be ran for Arkworks to blst representation of field elements, and verifying Groth16 in BN254 and BLS12381:

$ cd fastcrypto-zkp/
$ cargo bench

License

All crates licensed under either of

fastcrypto's People

Contributors

andll avatar aschran avatar bajpai244 avatar benr-ml avatar bmwill avatar chargarlic avatar cryptopapi997 avatar dependabot[bot] avatar ebmifa avatar erwanor avatar gooddaisy avatar hugehope avatar huitseeker avatar jnaulty avatar jonas-lj avatar joyqvq avatar kchalkias avatar mskd12 avatar mwtian avatar mystenmark avatar natalipatel8 avatar patrickkuo avatar punwai avatar rex4539 avatar stevenmia avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

fastcrypto's Issues

[crypto] Ad-hoc deserialization from ssh-keygen armored files

For genesis, we have node runners that like to generate their keypair (EdDSA) from the ssh-keygen tool (a well-trusted implementation).

We would like to be able to import and deserialize an EdDSA keypair as generated from ssk-keygen, which means interpreting the armored format for the private key. We would add this functionality as a simple pub function in out ed25519.rs, which would be integrated in our tooling Sui-side.

The rust crate rust-sshkeys provides a lot of tooling for reading the ssh pem files, but unfortunately, at the moment, only reads Pubkeys (whereas we would like to generate a keypair, hence reading private keys).

Use FastCryptoError in all schemes

Throughout fastcrypto, signature::Error is used as a default error. However, we should use the newly introduced FastCryptoError instead since signature::Error is from the signature crate.

Unsafe Fast Hash support

This will be required for benchmarking (similarly to empty/dummy signatures). It cannot empty (as collisions can break operations), maybe we can use the fastest non-cryptographically secure hash function for this purpose.

Add feature for `deterministic`

We should have a feature for functions that are 100% deterministic and do not include any randomness. This is required from blockchain projects that want to ensure no keygens or signing is available on tx execution and verification; usually only the deterministic verification part should be added as a dependency to avoid any accidental non-deterministic calls.

Implement MallocSizeOf for various structs

Currently, MallocSizeOf (a trait from mysten-infra) is implemented for PublicKeys, Signatures and AggregateSignatures for BLS123381 and Ed25519 in mysten-infra, because it is needed in Sui. However, this introduces som complexity because the fastcrypto version in Mysten-infra and Sui has to be exactly the same.

[crypto] Ed25519 security

Feedback from writing MystenLabs/narwhal#32 on our ed25519 implementation:

  • the "master" record of our keys is now a ed25519_dalek type, which is zeroized properly on drop => can we ensure this sort of thing is kept as a convention for future impls? perhaps with a unit test?
  • we do not perform any verifications on deserialization of a key, including:
    • canonical scalar representation for PrivKey,
    • small order element for Public keys and keypairs.
  • we should probably add warnings about small order components in keys, as they will disagree with single-signature verification.

Inspirations for accomplishing the above:

MoveVM supports schnorr aggregate signatures for Secp256K1.

MystenLabs/sui#4678

Solve the problem:Fully decentralized support for BTC Move contracts on the Sui network.

reference project: ChainX and ICP
their solution: BTC Light Node + Threshold Aggregate Signature + Smart Contract platform
For the BTC aggregate address to host BTC, only the addresses and accounts hosting BTC are decentralized enough to make BTC look as decentralized as POS. For example: Bind the aggregated custody account to the node account on the chain one by one, which is as decentralized as the POS chain.

There is still a problem with this way of thinking. Our OmniBTC wants to combine this solution of ChainX and the Lightning Network to be deployed on Sui to provide Sui with a fully decentralized BTC. It also allows BTC to carry the Move contract.

OmniBTC solution:
BTC Light Node + Threshold Aggregate Signature + BTC Lightning Network + MoveVM
Lightning Network solves the mutual trust problem between Alice and Bob.
We just need to replace Alice with a BTC aggregate signature account (co-hosted by a sufficiently decentralized POS node on Sui or a Dao administrator selected above).

BTC holders can fully trust the BTC to the Lightning Network, and then come to Sui to use any Defi and gamefi.
It all looks so cool. So we think it is necessary for MoveVM to support the schnorr signature algorithm of Secp256K1

@kchalkias

Rename to zip215 vs ed25519?

Wondering if we should avoid mentioning ed25519 directly and switch to zip215 for clarity. Obviously we should explain that it's an ed25519 mode.

[crypto] Implement display for our keys

At the moment our logs are riddled with the following output:

EdwardsPoint{
	X: FieldElement51([986312278720011, 308481451238697, 499818313507985, 2043434245230999, 529376140600966]),
	Y: FieldElement51([1122918900483925, 1840648545462405, 2049000554172011, 588549553128558, 2195713730548988]),
	Z: FieldElement51([1, 0, 0, 0, 0]),
	T: FieldElement51([167659880001565, 819468384591364, 1320141236765074, 698068994910468, 216442548444488])
}))

We should make sure we have a decent Display/Debug implemented for all public key types, especially as we're about to embed those data types that can contain keys of several types (for user keys) : we will want to e.g. derive / reuse Display on the Sui side.

Conversely, for the PrivateKey type, we should make sure our implementations of Debug / Display never display the byte content of the keys, but rather some "elided private ed25519 key" sort of string.

[crypto] Fast & Loose Key validation in FastNFT

The single error case of this TryFrom is just an invalid length error. There is a host of other problems completely ignored by the current implementation:

  • there are sequences of 32 bytes that will structurally not ever be usable as an Ed25519 Public Key in any way,
  • there are sequences of 32 bytes that will 100% be usable as an Ed25519 Public Key, but that demonstrably and unambiguously aim at tricking anybody who does so.

The above takes exactly none of that into account. Further, several of those checks will not be performed by check_internal's dalek::PublicKey::from_bytes (and the library has a nice warning to mention some of that).

I admit it's probably a completely orthogonal point to this PR, and worth tackling in a different issue (probably extracted from this comment), but I'd appreciate a spectacular comment on PublicKeyBytes making this clear. Here is an example of my personal minimum bar for the word "spectacular".

Originally posted by @huitseeker in MystenLabs/sui#94 (comment)

[crypto] Follow-ups to aggregate signature correctness in #750

  1. Should we apply the same fix to Ed25519PublicKey:: verify_batch() at line 70?
  2. Just curious, when batch is empty, does the verification succeed or fail?

Originally posted by @mwtian in MystenLabs/narwhal#750 (comment)

Moreover, MystenLabs/narwhal#750 probably shows a bunch of tests that live in Sui and check behavior with batches of signatures (to deal with "obligations") should probably live in Narwhal (to catch issues faster!).

/cc @joyqvq @punwai

Follow-ups of #562

Approving this, and we need a few extra upcoming PRs

  • clear api in our codebase for signing using a) randomized signatures b) deterministic RFC6979, default should be the randomized (just for consistency with most implementations out there and the main ECDSA standard).
  • an extra test for randomized sigs (sign with implA, verify with implB and vice versa)
  • comparison benchmark and test for ecrecover as well.
  • ECDSA-NM (which is already in progress)

Originally posted by @kchalkias in MystenLabs/narwhal#562 (review)

[crypto] Make sure the crypto API does not expose the hash of a message

At the moment signature verification passes a hash as an argument:
https://github.com/MystenLabs/narwhal/blob/36968d41de1680a8d79dbc3f7c301f322ba4ad1c/crypto/src/lib.rs#L210-L230

https://github.com/SalusaSecondus/CryptoGotchas/blob/master/README.md#signatures

An attacker mustn't be allowed to select the actual value being validated in the signature. (The hashing step in all standard signatures defends against that as they can only select the hash pre-image, not the value of the hash.) If they could then they could trivially craft a valid signature for an arbitrary public key by (essentially) generating a random signature and seeing what message would be verified by that signature and then returning that message/signature pair.

Keypair Serde is Broken

Description

Keypair serializes, but does not deserialize when trying to test it out in MystenLabs/narwhal#557. Does not break Narwhal nor Sui so continuing with MystenLabs/narwhal#557 anyways.

To replicate this issue, run the following test:

#[test]
fn test_kp_serde() {
    let kp = keys().pop().unwrap();
    let serialized = bincode::serialize(&kp).unwrap();
    let deserialized: Ed25519KeyPair = bincode::deserialize(&serialized).unwrap();
}

We get this error:

thread 'bls12377_tests::test_kp_serde' panicked at 'called `Result::unwrap()` on an `Err` value: Custom("invalid Base64 encoding")', crypto/src/tests/bls12377_tests.rs:375:75

Upon further digging, it looks like removing this line fixes the issue:

#[serde(tag = "type")] // REMOVE THIS LINE
pub struct Ed25519KeyPair {
    name: Ed25519PublicKey,
    secret: Ed25519PrivateKey,
}

[BLS experiments] PubKey addition optimization on aggregated sigs

This is low priority and probably over-optimization. BLS aggregated sig verification requires the list of pubkeys to verify against. If the size of this set is large (ie thousands), then aggregation requires some effort (thousand additions in BLS12-381 curve). There are a few tricks here:

  • cache most common subsets (ie for validators the top list of stake owners), an LRU might also be useful here
  • if we know the full set of signers (N) and because we need 67% (K) of sigs, sometimes it's better to have the SumAllPubKeys already cached and just remove those who didn't sign (by subtraction).

We should add hash wrappers as well for all Sui/Narwhal used functions

This will help to avoid surprises in Sui or Narwhal re using different sha2/sha3/keccak deps in different places. Ideally we should control every crypto algorithm in fastcrypto. After having these wrappers, we should update both Narwhal and Sui repos by invoking fastcrypto's api.

I was thinking that the same applies for base64, but let's wait for this, as Rust's popular base64 crate is working on the malleability fix.

[BLS] implement BLS12-381 with keys in G1 and sigs in G2

Also required to support drand.love that requires G1 keys. Ensure we have good struct names, so we can distinguish between BLS12381G1keys and BLS12381G2keys.

Note that Sui uses keys in G2 because they are reusable inside epochs, thus we want to optimize on sig size.

[EdDSA] Faster subgroup checks for Twisted Edwards curves

These methods are not yet implemented in ed25519-consensus's backend:
https://github.com/zkcrypto/curve25519-dalek-ng/blob/763a0faaf54752b06702b39e2296173ab76c2204/src/edwards.rs#L887-L920

We may want to implement them and/or contribute that implementation upstream.

[signatures] implement `verify_batch_empty_fail_different_msg`

Atm we only support batch verification of signatures over the same message, we should add functionality to batch verify signatures over different messages too. In practice we only expect impact on BLS timings, as ed25519 batch verification does not care if the message is the same or not.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.