Code Monkey home page Code Monkey logo

rust-cid's Introduction

rust-cid

Build Status Crates.io License Documentation Dependency Status Coverage Status

CID implementation in Rust.

Table of Contents

Usage

use multihash_codetable::{Code, MultihashDigest};
use cid::Cid;
use std::convert::TryFrom;

const RAW: u64 = 0x55;

fn main() {
    let h = Code::Sha2_256.digest(b"beep boop");

    let cid = Cid::new_v1(RAW, h);

    let data = cid.to_bytes();
    let out = Cid::try_from(data).unwrap();

    assert_eq!(cid, out);

    let cid_string = cid.to_string();
    assert_eq!(
        cid_string,
        "bafkreieq5jui4j25lacwomsqgjeswwl3y5zcdrresptwgmfylxo2depppq"
    );
    println!("{}", cid_string);
}

Your Cargo.toml needs these dependencies:

[dependencies]
cid = "0.7.0"

You can also run this example from this checkout with cargo run --example readme.

Testing

You can run the tests using this command: cargo test --all-features

You can run the tests for no_std using this command: cargo test --no-default-features

Maintainers

Captain: @dignifiedquire.

Contribute

Contributions welcome. Please check out the issues.

Check out our contributing document for more information on how we work, and about contributing in general. Please be aware that all interactions related to multiformats are subject to the IPFS Code of Conduct.

Small note: If editing the README, please conform to the standard-readme specification.

License

MIT ยฉ 2017 Friedel Ziegelmayer

rust-cid's People

Contributors

arqu avatar bantonsson avatar dependabot[bot] avatar dignifiedquire avatar dvc94ch avatar galargh avatar gnunicorn avatar gozala avatar hobofan avatar kayabanerve avatar koushiro avatar leavehouse avatar maciejhirsz avatar mattheworris avatar mxinden avatar paolobarbolini avatar rklaehn avatar stebalien avatar sunny-g avatar thomaseizinger avatar tomaka avatar tomusdrw avatar tyshko5 avatar vmx avatar web-flow avatar woss avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

rust-cid's Issues

Change Cid to depend on Multihash for storage

There is an optimization in multiformats/rust-multihash that stores the hash inline if it is is small. For rust-cid to benefit from this it would have to use a Multihash as storage in Cid instead of a Vec...

PartialEq, Hash, PartialOrd across Cid versions

Similar to #59 at the moment (0.5.1) the Cid is not equal across cid versions. I did not find anything from the specs or original posts on Cidv1 extension about this, but my intuition says cid::Version::V0 is merely a formatting option for std::fmt::Display.

What are the situations where V0 cid should not be equal to the same multihash and cid::Codec::DagProtobuf as V1? I guess the only valid use case is to separate on the Cid::version(&self) but for all content identification or addressing purposes these should be equal even if they have different string representations?

cidsum bin?

Given you need an IPFS server to generate checksums of files currently, it seems like a cidsum or ipldsum binary to quickly generate the CID hash for files would be a handy tool to provide from rust-cid.

Example doesn't build on stable rust (2018 edition)

I took the example code, tweaked it for Rust 2018, then tried to run it with Rust 1.33.0

This failed, but was fixed with Cargo.toml including:

cid = "0.3.0"
multihash = "0.8.0"

and the example code changing the use statements to:

use multihash;
use cid::{Cid, Codec, Version};

Since Rust 2018 is now the default for new projects with stable Rust, I think it'd make sense to update the example.

Inconsistency between cid in rust and python

Hello,

I am facing an issue as I wanted to generate from two different languages the cid and it appears they don't match

In python, the following snippet returns bafkrea3cnruq

    from multihash import encode, Multihash, decode
    from multiformats_cid import make_cid, CIDv1

    hash_digest = encode(b"bli", "sha2-256")
    cid = make_cid(1, "raw", hash_digest)
    print(cid)

In rust, the following snippet returns bafkreic7fqdfhr7xaov7tlblba6ck2t7xs3btbedhvc6mrevphoexvy76e

#[cfg(test)]
mod tests {

        use cid::{
            multibase::Base,
            multihash::{Code, MultihashDigest},
            Cid,
        };

    #[test]
    fn should_create_cid() {
        let hash = Code::Sha2_256.digest(b"bli");
        let cid = Cid::new_v1(RAW, hash);
        dbg!(&cid.to_string());
    }

Maybe I am missing something here, but I would hope that the same string digested with the same hasher and using the same codec would return the same ci.

Can someone help ?

Features accidentally pull in optional dependencies

Currently the feature definitions are not ideal. E.g. does the std feature define serde/std which leads to serde being a dependency, even if the serde-codec feature is not enabled. The same is true for other dependencies. As features are additive, it actually shouldn't be needed to explicitly set serde/std and the std feature would enable it anyway.

Besides that it would make sense to clean the whole things up and make use of the dep: and '?` syntax. We are already on a minimum supported Rust version of 1.60, so we can use that syntax.

The serde-codec feature could then also be renamed to just serde, following rust-multihash: https://github.com/multiformats/rust-multihash/blob/452a933396adcd5915c53563d5017df76ae3ec26/Cargo.toml#L24-L25

CID does not support base32 encoding

Which is now the default for ipfs. E.g.

$ echo 1 | ipfs dag put
bafyreihtx752fmf3zafbys5dtr4jxohb53yi3qtzfzf6wd5274jwtn5agu
let cid = Cid::from_str("bafyreihtx752fmf3zafbys5dtr4jxohb53yi3qtzfzf6wd5274jwtn5agu");

produces Err(UnknownCodec)

support canonicalizing cid's

for garbage collection to work the store needs to store the cid not just a multihash. also bitswap needs it for no obvious reason. the main advantage of using multihashes instead of cids would be (appart from the minuscle storage savings) that v0 and v1 cids containing the same multihash are treated the same.

Simple fixes would be:

  • add a into_v1 or canonicalize method to Cid
  • make to_bytes_v1 public or add a to_canonical_bytes method

Incompatible multihash

I'm rather new to Rust so unsure how to solve this on my end. Might be trivial, if so sorry for the noise.

Looks like the latest release of cid is 0.10.1, which uses multihash 0.18.1.

But libp2p-core, libp2p-identity, libp2p-noise, and multiaddr are on 0.19.1, as are my crates.

When I try to create a cid_v1 from a 0.19.1 multihash, it doesn't compile complaining that Multihash<64> and MultihashGeneric<64> have similar names but are distinct types.

It looks like the main branch has the 0.19.1 version of multiaddr. Is it possible to cut a new release of the CID crate?

Consider specifying MSRV

Hello, your most recent release (0.8.4) has bumped the crate's Minimum Supported Rust Version to 1.59 by making use of const generic defaults in serde's support.

This can be problematic in some instances, or confusing, especially since the release is a patch release so cargo upgrades it automatically by default...
I recommend you specify the MSRV in the readme and add a check in CI so any MSRV breaking changes are clearer.

Some project consider MSRV bumps to be minor updates, so users can bump the minor release when they are ready to upgrade

Missing test suite

There should be a test suite including spec compliance (at least the examples from the spec should be tested verbatim) and some roundtrip tests using quickcheck.

It would be great if an arbitrary was available for other people to use in their tests, but that is a nice to have.

please update parity-scale-codec abrove to 2.0.0

related to #81

parity-scale-codec has released new version 2.0.0, and now it's 2.1.1. It has more features for this crate, and some part is not compatible with 1.3.5 version. In our project, we need this new version parity-scale-codec, please update it.

Thanks.

Free `Arbitrary` implementation from codetable

Once #140 is merged, only the Arbitrary implementations require the multihash-codetable. That's quite a big dependency, hence it would make sense to hard-code the codec and use the sha2 crate directly.

This way multihash-codetable can be moved into the dev-dependency section.

Provide better debug instance for Cid

The debug instance of Cid currently produces this:

Cid { version: V1, codec: 113, hash: Multihash { code: 18, size: 32, digest: [32, 32, 27, 46, 22, 182, 139, 47, 188, 18, 109, 13, 26, 113, 233, 236, 125, 19, 124, 192, 214, 238, 151, 47, 200, 231, 146, 216, 111, 223, 23, 7, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] } }

Unless I want to debug the cid crate itself, this is almost never what I want.

Maybe it would be best to offer two alternative debug formats, one (compact, default) just does the same as display and dumps the string representation. The other produces the above (except maybe a hex string instead of the digest data).

cid const constructors unusable

Cid's new_v0 constructor is const fn, but it can't actually be used as such:

use cid::{multihash::Multihash, Cid};

const fn multihash() -> Multihash {
    panic!("https://github.com/multiformats/rust-multihash/issues/330")
}

const _: Cid = match Cid::new_v0(multihash()) {
    Ok(o) => o,
    Err(_) => panic!(),
};

Fails to compile:

error[E0493]: destructor of `Result<CidGeneric<64>, cid::Error>` cannot be evaluated at compile-time
  --> src/lib.rs:7:22
   |
7  | const _: Cid = match Cid::new_v0(multihash()) {
   |                      ^^^^^^^^^^^^^^^^^^^^^^^^ the destructor for this type cannot be evaluated in constants
...
10 | };
   | - value is dropped here

For more information about this error, try `rustc --explain E0493`.

This is presumably because the top level error type is non-copy/contains an io::Error which cannot be const

Io(io::Error),

Here is the root cause of the error in the constructor

rust-cid/src/cid.rs

Lines 78 to 87 in 1df4e3f

pub const fn new_v0(hash: Multihash<S>) -> Result<Self> {
if hash.code() != SHA2_256 || hash.size() != 32 {
return Err(Error::InvalidCidV0Multihash);
}
Ok(Self {
version: Version::V0,
codec: DAG_PB,
hash,
})
}

Fix is either:

  • drop the illusion of const from the new_v0 constructor. This would propogate to new, and is a breaking change
  • Fix multihash (see issue below), take a new multihash version1. Then have new_v0 return a new, const-compatible InvalidCidV0Multihash error in its constructor, and propogate that to new. This is also a breaking change. This is my preference, and I'm happy to create a PR for such a change.

Footnotes

  1. note that this crate is on multihash 0.18.1, but latest is 0.19.0 - is there a reason for the lag? โ†ฉ

Add support for string deserialization with serde

I'm working on a CLI tool that needs to read CIDs in JSONs but the current implementation does not support visit_str, though Cid::try_from already supports reading from &str.

I don't understand why CID isn't compatible with Serde's data model, the reason isn't documented either:

rust-cid/src/serde.rs

Lines 1 to 6 in dae650c

//! CID Serde (de)serialization for the IPLD Data Model.
//!
//! CIDs cannot directly be represented in any of the native Serde Data model types. In order to
//! work around that limitation. a newtype struct is introduced, that is used as a marker for Serde
//! (de)serialization.
extern crate alloc;

I tried my hand at doing something about it since the Cid::try_from + visit_str are right at hand but failed miserably, getting stumped by the MainEntryVisitor.

rust-cid/src/serde.rs

Lines 83 to 105 in dae650c

/// Main visitor to deserialize a CID.
///
/// This visitor has only a single entry point to deserialize CIDs, it's
/// `visit_new_type_struct()`. This ensures that it isn't accidentally used to decode CIDs
/// to bytes.
struct MainEntryVisitor<const SIZE: usize>;
impl<'de, const SIZE: usize> de::Visitor<'de> for MainEntryVisitor<SIZE> {
type Value = CidGeneric<SIZE>;
fn expecting(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
write!(fmt, "a valid CID in bytes, wrapped in an newtype struct")
}
fn visit_newtype_struct<D>(self, deserializer: D) -> Result<Self::Value, D::Error>
where
D: de::Deserializer<'de>,
{
deserializer.deserialize_bytes(BytesToCidVisitor)
}
}
deserializer.deserialize_newtype_struct(CID_SERDE_PRIVATE_IDENTIFIER, MainEntryVisitor)

I'm keen on discussing more and implementing a solution.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.