multiformats / rust-cid Goto Github PK
View Code? Open in Web Editor NEWCID in rust
CID in rust
I think ToCid is not needed anymore now that TryInto is in stable. It is basically the same thing, a conversion that might fail with an error, but more idiomatic.
Please review multiformats/cid#26
Once #140 is merged, only the Arbitrary
implementations require the multihash-codetable
. That's quite a big dependency, hence it would make sense to hard-code the codec and use the sha2
crate directly.
This way multihash-codetable
can be moved into the dev-dependency section.
for garbage collection to work the store needs to store the cid not just a multihash. also bitswap needs it for no obvious reason. the main advantage of using multihashes instead of cids would be (appart from the minuscle storage savings) that v0 and v1 cids containing the same multihash are treated the same.
Simple fixes would be:
I took the example code, tweaked it for Rust 2018, then tried to run it with Rust 1.33.0
This failed, but was fixed with Cargo.toml including:
cid = "0.3.0"
multihash = "0.8.0"
and the example code changing the use statements to:
use multihash;
use cid::{Cid, Codec, Version};
Since Rust 2018 is now the default for new projects with stable Rust, I think it'd make sense to update the example.
Hello,
I am facing an issue as I wanted to generate from two different languages the cid and it appears they don't match
In python, the following snippet returns bafkrea3cnruq
from multihash import encode, Multihash, decode
from multiformats_cid import make_cid, CIDv1
hash_digest = encode(b"bli", "sha2-256")
cid = make_cid(1, "raw", hash_digest)
print(cid)
In rust, the following snippet returns bafkreic7fqdfhr7xaov7tlblba6ck2t7xs3btbedhvc6mrevphoexvy76e
#[cfg(test)]
mod tests {
use cid::{
multibase::Base,
multihash::{Code, MultihashDigest},
Cid,
};
#[test]
fn should_create_cid() {
let hash = Code::Sha2_256.digest(b"bli");
let cid = Cid::new_v1(RAW, hash);
dbg!(&cid.to_string());
}
Maybe I am missing something here, but I would hope that the same string digested with the same hasher and using the same codec would return the same ci.
Can someone help ?
I'm rather new to Rust so unsure how to solve this on my end. Might be trivial, if so sorry for the noise.
Looks like the latest release of cid is 0.10.1, which uses multihash 0.18.1.
But libp2p-core, libp2p-identity, libp2p-noise, and multiaddr are on 0.19.1, as are my crates.
When I try to create a cid_v1 from a 0.19.1 multihash, it doesn't compile complaining that Multihash<64>
and MultihashGeneric<64>
have similar names but are distinct types.
It looks like the main branch has the 0.19.1 version of multiaddr. Is it possible to cut a new release of the CID crate?
Similar to #59 at the moment (0.5.1) the Cid is not equal across cid versions. I did not find anything from the specs or original posts on Cidv1 extension about this, but my intuition says cid::Version::V0
is merely a formatting option for std::fmt::Display
.
What are the situations where V0
cid should not be equal to the same multihash and cid::Codec::DagProtobuf
as V1
? I guess the only valid use case is to separate on the Cid::version(&self)
but for all content identification or addressing purposes these should be equal even if they have different string representations?
Given you need an IPFS server to generate checksums of files currently, it seems like a cidsum
or ipldsum
binary to quickly generate the CID hash for files would be a handy tool to provide from rust-cid.
There should be a test suite including spec compliance (at least the examples from the spec should be tested verbatim) and some roundtrip tests using quickcheck.
It would be great if an arbitrary was available for other people to use in their tests, but that is a nice to have.
related to #81
parity-scale-codec has released new version 2.0.0, and now it's 2.1.1. It has more features for this crate, and some part is not compatible with 1.3.5 version. In our project, we need this new version parity-scale-codec, please update it.
Thanks.
The example in the README is for version 0.10.1 and will not compile with 0.11.0
I am using the cid ="0.4.0"
and the code snippet you provided in the README file and it fails with this message.
Following code gets me the Cid as a string though (note rust beginnger):
let h = Sha2_256::digest(b"beep boop");
let cid = Cid::new(Version::V1, Codec::DagProtobuf, h);
match cid.clone() {
Ok(cid) => println!("cid is {}", cid),
Err(e) => println!("Error: {}", e),
}
when going to the github-pages
env and viewing the deployments this URL doesn't work https://multiformats.github.io/rust-cid/
If you traverse a file containing CIDs, you might want to keep track of your current position. Currently read_bytes()
only returns the new CID. As varints are involved, you cannot tell by how many bytes the reader was advanced. Hence I propose extending read_bytes()
to also return the number of byte read.
Instead of needing to add yet another crate to Cargo in order to use https://docs.rs/multibase/0.8.0/multibase/enum.Base.html for example, it would be great to have them re-exported in the crate.
It uses Vec::with_capacity(16)
which is too short. We also need a function that takes a &mut [u8]
to encode it into an existing buffer.
The current error handling provides poor reporting. For example any error within Multihash or Multibase just lead to an Error::ParsingError
without much further information.
I propose using https://crates.io/crates/thiserror, so that we can nest the actual underlying error.
Which is now the default for ipfs. E.g.
$ echo 1 | ipfs dag put
bafyreihtx752fmf3zafbys5dtr4jxohb53yi3qtzfzf6wd5274jwtn5agu
let cid = Cid::from_str("bafyreihtx752fmf3zafbys5dtr4jxohb53yi3qtzfzf6wd5274jwtn5agu");
produces Err(UnknownCodec)
Codec went away, but the sample code in readme.md still documents it.
It appears you need access to DAG_PB / SHA2_256 for Cid, but it is now inaccessible in cid.rs to consumers.
Currently the feature definitions are not ideal. E.g. does the std
feature define serde/std
which leads to serde
being a dependency, even if the serde-codec
feature is not enabled. The same is true for other dependencies. As features are additive, it actually shouldn't be needed to explicitly set serde/std
and the std
feature would enable it anyway.
Besides that it would make sense to clean the whole things up and make use of the dep:
and '?` syntax. We are already on a minimum supported Rust version of 1.60, so we can use that syntax.
The serde-codec
feature could then also be renamed to just serde
, following rust-multihash: https://github.com/multiformats/rust-multihash/blob/452a933396adcd5915c53563d5017df76ae3ec26/Cargo.toml#L24-L25
<li>The three open source projects GitHub members have most contributed ๐ฉโ๐ป to are:
Originally posted by @matthiaswenz in github/.github#105 (comment)
There is an optimization in multiformats/rust-multihash
that stores the hash inline if it is is small. For rust-cid to benefit from this it would have to use a Multihash as storage in Cid instead of a Vec...
Hello, your most recent release (0.8.4) has bumped the crate's Minimum Supported Rust Version to 1.59 by making use of const generic defaults in serde's support.
This can be problematic in some instances, or confusing, especially since the release is a patch release so cargo upgrades it automatically by default...
I recommend you specify the MSRV in the readme and add a check in CI so any MSRV breaking changes are clearer.
Some project consider MSRV bumps to be minor updates, so users can bump the minor release when they are ready to upgrade
The debug instance of Cid currently produces this:
Cid { version: V1, codec: 113, hash: Multihash { code: 18, size: 32, digest: [32, 32, 27, 46, 22, 182, 139, 47, 188, 18, 109, 13, 26, 113, 233, 236, 125, 19, 124, 192, 214, 238, 151, 47, 200, 231, 146, 216, 111, 223, 23, 7, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] } }
Unless I want to debug the cid crate itself, this is almost never what I want.
Maybe it would be best to offer two alternative debug formats, one (compact, default) just does the same as display and dumps the string representation. The other produces the above (except maybe a hex string instead of the digest data).
once const generics are supported a block of the ipld amt should be all in one struct:
struct Amt<const W: u32, T: DagCbor> {
height: u32
width: u32,
data: [CidOrT<T>; W],
}
The max encoded size of a block should be checkable to be lower than MAX_BLOCK_SIZE
.
I'm working on a CLI tool that needs to read CIDs in JSONs but the current implementation does not support visit_str
, though Cid::try_from
already supports reading from &str
.
I don't understand why CID isn't compatible with Serde's data model, the reason isn't documented either:
Lines 1 to 6 in dae650c
I tried my hand at doing something about it since the Cid::try_from
+ visit_str
are right at hand but failed miserably, getting stumped by the MainEntryVisitor
.
Lines 83 to 105 in dae650c
I'm keen on discussing more and implementing a solution.
Currently, the CID Display
implementation depends on the alloc crate/feature. Fix multiformats/rust-multibase#33 then remove this requirement.
Cid
's new_v0
constructor is const fn
, but it can't actually be used as such:
use cid::{multihash::Multihash, Cid};
const fn multihash() -> Multihash {
panic!("https://github.com/multiformats/rust-multihash/issues/330")
}
const _: Cid = match Cid::new_v0(multihash()) {
Ok(o) => o,
Err(_) => panic!(),
};
Fails to compile:
error[E0493]: destructor of `Result<CidGeneric<64>, cid::Error>` cannot be evaluated at compile-time
--> src/lib.rs:7:22
|
7 | const _: Cid = match Cid::new_v0(multihash()) {
| ^^^^^^^^^^^^^^^^^^^^^^^^ the destructor for this type cannot be evaluated in constants
...
10 | };
| - value is dropped here
For more information about this error, try `rustc --explain E0493`.
This is presumably because the top level error type is non-copy/contains an io::Error which cannot be const
Line 32 in 1df4e3f
Here is the root cause of the error in the constructor
Lines 78 to 87 in 1df4e3f
Fix is either:
const
from the new_v0
constructor. This would propogate to new
, and is a breaking changenew_v0
return a new, const-compatible InvalidCidV0Multihash
error in its constructor, and propogate that to new
. This is also a breaking change. This is my preference, and I'm happy to create a PR for such a change.note that this crate is on multihash 0.18.1
, but latest is 0.19.0
- is there a reason for the lag? โฉ
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.