monadic-xyz / ipfs Goto Github PK
View Code? Open in Web Editor NEWHaskell libraries for interacting with IPFS
Home Page: https://ipfs.io
License: BSD 3-Clause "New" or "Revised" License
Haskell libraries for interacting with IPFS
Home Page: https://ipfs.io
License: BSD 3-Clause "New" or "Revised" License
Would be nice to get some interactions (create git repo, add commits, push, clone) going programmatically. Not sure how to run ipfs daemon on CI.
See oscoin/oscoin#505 . We now have a bunch of tests in oscoin
that belongs to multibase
and multihash-cryptonite
, really:
tests :: TestTree
tests = testGroup "Multihash"
[ testGroup "Examples from https://github.com/multiformats/multihash"
[ testGroup "sha1(\"multihash\")"
[ testCase "Base16" $
example BaseN.Base16 C.SHA1 "multihash"
"111488c2f11fb2ce392acb5b2986e640211c4690073e"
, testCase "Base58" $
example BaseN.Base58btc C.SHA1 "multihash"
"5dsgvJGnvAfiR3K6HCBc4hcokSfmjj"
, testCase "Base64" $
example BaseN.Base64 C.SHA1 "multihash"
"ERSIwvEfss45KstbKYbmQCEcRpAHPg=="
]
, testGroup "sha2-256(\"multihash\")"
[ testCase "Base16" $
example BaseN.Base16 C.SHA256 "multihash"
"12209cbc07c3f991725836a3aa2a581ca2029198aa420b9d99bc0e131d9f3e2cbe47"
, testCase "Base58" $
example BaseN.Base58btc C.SHA256 "multihash"
"QmYtUc4iTCbbfVSDNKvtQqrfyezPPnFvE33wFmutw9PBBk"
, testCase "Base64" $
example BaseN.Base64 C.SHA256 "multihash"
"EiCcvAfD+ZFyWDajqipYHKICkZiqQgudmbwOEx2fPiy+Rw=="
]
]
, testGroup "Roundtrip"
[ testProperty "Base16: decode . encode = id" . property $ do
bs <- forAll genBytes
propRoundtrip BaseN.Base16 C.Blake2b_256 bs
, testProperty "Base58: decode . encode = id" . property $ do
bs <- forAll genBytes
propRoundtrip BaseN.Base58btc C.Blake2b_256 bs
, testProperty "Base64: decode . encode = id" . property $ do
bs <- forAll genBytes
propRoundtrip BaseN.Base64 C.Blake2b_256 bs
]
]
propRoundtrip
:: ( Multihashable a
, DecodeBase b
)
=> Base b
-> a
-> ByteString
-> PropertyT IO ()
propRoundtrip base algo bs =
let digest = C.hashWith algo bs
enc = encodeAtBase base . Multihash.fromDigest
dec = decodeAtBase base . BaseN.encodedBytes
in (dec . enc) digest === Right digest
example
:: Multihashable a
=> BaseN.Base b
-> a
-> ByteString
-> ByteString
-> Assertion
example base algo input expected =
BaseN.encodedBytes (multihash base algo input) @?= expected
--------------------------------------------------------------------------------
genBytes :: MonadGen m => m ByteString
genBytes = Gen.utf8 (Range.constantFrom 8 8 512) Gen.unicodeAll
multihash
:: Multihashable a
=> BaseN.Base b
-> a
-> ByteString
-> BaseN.AtBase b
multihash base algo bs =
encodeAtBase base $ Multihash.multihash algo bs
When I follow the IPNS example pushing results in an error
$ repoid=$(ipfs key gen --type=ed25519 myrepo)
$ git remote add ipns ipfs://ipns/$repoid
$ git push ipns
Resolving IPNS name 12D3KooWDzhxCXPWYet1NwKmWQomkqDVypV3VWED3b2zmMznNsYd
git-remote-ipfs: FailureResponse (Response {responseStatusCode = Status {statusCode = 500,
statusMessage = "Internal Server Error"}, responseHeaders = fromList [("Access-Control-Allow-
Headers","X-Stream-Output, X-Chunked-Output, X-Content-Length"),("Access-Control-Expose-
Headers","X-Stream-Output, X-Chunked-Output, X-Content-Length"),("Content-Type","application/json"),
("Server","go-ipfs/0.4.18"),("Trailer","X-Stream-Error"),("Vary","Origin"),("Date","Fri, 25 Jan 2019 14:05:10
GMT"),("Transfer-Encoding","chunked")], responseHttpVersion = HTTP/1.1, responseBody = "{\"Message
\":\"could not resolve name\",\"Code\":0,\"Type\":\"error\"}\n"})
The problem is that the IPNS record for the key was never set.
A workaround is to set the the IPNS record to some dummy entry
ipfs name publish --key myrepo $(echo | bin/rad-ipfs add -q)
There are some opportunities for concurrency (mainly network IO with the ipfs daemon). Would require a bit of restructuring of the application, though
When installing with nix:
setupCompileFlags: -package-db=/build/setup-package.conf.d -j4 +RTS -A64M -RTS -threaded -rtsopts
[1 of 1] Compiling Main ( Setup.hs, /build/Main.o )
Setup.hs:3:1: error:
Could not find module ‘Distribution.Extra.Doctest’
Perhaps you meant Distribution.Simple.Doctest (from Cabal-3.2.1.0)
Use -v (or `:set -v` in ghci) to see a list of the files searched for.
|
3 | import Distribution.Extra.Doctest (defaultMainWithDoctests)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Output some meaningful and/or entertaining progress during push/fetch.
It would be nice to have some semi-scientific experiments which could help answer the following questions:
Even with a medium sized repo like oscoin/radicle
a push eats up all of my 16GB of RAM.
Not quite sure this codepath actually works.
ApiV0PinLs
has arg
as required argument, while https://docs.ipfs.io/reference/api/http/#api-v0-pin-ls states that it is optional. By enforcing the arg
parameter one is unable to list all pinned objects of a node. Would it be possible to rectify this? Is editing the file and sending a PR an option or would those changes be lost because you run the auto-gen?
As expected, this creates too much memory fragmentation. Trouble is, we will have to re-implement all BaseN-encoding on top of an alternative representation (see also #7), as otherwise we'll end up incurring a memcpy
in almost every place multihash
es are used.
Right now, I cannot write a function that uses Multibase.encode . encodeAtBase base
where it takes base
as an argument because I cannot write the ToCode
class restriction, since that class is not exported.
Should respect the environment variables IPFS_API_HOST
and IPFS_API_PORT
Right now uses a bouquet of baseN packages, which also do not support all the variants demanded by multibase
. It might be worth inlining all of those, and add efficient implementations of the variants (eg. upper/lower case alphabets).
This stemmed from a conversation with @kim . Currently in oscoin
we have a BaseN.hs
module which is quite pleasant as it uses data kinds to provide type safe base conversions, something that (from what I could see by skimming this library) it's a bit more verbose in the current implementation.
Ideally we should pull the oscoin
code into this repo and finally switch oscoin
to be using this library.
Have I synthesised things correctly?
I've been installing from git, but would be nice to use version from hackage for CI, etc.
what information would a proficient haskell user need in order to install the package and use it within their project.
describe any important design decisions made within this repo
link to relevant IPFS documentation or tutorials
When cloning an existing project, I get the following error:
testing ; rad project checkout 12D3KooWSxYcebkWUAFzoinjtVjFviUBNc5gTcPwJMQdS3APhsac
Test
Initialized empty Git repository in /Users/cory/Code/testing/Test/.git/
Resolving IPNS name 12D3KooWSNQQiHkDatpsAicSVM2MuAGQkb9JrdjcrfYGAEQL1yCW
Unexpected link type: 0 -- Object (fromList [("Objects",Array [Object (fromList [("Hash",String "QmdH1b5Rcdqxn64Qe3QzdPjrvjMhLzZjN8DaQeaF4APBM5/refs/heads"),("Links",Array [Object (fromList [("Hash",String "z8mWaFMWEuUL1irweNi8qzKtjb9qvbkcA"),("Size",Number 0.0),("Name",String "master"),("Type",Number 0.0)])])])])]) (src/Network/IPFS/Git/RemoteHelper/Client.hs:136)
The repo is currently not available. Cloning "Test" from "ipfs://ipns/12D3KooWSNQQiHkDatpsAicSVM2MuAGQkb9JrdjcrfYGAEQL1yCW" failed.
This may be because you have no internet connection, or the IPNS link is stale, or because no online node with the data could be found.
This error comes up when i am hosting the project from another computer on the same LAN, but also i get this same error when trying to clone in a different folder on the same device where the repo is being hosted from..
Will file as an issue in the radicle project as well for visibility, but it seems like this is specific to the git-ipfs integration.
The dubious Base1
can be skipped, I guess.
Seems at least the base32 bytestring library is broken under GHC9
Seems useful to be able to tag a Multibase
value by its Base
, to be able to enforce a certain encoding in application code.
I was looking to manipulate and serialize/deserialize IPLD objects, and I came across a few of the libraries listed here in the process.
It looks like the current set of libraries does not include the above, so I figured I'd make an issue in case you thought it was in scope.
I may end up making the libraries myself in my free time, likely on top of some of your libraries, in which case I'll open source them and we can discuss if it's worth merging them into your repo later.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.