cryptix / git-remote-ipfs Goto Github PK
View Code? Open in Web Editor NEWgit transport helper for ipfs
License: MIT License
git transport helper for ipfs
License: MIT License
Would be nice to work out how to get the ipfs shell api from an embedded node.
Would also benefit @noffle's ipget and others.
Now that clone/fetch is working, push is the next step.
For now, I just plan to add commits to a remote, and print the new root hash of that repo. Updating IPNS entries will come later.
I'm also not sure if I can update the local path of the remote that was used after a push. Might need a 'git remote rm ipfs $oldHash && git remote add ipfs $newHash' dance until ipns is added... (dislike..)
For the mechanics: When running git push
with an ipfs origin, git asks the helper for a list of remote refs (just like with fetch, but with the for-push argument, not sure what that is about)
>list for-push
<3f7ba9486ffcaa277b28598c7bfc061bffcfb10e HEAD
<3f7ba9486ffcaa277b28598c7bfc061bffcfb10e refs/heads/master
Next, git will send one ore more push <src>:<dst>
commands.
>push refs/heads/master:refs/heads/master
At that point I already know the hash of that remote ref (I just received it myself). I think the idea is, to walk down the history of the local repo (GIT_DIR
) and find that remote ref. Everything up from that point, are the commits we need to push.
Open questions:
.git/object/ab/cdefs..
or do I need to update some index? Refs at least, I guess.ipfs://new
to publish new repos?we dont want people using ipfs://<hash>
-- we want people using /ipfs/<hash>
. of course, this doesn't work with how git helpers work, but see the discussion over at ipfs/kubo#1678
Since we have not yet settled on an approach there o/ (i will update here when we do), maybe for now switch to using ipfs://ipfs/<hash>
?
Readme doesnt tell me how i can install this and use it. Would be great to have a nice readme like https://github.com/diasdavid/ipscend
seeing this from time to time:
/tmp/go-stress236604264
--- FAIL: TestClone_unpacked (3.16s)
fetch_test.go:91: tmpDir created: /tmp/git-remote-ipfs-test-e8e816fd1bfb81577674856923dc86c17fdc48ac
fetch_test.go:66: 'git clone ipfs://QmYFpZJs82hLTyEpwkzVpaXGUabVVwiT8yrd6TK81XnoGB/unpackedTest /tmp/git-remote-ipfs-test-e8e816fd1bfb81577674856923dc86c17fdc48ac':
Cloning into '/tmp/git-remote-ipfs-test-e8e816fd1bfb81577674856923dc86c17fdc48ac'...
error: object file /tmp/git-remote-ipfs-test-e8e816fd1bfb81577674856923dc86c17fdc48ac/.git/objects/c6/94d28ad470f3a1d8e7bc8a0baf092f04773a9c is empty
fatal: loose object c694d28ad470f3a1d8e7bc8a0baf092f04773a9c (stored in /tmp/git-remote-ipfs-test-e8e816fd1bfb81577674856923dc86c17fdc48ac/.git/objects/c6/94d28ad470f3a1d8e7bc8a0baf092f04773a9c) is corrupt
fatal: remote did not send all necessary objects
fetch_test.go:77: exit status 128
FAIL
Got an error while trying to install
$ go get -u github.com/cryptix/git-remote-ipfs
# github.com/cryptix/git-remote-ipfs/internal/path
/home/username/go/src/github.com/cryptix/git-remote-ipfs/internal/path/path.go:92:16: cannot use c (type cid.Cid) as type *cid.Cid in argument to FromCid
With
$ go version
go version go1.11 linux/amd64
$ uname -orm
4.14.72-1-lts x86_64 GNU/Linux
With
C:\devel\go> bin\go version
go version go1.6 windows/amd64
compilation fails:
C:\devel\go> go get -u github.com/cryptix/git-remote-ipfs
package code.google.com/p/go-uuid/uuid: unable to detect version control system for code.google.com/ path
On Mac with Go 1.5.3 this merely gives me a warning.
tl;dr: the initial git stdio interface is there but i need some feedback on git internals
09:55 < cryptix> jbenet: i fell into the trap of starting to implement a protocol helper for git yesterday. but could use some insight from somebdoy more familiar with the
interface.. the one man document i found was only helpful so far
09:55 < cryptix> https://github.com/cryptix/git-remote-ipfs/blob/master/main.go
09:55 < cryptix> ie git clone ipfs://$somepath
09:56 < cryptix> maybe chris or whyrusleeping can shed some light on my questions
09:56 <@jbenet> cryptix: very cool. im not super familiar but maybe i can take a look?
09:57 < cryptix> sure. so far i only delt with the 'fetch' capability (thats all gittorrent implements), i can fetch a bare repo under ipfs://$path already and answer to
the 'list' command, wich wants a list of hashes and refs and than requests 'fetch $sha1 HEAD' (basically all refs afterwards)
09:58 < cryptix> but i'm not sure on the format it expects than
09:58 < cryptix> like, how to answer that 'fetch $hash $ref' command is beyond me from the docs
10:00 <@jbenet> cryptix: links pls?
10:00 < cryptix> i basically wanted a 'post-hook' that publishes commits to ipfs since dtnconf. whyrusleeping git-ipfs-rehost already does a lot of that
10:01 < cryptix> jbenet: sorry, its in the main.go too
10:01 < cryptix> https://git-scm.com/docs/gitremote-helpers
10:01 <@jbenet> also maybe we want ipfs://ipfs/<hash> and ipfs://ipns/<hash> -- otherwise we'd need two protocol handlers
10:02 <@jbenet> though not sure, i'm as annoyed by "ipfs://ipfs" as anyone :)
10:02 < cryptix> yea.. once the basics are done, having two helpers for ipfs and ipns is pretty trivial
10:03 < cryptix> its just that it wants $proto://$path or $proto::$path which really annys me but meh..
10:05 < cryptix> btw git clone http://$gateway/$path from git-ipfs-rehost already works - i just wanted nicer integration. someday you could have a 'git push' capable ipns
remote for instance
10:06 <@jbenet> yeah exactly, i think this is definitely wanted
10:06 <@jbenet> im looking at the git source to find remote impls
10:06 <@jbenet> so far i've found: https://github.com/git/git/tree/77bd3ea9f54f1584147b594abc04c26ca516d987/contrib/persistent-https
10:07 < cryptix> there is git-remote-testgit but it only explains the stdin/out interface how git and the helper are talking
10:07 <@jbenet> the git source assumes so much is installed-- there are python scripts
10:07 < cryptix> https://github.com/git/git/blob/master/git-remote-testgit.sh
10:08 <@jbenet> https://github.com/felipec/git-remote-hg
10:08 < cryptix> than there is https://github.com/cjb/GitTorrent/blob/master/git-remote-gittorrent ofc
10:09 <@jbenet> yep -- may want to ask cjb when he's online. (nyc, so should be around in a few hours)
10:09 < cryptix> afaict it directly fetches the $sha1 hashes from the 'fetch $sha1 $refName' command requests
10:10 < cryptix> maybe we could have another git-ipfs-rehost that unpacks the commits in a way that we can directly request those sha1 hahses from ipfs but yea.. thats
where my git understanding gets muddy :)
10:10 <@jbenet> finding "fetch" in https://github.com/felipec/git-remote-hg/blob/master/git-remote-hg isnt very promising
10:11 <@jbenet> oh yeah that's trickt
10:11 <@jbenet> because we re-wrap all our objects with our merkledag format
10:11 <@jbenet> so the graph changes a bit
10:11 <@jbenet> what we could do is fetch the objects and look into them
10:12 <@jbenet> or have an "import git graph" thing that creates objects where link _names_ are the git sha1 hashes
10:12 <@jbenet> so we'd have mappings like $sha1 : $ipfsmultihash
10:12 < cryptix> yup that sounds promising
10:13 <@jbenet> the git-ipfs-rehost is a great hack because it leverages the fact that git repos + the http transport use unix files :) -- but this protocol is more lower
level
10:14 <@jbenet> ok so-- can you help me trace a full request here? like what does git ask from our handler?
10:14 < cryptix> yup - i guess you could also get away with just dumping the bare repo in the .git dir but i think its better to follow it's rules :)
10:15 <@jbenet> (may be useful to write it out as a set of pseudocode function calls in a gist)
10:15 < cryptix> jbenet: sure - lets be ipfs agonistic for a second
10:15 <@jbenet> yep
10:17 < cryptix> jbenet: https://etherpad.mozilla.org/nFW7hausSF
10:39 <@jbenet> hey chriscool if you're around may want to take a look at this etherpad \o
Sponsorship options are missing from this repository with over a hundred stars.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.