Code Monkey home page Code Monkey logo

git-remote-ipfs's People

Contributors

cbluth avatar cryptix avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

git-remote-ipfs's Issues

Push

Now that clone/fetch is working, push is the next step.

For now, I just plan to add commits to a remote, and print the new root hash of that repo. Updating IPNS entries will come later.

I'm also not sure if I can update the local path of the remote that was used after a push. Might need a 'git remote rm ipfs $oldHash && git remote add ipfs $newHash' dance until ipns is added... (dislike..)

For the mechanics: When running git push with an ipfs origin, git asks the helper for a list of remote refs (just like with fetch, but with the for-push argument, not sure what that is about)

>list for-push
<3f7ba9486ffcaa277b28598c7bfc061bffcfb10e HEAD
<3f7ba9486ffcaa277b28598c7bfc061bffcfb10e refs/heads/master 

Next, git will send one ore more push <src>:<dst> commands.

>push refs/heads/master:refs/heads/master

At that point I already know the hash of that remote ref (I just received it myself). I think the idea is, to walk down the history of the local repo (GIT_DIR) and find that remote ref. Everything up from that point, are the commits we need to push.

Open questions:

  • Can I just add those commit objects (and the trees and blobs that it depends on, of course) into the remote .git/object/ab/cdefs.. or do I need to update some index? Refs at least, I guess.
  • Should I pack them?
  • Look into refspec as they define how to update which reference (branches)
  • updating remote paths?
  • Have a special case like ipfs://new to publish new repos?
  • deleting remote branches
  • force updating

use full path after protocol

we dont want people using ipfs://<hash> -- we want people using /ipfs/<hash>. of course, this doesn't work with how git helpers work, but see the discussion over at ipfs/kubo#1678

Since we have not yet settled on an approach there o/ (i will update here when we do), maybe for now switch to using ipfs://ipfs/<hash> ?

fatal: loose object ... (stored in /tmp/git-remote-ipfs-test-.../.git/objects/aa/bbb...) is corrupt

seeing this from time to time:

/tmp/go-stress236604264
--- FAIL: TestClone_unpacked (3.16s)
    fetch_test.go:91: tmpDir created: /tmp/git-remote-ipfs-test-e8e816fd1bfb81577674856923dc86c17fdc48ac
    fetch_test.go:66: 'git clone ipfs://QmYFpZJs82hLTyEpwkzVpaXGUabVVwiT8yrd6TK81XnoGB/unpackedTest /tmp/git-remote-ipfs-test-e8e816fd1bfb81577674856923dc86c17fdc48ac':
        Cloning into '/tmp/git-remote-ipfs-test-e8e816fd1bfb81577674856923dc86c17fdc48ac'...
        error: object file /tmp/git-remote-ipfs-test-e8e816fd1bfb81577674856923dc86c17fdc48ac/.git/objects/c6/94d28ad470f3a1d8e7bc8a0baf092f04773a9c is empty
        fatal: loose object c694d28ad470f3a1d8e7bc8a0baf092f04773a9c (stored in /tmp/git-remote-ipfs-test-e8e816fd1bfb81577674856923dc86c17fdc48ac/.git/objects/c6/94d28ad470f3a1d8e7bc8a0baf092f04773a9c) is corrupt
        fatal: remote did not send all necessary objects
    fetch_test.go:77: exit status 128
FAIL

cannot use c (type cid.Cid)

Got an error while trying to install

$ go get -u github.com/cryptix/git-remote-ipfs
# github.com/cryptix/git-remote-ipfs/internal/path
/home/username/go/src/github.com/cryptix/git-remote-ipfs/internal/path/path.go:92:16: cannot use c (type cid.Cid) as type *cid.Cid in argument to FromCid

With

$ go version
go version go1.11 linux/amd64
$ uname -orm
4.14.72-1-lts x86_64 GNU/Linux

Compile under Go 1.6

With

C:\devel\go> bin\go version
go version go1.6 windows/amd64

compilation fails:

C:\devel\go> go get -u github.com/cryptix/git-remote-ipfs
package code.google.com/p/go-uuid/uuid: unable to detect version control system for code.google.com/ path

On Mac with Go 1.5.3 this merely gives me a warning.

get basic cloning working

tl;dr: the initial git stdio interface is there but i need some feedback on git internals

  • do a nasty fetch of a full bare repo and run 'git ls-remote' inside to answer the 'list' command
  • how to anser 'fetch $sha1 $refname' commands? see 4) revison 1 of this etherpad for an inital idea
09:55 < cryptix> jbenet: i fell into the trap of starting to implement a protocol helper for git yesterday. but could use some insight from somebdoy more familiar with the 
                 interface.. the one man document i found was only helpful so far
09:55 < cryptix> https://github.com/cryptix/git-remote-ipfs/blob/master/main.go
09:55 < cryptix> ie git clone ipfs://$somepath
09:56 < cryptix> maybe chris or whyrusleeping can shed some light on my questions
09:56 <@jbenet> cryptix: very cool. im not super familiar but maybe i can take a look?
09:57 < cryptix> sure. so far i only delt with the 'fetch' capability (thats all gittorrent implements), i can fetch a bare repo under ipfs://$path already and answer to 
                 the 'list' command, wich wants a list of hashes and refs and than requests 'fetch $sha1 HEAD' (basically all refs afterwards)
09:58 < cryptix> but i'm not sure on the format it expects than
09:58 < cryptix> like, how to answer that 'fetch $hash $ref' command is beyond me from the docs
10:00 <@jbenet> cryptix: links pls?
10:00 < cryptix> i basically wanted a 'post-hook' that publishes commits to ipfs since dtnconf. whyrusleeping git-ipfs-rehost already does a lot of that
10:01 < cryptix> jbenet: sorry, its in the main.go too
10:01 < cryptix> https://git-scm.com/docs/gitremote-helpers
10:01 <@jbenet> also maybe we want ipfs://ipfs/<hash> and ipfs://ipns/<hash> -- otherwise we'd need two protocol handlers
10:02 <@jbenet> though not sure, i'm as annoyed by "ipfs://ipfs" as anyone :)
10:02 < cryptix> yea.. once the basics are done, having two helpers for ipfs and ipns is pretty trivial
10:03 < cryptix> its just that it wants $proto://$path or $proto::$path which really annys me but meh..
10:05 < cryptix> btw git clone http://$gateway/$path from git-ipfs-rehost already works - i just wanted nicer integration. someday you could have a 'git push' capable ipns 
                 remote for instance
10:06 <@jbenet> yeah exactly, i think this is definitely wanted
10:06 <@jbenet> im looking at the git source to find remote impls
10:06 <@jbenet> so far i've found: https://github.com/git/git/tree/77bd3ea9f54f1584147b594abc04c26ca516d987/contrib/persistent-https
10:07 < cryptix> there is git-remote-testgit but it only explains the stdin/out interface how git and the helper are talking
10:07 <@jbenet> the git source assumes so much is installed-- there are python scripts
10:07 < cryptix> https://github.com/git/git/blob/master/git-remote-testgit.sh
10:08 <@jbenet> https://github.com/felipec/git-remote-hg 
10:08 < cryptix> than there is https://github.com/cjb/GitTorrent/blob/master/git-remote-gittorrent ofc
10:09 <@jbenet> yep -- may want to ask cjb when he's online. (nyc, so should be around in a few hours)
10:09 < cryptix> afaict it directly fetches the $sha1 hashes from the 'fetch $sha1 $refName' command requests
10:10 < cryptix> maybe we could have another git-ipfs-rehost that unpacks the commits in a way that we can directly request those sha1 hahses from ipfs but yea.. thats 
                 where my git understanding gets muddy :)
10:10 <@jbenet> finding "fetch" in https://github.com/felipec/git-remote-hg/blob/master/git-remote-hg isnt very promising
10:11 <@jbenet> oh yeah that's trickt
10:11 <@jbenet> because we re-wrap all our objects with our merkledag format
10:11 <@jbenet> so the graph changes a bit
10:11 <@jbenet> what we could do is fetch the objects and look into them
10:12 <@jbenet> or have an "import git graph" thing that creates objects where link _names_ are the git sha1 hashes
10:12 <@jbenet> so we'd have mappings like $sha1 : $ipfsmultihash
10:12 < cryptix> yup that sounds promising
10:13 <@jbenet> the git-ipfs-rehost is a great hack because it leverages the fact that git repos + the http transport use unix files :) -- but this protocol is more lower 
                level
10:14 <@jbenet> ok so-- can you help me trace a full request here? like what does git ask from our handler?
10:14 < cryptix> yup - i guess you could also get away with just dumping the bare repo in the .git dir but i think its better to follow it's rules :)
10:15 <@jbenet> (may be useful to write it out as a set of pseudocode function calls in a gist)
10:15 < cryptix> jbenet: sure - lets be ipfs agonistic for a second
10:15 <@jbenet> yep
10:17 < cryptix> jbenet: https://etherpad.mozilla.org/nFW7hausSF
10:39 <@jbenet> hey chriscool if you're around may want to take a look at this etherpad \o

cc @jbenet @whyrusleeping @chriscool @cjb

Add FUNDING.yaml

Sponsorship options are missing from this repository with over a hundred stars.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.