apenwarr / git-subtrac Goto Github PK

View Code? Open in Web Editor NEW

378.0 378.0 15.0 79 KB

Keep the content for your git submodules all in one place: the parent repo.

License: Apache License 2.0

Go 100.00%

git-subtrac's People

Contributors

Stargazers

Watchers

Forkers

mparker17 fcantournet brunom riduidel moneytech 4144 rafasc isgasho burdiyan amitmahbubani shamrin frankfanslc iq-scm stool233

git-subtrac's Issues

Installation fails

❯ go version
go version go1.20.2 darwin/arm64
❯ go install github.com/apenwarr/git-subtrac
go: 'go install' requires a version when current directory is not in a module
  Try 'go install github.com/apenwarr/git-subtrac@latest' to install the latest version
❯ go install github.com/apenwarr/git-subtrac@latest
go: github.com/apenwarr/git-subtrac@latest (in github.com/apenwarr/[email protected]):
  The go.mod file for the module providing named packages contains one or
  more replace directives. It must not contain directives that would cause
  it to be interpreted differently than if it were the main module.
❯ go get -v github.com/apenwarr/git-subtrac
go: go.mod file not found in current directory or any parent directory.
  'go get' is no longer supported outside a module.
  To build and install a command, use 'go install' with a version,
  like 'go install example.com/cmd@latest'
  For more information, see https://golang.org/doc/go-get-install-deprecation
  or run 'go help get' or 'go help install'.

I'm not familiar with Go, so I can't tell the problem resides in my environment or my project or git-subtrac, but seems like this software is only available for Go project?

Segfault from `update`

Hey! I've run into a segfault when trying to do git subtrac update:

Scanning branch: refs/heads/master
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x78d56c]

goroutine 1 [running]:
github.com/apenwarr/git-subtrac/subtrac.(*Cache).tracCommit(0xc0000e0a80, 0xc0007eba40, 0x34, 0xc0003aea50, 0xc081097bde, 0xc0003aea50, 0x0)
	/path/to/src/gopath/src/github.com/apenwarr/git-subtrac/subtrac/subtrac.go:229 +0xf6c
github.com/apenwarr/git-subtrac/subtrac.(*Cache).tracTree(0xc0000e0a80, 0xc000135c50, 0x24, 0xc00007a870, 0x6, 0x85fa1e, 0x1)
	/path/to/src/gopath/src/github.com/apenwarr/git-subtrac/subtrac/subtrac.go:547 +0xd78
github.com/apenwarr/git-subtrac/subtrac.(*Cache).tracTree(0xc0000e0a80, 0xc000023c60, 0x1d, 0xc00007a7d0, 0x6, 0x85fa1e, 0x1)
	/path/to/src/gopath/src/github.com/apenwarr/git-subtrac/subtrac/subtrac.go:562 +0x43f
github.com/apenwarr/git-subtrac/subtrac.(*Cache).tracTree(0xc0000e0a80, 0xc000023be0, 0x16, 0xc00007a6e0, 0x3, 0x85fa1e, 0x1)
	/path/to/src/gopath/src/github.com/apenwarr/git-subtrac/subtrac/subtrac.go:562 +0x43f
github.com/apenwarr/git-subtrac/subtrac.(*Cache).tracTree(0xc0000e0a80, 0xc000023b00, 0x12, 0xc00007a5f0, 0x1, 0xc000023b00, 0x12)
	/path/to/src/gopath/src/github.com/apenwarr/git-subtrac/subtrac/subtrac.go:562 +0x43f
github.com/apenwarr/git-subtrac/subtrac.(*Cache).tracCommit(0xc0000e0a80, 0xc000028469, 0x11, 0xc0001720f0, 0xc013dabada, 0xc0001720f0, 0x0)
	/path/to/src/gopath/src/github.com/apenwarr/git-subtrac/subtrac/subtrac.go:217 +0x2e6
github.com/apenwarr/git-subtrac/subtrac.(*Cache).TracByRef(0xc0000e0a80, 0xc000028469, 0x11, 0x1, 0x1, 0xc000028469)
	/path/to/src/gopath/src/github.com/apenwarr/git-subtrac/subtrac/subtrac.go:188 +0x2eb
github.com/apenwarr/git-subtrac/subtrac.(*Cache).UpdateBranchRefs.func1(0xc000035b40, 0xc000035b40, 0x0)
	/path/to/src/gopath/src/github.com/apenwarr/git-subtrac/subtrac/subtrac.go:143 +0x150
github.com/go-git/go-git/plumbing/storer.(*referenceFilteredIter).ForEach(0xc00000fbe0, 0xc00000fc40, 0x0, 0x0)
	/path/to/src/gopath/src/github.com/go-git/go-git/plumbing/storer/reference.go:82 +0xa4
github.com/apenwarr/git-subtrac/subtrac.(*Cache).UpdateBranchRefs(0xc0000e0a80, 0xc00000e080, 0x2)
	/path/to/src/gopath/src/github.com/apenwarr/git-subtrac/subtrac/subtrac.go:137 +0x1a6
main.main()
	/path/to/src/gopath/src/github.com/apenwarr/git-subtrac/git-subtrac.go:77 +0x6eb

I'm a bit lost how to diagnose what went wrong ...

This error started showing up after I tried using --depth on a git submodule add so I suspect it is somehow related to that. I did however completely remove the submodule again and tried recreating it without --depth. This did not fix it, unfortunately :(

Any idea A) what might have gone wrong here and B) how I can fix my repo so it works again?

Is it possible to share folders without the folders being Git repositories themselves?

I have described my situation here. It seems clear that if the "shared folders" were submodules, subtrees, or subrepos, then Repo 1 and Repo 2 can both enjoy possession of them, but if I have 1000s of shared folders, I don't want to make each of them a Git repository, and neither do I want to make a third Git repository just for the common folders. Is there a way to do this in Git?

I also asked this question here: ingydotnet/git-subrepo#573

Tries to reference non-existant submodules

I suspect this is actually a bug elsewhere, since I didn't see your code that could be doing this, but I only took a quick peek in any case.

My dotfiles have a bunch of submodules and I tried to run git-subtrac on them. Here's the output I get:

$ git subtrac -v update
Scanning branch: refs/heads/master
Searching submodules for: refs/heads/master/awesome/charitable@7695f60a9e
git-subtrac: git repo(vim/bundle/sleuth): submodule not initialized (fetch it manually? or try --exclude)

That submodule is not in my master branch:

$ git checkout master ; git submodule
Already on 'master'
 7695f60a9e2623ed42ab8a4581711dcad13bc0e9 awesome/charitable (v1.0-2-g7695f60)
 c0fbaa4c543f8bc315726d811f1c99ae4eba5e78 awesome/vicious (v2.1.1-277-gc0fbaa4)
 a78708f52d7577c129e78b2e9ed606da729628c7 git-hub (github-issue-141-fix/1-187-ga78708f)
 cdd44dcf54dd6900e9699e957ba4b203d3021d1d vim/pack/vanilla/opt/FastFold (5.0-28-gcdd44dc)
 f045452743736d53c7471bfb4698fd48569dc1e0 vim/pack/vanilla/opt/airline/airline (v0.9-187-gf045452)
 b7ec11c93d12ce8f64cfdae363cbab3d2e69dfd4 vim/pack/vanilla/opt/ale (v2.2.0-27-gb7ec11c9)
 f5726c4bbe84a762d5ec62d57af439138a36af76 vim/pack/vanilla/opt/better-whitespace (remotes/origin/HEAD)
 ebc568c3992d9002d1d35b85737dfa0d9ce70d9f vim/pack/vanilla/opt/ctrlp (1.80-61-gebc568c)
 ab7470d4b03bae9880bf2b5cef60fc0fb51b1101 vim/pack/vanilla/opt/dispatch (v1.6-9-gab7470d)
 6b8b5a4cd77159eca30166b00097266f1d8e1faa vim/pack/vanilla/opt/editorconfig (0.7.0-2-g6b8b5a4)
 05d82b87711c6c8b9b7389bfb91c24bc4f62aa87 vim/pack/vanilla/opt/exchange (heads/master)
 acf3cb44003eb758d73034c3a86b23af32652dfb vim/pack/vanilla/opt/fugitive (v2.5-57-gacf3cb4)
 43a13dbbc9aae85338877329ed28c9e4d8488db1 vim/pack/vanilla/opt/fugitive-gitlab (remotes/origin/HEAD)
 8aac968f1a617cc1c346707242af181e6b1f467e vim/pack/vanilla/opt/gitgutter (remotes/origin/HEAD)
 806674b9d799bebda41122ce143c234e961bbc97 vim/pack/vanilla/opt/govim (v0.0.26)
 5b8bd0378758c1d9550d8429bef24b3d6d78b592 vim/pack/vanilla/opt/goyo (1.6.0-30-g5b8bd03)
 102b68348eff0d639ce88c5094dab0fdbe4f7c55 vim/pack/vanilla/opt/lastplace (v1.0.0-85-g102b683)
 79a834a36b0b2e7a6e4c0b236e4927945ab3e95f vim/pack/vanilla/opt/lost (v1.0)
 ea6192dc90eec09575a8f38dba3cf7c5ca9147f8 vim/pack/vanilla/opt/matchmaker (0.2.1-8-gea6192d)
 dd9289af03abafa76b28c503e20747ff7d7d89e5 vim/pack/vanilla/opt/multiple-cursors (v2.2-208-gdd9289a)
 06da921608b971fb47603671bcafdb2843992eb3 vim/pack/vanilla/opt/pathogen (v2.4-16-g06da921)
 21d0a0d795336acf8a9306da35f379c32cfc5e08 vim/pack/vanilla/opt/perl (vim-7.4.0000-397-g21d0a0d)
 e8227e3e9bc799245b7df9045ab246266b287725 vim/pack/vanilla/opt/quick-scope (v1.1.1-13-ge8227e3)
 2f61753363568b1f819e659e152598e21bd4c5bf vim/pack/vanilla/opt/regedit (heads/master)
 43d2678fa59d068c815d8298331c195e850ff5a7 vim/pack/vanilla/opt/repeat (v1.2-3-g43d2678)
 00e1e7fcdbc6d753e0bc8043e0d2546fa81bf367 vim/pack/vanilla/opt/tabular (1.0.0-1-g00e1e7f)
 0c826f1eb31f68648fa82cb25ed06031dc846a44 vim/pack/vanilla/opt/terminus (1.3-6-g0c826f1)
 d1920ce9353dc5a92c4cf26b3ad8ae03e023bb98 vim/pack/vanilla/opt/textobj-between (v0.1.0-6-gd1920ce)
 64a856c9dff3425ed8a863b9ec0a21dbaee6fb3a vim/pack/vanilla/opt/textobj-entire (0.0.4)
 41fcdacaa5c763063a4b4f715cc22962075f22d6 vim/pack/vanilla/opt/textobj-underscore (heads/master)
 e231b65797b5765b3ee862d71077e9bd56f3ca3e vim/pack/vanilla/opt/textobj-user (0.7.5)
 37259722f45996733fd309add61385a4ad88bdb9 vim/pack/vanilla/opt/visual-star-search (0.2-7-g3725972)
 528a59f26d12278698bb946f8fb82a63711eec21 vim/pack/vanilla/start/colors-solarized (heads/master)
 141d9d32a9fb58fe474fcc89cd7221eb2dd57b3a vim/pack/vanilla/start/commentary (v1.3-14-g141d9d3)
 10da325fb032a1acfa9222d273459f53bad30ba4 vim/pack/vanilla/start/eunuch (v1.2-1-g10da325)
 dcbf3f9863fae143b952db0bd6bacf6180c4853b vim/pack/vanilla/start/grepper (v1.2-253-gdcbf3f9)
 95a576210dc4408a4804a0a62a9eae90d701026b vim/pack/vanilla/start/obsession (remotes/origin/HEAD)
 cf747d1dcde64944c209486d1ffaf5c45a147674 vim/pack/vanilla/start/projectionist (v1.0-42-gcf747d1)
 9b74a5faf5f8f90f1729ef054af99072ba061985 vim/pack/vanilla/start/solarized8 (v1.1.0-2-g9b74a5f)
 597068870b8f093a8b2d11536c62ff31222ee8d0 vim/pack/vanilla/start/surround (v2.1-14-g5970688)
 d6325994b3c16ce36fd494c47dae4dab8d21a3da vim/pack/vanilla/start/unimpaired (v2.0-1-gd632599)
 c38ea2195a43747aedf0bb4b7eb5aa8870260296 vim/pack/vanilla/start/vinegar (v1.0-8-gc38ea21)

Am I missing something silly? I will work around it with exclude, probably, but I don't see why it's even mentioning this.

git-subtrac runs into `git-subtrac: git status` errors all the time

I'm having issues getting git-subtrac to work with my repo https://github.com/telotortium/emacs.d, which uses several submodules. After running git clone https://github.com/telotortium/emacs.d and git submodule update --init --recursive inside the repo, I can run git subtrac --auto-exclude update. I then want to follow the advice of the README and set url = . for all the submodules in .gitmodules. I do this, and then attempt to run git fetch --all in all the submodules. When I go back up to the main repo, I then encounter errors like this:

git subtrac --verbose --auto-exclude update
Scanning branch: refs/heads/master
  add 160b1c4552 refs/heads/master/cache/
Searching submodules for: refs/heads/master/lisp/git-auto-commit-mode@f89ed36c38
git-subtrac: git status(lisp/org-gtasks.git): invalid checksum (fetch it manually? or try --exclude)

The specific submodule in the error message changes when I run the command repeatedly.

How to update submodules

After running git-subtract --auto-exclude update I was trying to update my submodules to their latest commits. The Readme states that all the git submodule commands keep working, so I tried git submodule foreach git pull origin master which pulled from the super project’s master branch and broke everything. What would be the correct command to update the submodules?

format string error

tryFetchFromSubmodules has a format string error:

	for i := range repos {
		subpath := paths[i]
		subr := repos[i]
		c.infof("   path=%q subr=%q\n", subpath, subr)

Outputs:

Searching submodules for: decbde0a79d6462b8a9c009fddc24d5c40dd913b/oss@25b021388b 25b021388b83c80d1c717479ba9694e713af989d
   path="oss" subr=&{%!q(*filesystem.Storage=&{0xc000adf960 0xc000afea00 {{false false 0} 0xc000d84960 0xc000afea00 map[] [] 0 map[]} {0xc000afea00} {0xc000afea00} {0xc000afea00} {0xc000afea00} {0xc000afea00}}) map[] %!q(*chroot.ChrootHelper=&{0xc00000f740 /home/bradfitz/src/tailscale.io/oss})}
  ...found! in oss

Error on git subtrac update: index out of range error

This happened on a fresh git repo with just one commit.

Go version: go version go1.13.8 linux/amd64

Error message:

Scanning branch: refs/heads/master
Searching submodules for: refs/heads/master/audit-trigger@701646cc2c 70164
6cc2cb937c525cd939ad3329cbd4eb95e0c
panic: runtime error: index out of range [0] with length 0

Contents of the .gitmodules file:

───────┬──────────────────────────────────────────────────────────────────
       │ File: .gitmodules
───────┼──────────────────────────────────────────────────────────────────
   1   │ [submodule "audit-trigger"]
   2   │     path = audit-trigger
   3   │     url = https://github.com/hasura/audit-trigger

subtrac --auto-exclude panics when trying to resolve deleted submodules

this is a follow-up of #2; I added a commit to ccmtaylor/subtrac-test to remove the submodule.

git-subtrac is unable to resolve submodules that have been removed in the current branch. Not sure if this is expected; maybe git-subtrac needs to look at a previous version of .gitignore?

$ git clone https://github.com/ccmtaylor/subtrac-test --recurse-submodules
Cloning into 'subtrac-test'...
remote: Enumerating objects: 7, done.
remote: Counting objects: 100% (7/7), done.
remote: Compressing objects: 100% (4/4), done.
remote: Total 7 (delta 0), reused 7 (delta 0), pack-reused 0
Unpacking objects: 100% (7/7), done.

$ cd subtrac-test

$ git subtrac -v update
Scanning branch: refs/heads/master
  add 4f0f106d83 refs/heads/master/
Searching submodules for: refs/heads/master~1/cookbooks/apache2@7b9de411db
git-subtrac: refs/heads/master~1/cookbooks/apache2@7b9de411db: 7b9de411dbefb628f8ac779b971da96d915ab86c not found. (fetch it manually? or try --exclude)

However, when I run git-subtrac with the --auto-exclude flag, it panics with a SIGSEGV:

$ git subtrac -v --auto-exclude update
Scanning branch: refs/heads/master
  add 4f0f106d83 refs/heads/master/
Searching submodules for: refs/heads/master~1/cookbooks/apache2@7b9de411db
Excluding 7b9de411dbefb628f8ac779b971da96d915ab86c
  add fbdc2a594f refs/heads/master~1/cookbooks/
  add 8d1c14bc25 refs/heads/master~1/
  add e17b5a0633 refs/heads/master~1
  add c89b104d9a refs/heads/master
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x783dc8]

goroutine 1 [running]:
main.(*Cache).UpdateBranchRefs(0xc000021a40, 0x1, 0xc000093f80)
	/home/christophertaylor/src/github.com/apenwarr/git-subtrac/subtrac.go:155 +0x278
main.main()

Readme installation instructions (go install) don't work

Windows 10 x64. I just installed Go through the .msi installer.

This project's readme tells me to run go install github.com/apenwarr/git-subtrac. It doesn't work.

can't load package: package github.com/apenwarr/git-subtrac: cannot find package "github.com/apenwarr/git-subtrac" in any of:
c:\go\src\github.com\apenwarr\git-subtrac (from $GOROOT)
C:\Users\nyanpasu\go\src\github.com\apenwarr\git-subtrac (from $GOPATH)

I have to run go get github.com/apenwarr/git-subtrac first, which takes several minutes and produces no terminal output whatsoever. Afterwards, go install github.com/apenwarr/git-subtrac is instant. I don't know if it's necessary or not.

Now running git-subtrac outputs git-subtrac: git: 0xc0000e65c0: repository does not exist outside of a repo, and provides a help message within a repository.

Improve README

Currently, README seems quite clear ... for people knowing the nuts and bolts of git.
But for not-so-good developers (like me), things seems a little ... opaque.

Can you add an example of how subtrac transforms a repository (typically, I would like to see how the submodules are transformed into valid commits)? I would also like to see how to add new submodules to a project where subtrac is already used.

subtrac doesn't handle submodules in submodules

I'm not entirely sure, but I believe there is an issue with handling submodules that reference submodules themselves. I've setup ccmtaylor/subtrac-test as a reproduction example:

$ git clone https://github.com/ccmtaylor/subtrac-test --recurse-submodules
Cloning into 'subtrac-test'...
remote: Enumerating objects: 4, done.
remote: Counting objects: 100% (4/4), done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 4 (delta 0), reused 4 (delta 0), pack-reused 0
Unpacking objects: 100% (4/4), done.
Submodule 'cookbooks/apache2' ([email protected]:sous-chefs/apache2) registered for path 'cookbooks/apache2'
Cloning into '/home/christophertaylor/src/github.com/ccmtaylor/subtrac-test/cookbooks/apache2'...
remote: Enumerating objects: 137, done.        
remote: Counting objects: 100% (137/137), done.        
remote: Compressing objects: 100% (96/96), done.        
remote: Total 10994 (delta 51), reused 82 (delta 23), pack-reused 10857        
Receiving objects: 100% (10994/10994), 2.02 MiB | 4.06 MiB/s, done.
Resolving deltas: 100% (6989/6989), done.
Submodule path 'cookbooks/apache2': checked out '7b9de411dbefb628f8ac779b971da96d915ab86c'
Submodule 'foodcritic/sc' (https://github.com/sous-chefs/sc-foodcritic-rules.git) registered for path 'cookbooks/apache2/foodcritic/sc'
Cloning into '/home/christophertaylor/src/github.com/ccmtaylor/subtrac-test/cookbooks/apache2/foodcritic/sc'...
remote: Enumerating objects: 213, done.        
remote: Total 213 (delta 0), reused 0 (delta 0), pack-reused 213        
Receiving objects: 100% (213/213), 137.58 KiB | 741.00 KiB/s, done.
Resolving deltas: 100% (72/72), done.
Submodule path 'cookbooks/apache2/foodcritic/sc': checked out 'ab11cfb7c164b227b0386b12b984867fa8622f1a'

$ git-subtrac -v update
Scanning branch: refs/heads/master
Searching submodules for: refs/heads/master/cookbooks/apache2@7b9de411db
  ...found! in cookbooks/apache2
  add 72f6d8544f refs/heads/master/cookbooks/apache2@7b9de411db/.github/
  add 1141fcb0f4 refs/heads/master/cookbooks/apache2@7b9de411db/attributes/
  add 5fc2b0286a refs/heads/master/cookbooks/apache2@7b9de411db/definitions/
  add 0cc1965116 refs/heads/master/cookbooks/apache2@7b9de411db/files/default/
  add 65f41c9241 refs/heads/master/cookbooks/apache2@7b9de411db/files/
Searching submodules for: refs/heads/master/cookbooks/apache2@7b9de411db/foodcritic/sc@ab11cfb7c1
  ...not in cookbooks/apache2
git-subtrac: refs/heads/master/cookbooks/apache2@7b9de411db/foodcritic/sc@ab11cfb7c1: ab11cfb7c164b227b0386b12b984867fa8622f1a not found. (fetch it manually? or try --exclude)

Note that git is able to checkout commit ab11cfb7c164b227b0386b12b984867fa8622f1a during git clone.