Code Monkey home page Code Monkey logo

git-explode's Introduction

git-explode

git-explode is a tool for automatically exploding a single git branch into a number of smaller branches which are textually independent. It uses git-deps to automatically detect textual dependencies between the commits in the branch, and calculates the grouping and ordering of commits into independent sub-topics from which the new branches are created.

I have blogged about git-explode and related tools, and also publically spoken about the tool several times:

Use case #1

The most obvious use case for this tool is helping improve the "hygiene" of branch management, so that each branch in your repository is tightly and cleanly scoped to a single logical topic.

For example during work on feature branch, you might become aware of an opportunity to refactor some existing code, and might decide to take advantage of that opportunity immediately, by adding refactoring commits to the tip of the feature branch. And during the refactoring, you may even spot a bug, and end up also adding a bugfix to the same feature branch.

So now you have a feature branch which is polluted by commits which perform refactoring and bugfixing. If you were to submit this branch for code review as a single GitHub pull request (or GitLab merge request, or Gerrit change topic), it would be a lot harder for your collaborators to review than if you had separately submitted three smaller reviews, one for the bugfix, one for the refactoring, and one for the new feature.

In this scenario, git-explode comes to the rescue! Rather than you having to manually separate out the commits into topic branches, it can do all the hard work for you with a single command.

Textual vs. semantic (in)dependence

Astute readers will note that textual independence (as detected by git-deps and used by git-explode) is not the same as semantic / logical independence. Textual independence means that the changes can be applied in any order without incurring conflicts, but this is not a reliable indicator of logical independence. (This caveat is also noted in the README for git-deps.)

For example a change to a function and corresponding changes to the tests and/or documentation for that function would typically exist in different files. So if those changes were in separate commits within a branch, running git-explode on the branch would place those commits in separate branches even though they are logically related, because changes in different files (or even in different areas of the same files) are textually independent.

So in this case, git-explode would not behave exactly how we might want. And for as long as AI is an unsolved problem, it is very unlikely that it will ever develop totally reliable behaviour. So does that mean git-explode is useless? Absolutely not!

Firstly, when best practices for commit structuring are adhered to, changes which are strongly logically related should be placed within the same commit anyway. So in the example above, a change to a function and corresponding changes to the tests and/or documentation for that function should all be within a single commit. (Although this is not the only valid approach; for a more advanced meta-history grouping mechanism, see git-dendrify.)

Secondly, whilst textual independence does not imply logical independence, the converse is much more commonly true: logical independence typically implies textual independence. So while it might not be too uncommon for git-explode to separate logically-related changes into different branches, it should be pretty rare that it groups logically unrelated changes on the same branch. Combining this with the fact that git makes it easier to join commits on separate branches back together into one branch than to split them apart suggests that git-explode still has plenty of potential for saving effort.

Thirdly, it is often unhelpful to allow the quest for the perfect become the enemy of the good - a tool does not have to be perfect to be useful; it only has to be better than performing the same task without the tool.

Further discussion on these points can be found in an old thread from the git mailing list.

Ultimately though, "the proof is in the pudding", so try it out and see!

Other use cases

As already explained above, the most obvious use case is cleaning up the mess caused by logically independent commits all mashed together into one branch. However here are some further use cases. If you can think of others, please submit them!

Use case #2: Decompose changes for porting

If you need to backport or forward-port changes between two branches, git-explode could be used to first decompose the source branch into textually independent topic branches. Then before any porting starts, informed decisions can be made about which topics to port and which not to port, and in which order. Each decomposed topic branch is guaranteed to be textually independent from the others, so they can be ported separately - indeed even concurrently by different people - thereby greatly reducing the risk of conflicts when the independent topic branches are merged together into the target branch.

Use case #3: Publishing a previously private codebase

Emmet's idea about a company who needs to publish a private codebase but needs to clean it up first. Similar to 1. but on a bigger scale.

Use case #4: Breaking down giant commits

Split giant commit into commits one per hunk, then regroup into topics along with previous related commits. (Note that the previous related commits are required here for the regrouping to work, since hunks within a single commit are by definition independent of each other.)

Installation

Please see the INSTALL.rst file.

Usage

Usage is fairly self-explanatory if you run git explode -h:

usage: git-explode [-h] [--version] [-d] [-p PREFIX] [-c NUM] BASE HEAD

Explode linear sequence of commits into topic branches

positional arguments:
  BASE                  base of sequence to explode
  HEAD                  head of sequence to explode

optional arguments:
  -h, --help            show this help message and exit
  --version             show program's version number and exit
  -d, --debug           Show debugging
  -p PREFIX, --prefix PREFIX
                        prefix for all created topic branches
  -c NUM, --context-lines NUM
                        Number of lines of diff context to use [1]

Development / support / feedback

Please see the CONTRIBUTING.rst file.

History

I first announced the intention to build this tool on the git mailing list in May 2016; however at the time I was under the mistaken impression that I could build it out of the git-splice and git-transplant commands which I was working on at that time.

Thanks to SUSE's generous Hack Week policy, I have had the luxury of working on this as a Hack Week project.

In May 2018 I took advantage of another Hack Week to apply more polish and make the first release. This was in preparation for demonstrating the software at a Meetup event of the Git London User Group.

License

Released under GPL version 2 in order to be consistent with git's license, but I'm open to the idea of dual-licensing if there's a convincing reason.

git-explode's People

Contributors

aspiers avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

git-explode's Issues

install pygit2: fatal error: 'git2.h' file not found

maybe this is not a git-explode issue, but still I've run into it:

sudo pip install git-deps

results in:

 creating build/temp.macosx-10.11-x86_64-2.7
    building '_pygit2' extension
    creating build/temp.macosx-10.11-x86_64-2.7/src
    clang -fno-strict-aliasing -fno-common -dynamic -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/usr/local/include -I/usr/local/include -I/usr/local/opt/openssl/include -I/usr/local/opt/sqlite/include -I/usr/local/Cellar/python@2/2.7.15/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c src/blob.c -o build/temp.macosx-10.11-x86_64-2.7/src/blob.o
    In file included from src/blob.c:30:
    src/blob.h:33:10: fatal error: 'git2.h' file not found
    #include <git2.h>
             ^
    1 error generated.
    error: command 'clang' failed with exit status 1
    
    ----------------------------------------
Command "/usr/local/opt/python@2/bin/python2.7 -u -c "import setuptools, tokenize;__file__='/private/tmp/pip-install-u4aDlM/pygit2/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /private/tmp/pip-record-_osVGz/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/tmp/pip-install-u4aDlM/pygit2/

Ignores --prefix

The command used for #7 was:

% git explode -p zenspider/zeit_ main zenspider/zeitwerk -d

and as you can see, it was creating branches with the names topic#

Completely bogs on large codebase

It looks like it is ignoring my BASE, but I suspect it is looking at all the surrounding blame hunks. According to -d it is looking at a LOT of commits. It bogs the whole thing down. Your demo in the video is snappy. This hangs. And hangs. Some notion of progress or something would help.

Also, it can't be killed with ^c... I have to suspend and kill -9 %1?

UnicodeDecodeError: 'charmap' codec can't decode byte 0x98

I can't run your extension. As I understand it doesn't work with russian letters. I got an error:

$ git-explode master^ master
Traceback (most recent call last):
File "C:/Users/zyrianovdv/AppData/Local/Programs/Python/Python36-32/Scripts/git-explode", line 10, in
sys.exit(run())
File "C:\users\zyrianovdv\appdata\local\programs\python\python36-32\lib\site-packages\git_explode\cli.py", line 68, in run
main(sys.argv[1:])
File "C:\users\zyrianovdv\appdata\local\programs\python\python36-32\lib\site-packages\git_explode\cli.py", line 64, in main
exploder.run()
File "C:\users\zyrianovdv\appdata\local\programs\python\python36-32\lib\site-packages\git_explode\exploder.py", line 39, in run
commits, deps_from, deps_on = self.get_dependencies()
File "C:\users\zyrianovdv\appdata\local\programs\python\python36-32\lib\site-packages\git_explode\exploder.py", line 67, in get_dependencies
detector.find_dependencies(rev)
File "C:\users\zyrianovdv\appdata\local\programs\python\python36-32\lib\site-packages\git_deps\detector.py", line 122, in find_dependencies
self.find_dependencies_with_parent(dependent, parent)
File "C:\users\zyrianovdv\appdata\local\programs\python\python36-32\lib\site-packages\git_deps\detector.py", line 147, in find_dependencies_with_parent
self.blame_hunk(dependent, parent, path, hunk)
File "C:\users\zyrianovdv\appdata\local\programs\python\python36-32\lib\site-packages\git_deps\detector.py", line 172, in blame_hunk
blame = subprocess.check_output(cmd, universal_newlines=True)
File "C:\users\zyrianovdv\appdata\local\programs\python\python36-32\lib\subprocess.py", line 336, in check_output
**kwargs).stdout
File "C:\users\zyrianovdv\appdata\local\programs\python\python36-32\lib\subprocess.py", line 405, in run
stdout, stderr = process.communicate(input, timeout=timeout)
File "C:\users\zyrianovdv\appdata\local\programs\python\python36-32\lib\subprocess.py", line 830, in communicate
stdout = self.stdout.read()
File "C:\users\zyrianovdv\appdata\local\programs\python\python36-32\lib\encodings\cp1251.py", line 23, in decode
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x98 in position 348: character maps to

I created simple repository for reproduction my case. You need apply patches: 0001-add-file.patch, 0002-some-changes-1.patch, 0003-some-changes-2.patch and then git-explode master^ master for reproduction.
patches.zip

Crash creating exploded topics

Once it did finish analysis, it crashed:

Jul 26 18:14:52 DEBUG  Exploding 0e1526a1 ARGH
git checkout -q -b topic1 main
git cherry-pick 0e1526a1dfed84807adc366ce0278a0cafb008ba
Jul 26 18:14:52 DEBUG    Assigned topic1 to 0e1526a1dfed84807adc366ce0278a0cafb008ba
Jul 26 18:14:52 DEBUG  - cherry-picked 0e1526a1 as e0b57735 (ARGH)
Jul 26 18:14:52 DEBUG  Exploding 5228ebef Speed up codeowner glob test by using ruby, not bash
git checkout -q -b topic2 main
git cherry-pick 5228ebef053318e574120ddae698a6e831fb61d7
Jul 26 18:14:53 DEBUG    Assigned topic2 to 5228ebef053318e574120ddae698a6e831fb61d7
Jul 26 18:14:53 DEBUG  - cherry-picked 5228ebef as 8269a2b4 (Speed up codeowner glob test by using ruby, not bash)
Jul 26 18:14:53 DEBUG  Exploding dfba4974 Add many acronyms to config/initializers/inflections.rb
git checkout -q -b topic3 main
git cherry-pick dfba4974279cbabb35c0aa0dde2a8ab796c09fd4
Jul 26 18:14:53 DEBUG    Assigned topic3 to dfba4974279cbabb35c0aa0dde2a8ab796c09fd4
Jul 26 18:14:53 DEBUG  - cherry-picked dfba4974 as 5b1d79d9 (Add many acronyms to config/initializers/inflections.rb)
Jul 26 18:14:53 DEBUG  + pushed to queue: 41307936 inflect JSON and YAML correctly
Jul 26 18:14:53 DEBUG  + pushed to queue: 1f590ec0 inflect HC as a non-acronym
Jul 26 18:14:53 DEBUG  + pushed to queue: c421f3e1 inflect CIA correctly
Jul 26 18:14:53 DEBUG  + pushed to queue: 560f874e inflect API as a non-acronym
Jul 26 18:14:53 DEBUG  Exploding 560f874e inflect API as a non-acronym
Jul 26 18:14:53 DEBUG    deps: dfba4974
git cherry-pick 560f874e8124d9cad6ea55814bdb519f0d8da719
Jul 26 18:14:53 DEBUG    Unassigned topic3 from dfba4974279cbabb35c0aa0dde2a8ab796c09fd4
Jul 26 18:14:53 DEBUG    Assigned topic3 to 560f874e8124d9cad6ea55814bdb519f0d8da719
Jul 26 18:14:53 DEBUG  - cherry-picked 560f874e as 11b71822 (inflect API as a non-acronym)
Jul 26 18:14:53 DEBUG  Exploding c421f3e1 inflect CIA correctly
Jul 26 18:14:53 DEBUG    deps: dfba4974
Jul 26 18:14:53 DEBUG    Assigned topic4 to dfba4974279cbabb35c0aa0dde2a8ab796c09fd4
Traceback (most recent call last):
  File "/Users/ryan.davis/Library/Python/3.11/bin/git-explode", line 10, in <module>
    sys.exit(run())
             ^^^^^
  File "/Users/ryan.davis/Code/git/aspiers/git-explode/git_explode/cli.py", line 71, in run
    main(sys.argv[1:])
  File "/Users/ryan.davis/Code/git/aspiers/git-explode/git_explode/cli.py", line 67, in main
    exploder.run()
  File "/Users/ryan.davis/Code/git/aspiers/git-explode/git_explode/exploder.py", line 40, in run
    self.explode(commits, deps_from, deps_on)
  File "/Users/ryan.davis/Code/git/aspiers/git-explode/git_explode/exploder.py", line 107, in explode
    self.prepare_cherrypick_base(sha, deps, commits)
  File "/Users/ryan.davis/Code/git/aspiers/git-explode/git_explode/exploder.py", line 130, in prepare_cherrypick_base
    self.checkout_new_dependent_topic(deps)
  File "/Users/ryan.davis/Code/git/aspiers/git-explode/git_explode/exploder.py", line 189, in checkout_new_dependent_topic
    base = self.exploded[deps[0]]
                         ~~~~^^^
TypeError: 'dict_keys' object is not subscriptable

Needs submission to PyPI

Currently INSTALL.md says to do pip install git-explode but that won't work until I submit to PyPI.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.