seeraven / gitcache Goto Github PK

View Code? Open in Web Editor NEW

31.0 31.0 6.0 321 KB

Local cache for git repositories to speed up working with large repositories and multiple clones.

License: BSD 3-Clause "New" or "Revised" License

Makefile 0.61% Python 99.39%

gitcache's People

Contributors

Stargazers

Watchers

Forkers

jaw shinh eremenko789 niikoo youw m-tmatma

gitcache's Issues

Unable to clone using git submodule

Hello,
I am trying to use gitcache but it throws me an error.

PS B:\gitlab\0\-software> gitcache git submodule update --init --recursive --depth 1 --recursive
2024-03-14 18:31:37 Save configuration file /.gitcache\config.
2024-03-14 18:31:38 Starting Fetch.
2024-03-14 18:31:40 Fetch was successfully completed within 1.5 seconds.
2024-03-14 18:31:40 Starting Fetch.
2024-03-14 18:31:42 Fetch was successfully completed within 1.5 seconds.
2024-03-14 18:31:43 Starting Fetch.
2024-03-14 18:31:44 Fetch was successfully completed within 1.3 seconds.
2024-03-14 18:31:44 Starting Fetch.
2024-03-14 18:31:46 Fetch was successfully completed within 1.5 seconds.
2024-03-14 18:31:46 Starting Fetch.
2024-03-14 18:31:48 Fetch was successfully completed within 1.3 seconds.
Traceback (most recent call last):
  File "\\vboxsvr\vagrant\gitcache", line 47, in <module>
  File "git_cache\git_command.py", line 113, in handle_git_command
  File "git_cache\commands\submodule_update.py", line 115, in git_submodule_update
IndexError: list index out of range
[2464] Failed to execute script 'gitcache' due to unhandled exception!
PS B:\gitlab\0\-software> gitcache git submodule update
2024-03-14 18:31:58 Starting Fetch.
2024-03-14 18:32:00 Fetch was successfully completed within 1.3 seconds.
2024-03-14 18:32:00 Starting Fetch.
2024-03-14 18:32:02 Fetch was successfully completed within 2.0 seconds.
2024-03-14 18:32:03 Starting Fetch.
2024-03-14 18:32:05 Fetch was successfully completed within 1.8 seconds.
2024-03-14 18:32:05 Starting Fetch.
2024-03-14 18:32:07 Fetch was successfully completed within 1.5 seconds.
2024-03-14 18:32:07 Starting Fetch.
2024-03-14 18:32:08 Fetch was successfully completed within 1.3 seconds.
Traceback (most recent call last):
  File "\\vboxsvr\vagrant\gitcache", line 47, in <module>
  File "git_cache\git_command.py", line 113, in handle_git_command
  File "git_cache\commands\submodule_update.py", line 115, in git_submodule_update
IndexError: list index out of range
[6956] Failed to execute script 'gitcache' due to unhandled exception!
PS B:\gitlab\0\-software> gitcache git submodule update --init
2024-03-14 18:34:30 Starting Fetch.
2024-03-14 18:34:31 Fetch was successfully completed within 1.5 seconds.
2024-03-14 18:34:32 Starting Fetch.
2024-03-14 18:34:33 Fetch was successfully completed within 1.5 seconds.
2024-03-14 18:34:34 Starting Fetch.
2024-03-14 18:34:35 Fetch was successfully completed within 1.5 seconds.
2024-03-14 18:34:36 Starting Fetch.
2024-03-14 18:34:37 Fetch was successfully completed within 1.5 seconds.
2024-03-14 18:34:38 Starting Fetch.
2024-03-14 18:34:39 Fetch was successfully completed within 1.3 seconds.
Traceback (most recent call last):
  File "\\vboxsvr\vagrant\gitcache", line 47, in <module>
  File "git_cache\git_command.py", line 113, in handle_git_command
  File "git_cache\commands\submodule_update.py", line 115, in git_submodule_update
IndexError: list index out of range
[1808] Failed to execute script 'gitcache' due to unhandled exception!

If I run simply git submodule update (...) it works, but gitcache shows 0 stats.
I created the symlink and gitcache is before git in the PATH.

Any help is appreciated! Thanks a lot

git clone --recurse-submodules not cached submodules

When I run git clone --recurse-submodules <repo>, submodules not cached.

No password prompt on Windows

When using a password secured ssh key, gitcache does not ask for a password on Windows but simply fails to clone:

2024-04-25 19:19:44 Starting Initial clone of [email protected]:<my repo>.git into D:\gitcache\mirrors\github.com\<my repo>.
Cloning into bare repository 'D:\gitcache\mirrors\github.com\<my repo>\git'...
[email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
2024-04-25 19:19:46 Command 'C:\Program Files\Git\cmd\git.exe clone --progress --mirror [email protected]:<my repo>.git D:\gitcache\mirrors\github.com\<my repo>\git' failed with return code 128. Starting retry 1 of 3.

On Linux the password prompt is shown as expected.

Discovered and reported by @Youw

Repeat Initial when clone

I export PATH with gitcache, and I run git clone https://github.com/seeraven/gitcache.git

It output as below:

root@1ffc1000999a:/# git clone https://github.com/seeraven/gitcache.git 
2023-11-01 14:41:48 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:48 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:48 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:48 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:48 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:48 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:48 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:48 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:48 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:48 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:49 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:49 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:49 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:49 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:49 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:49 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:49 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:49 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:49 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:49 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:49 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:49 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:49 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:49 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:49 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:49 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:49 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:49 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:49 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
^CTraceback (most recent call last):
  File "/gitcache/git", line 40, in <module>
    handle_git_command(sys.argv[0:1], sys.argv[1:])
  File "/gitcache/git_cache/git_command.py", line 95, in handle_git_command
    sys.exit(git_clone(git_options))
  File "/gitcache/git_cache/commands/clone.py", line 53, in git_clone
    return mirror.clone_from_mirror(git_options)
  File "/gitcache/git_cache/git_mirror.py", line 272, in clone_from_mirror
    if not self.update(ref):
  File "/gitcache/git_cache/git_mirror.py", line 149, in update
    return self._clone(ref)
  File "/gitcache/git_cache/git_mirror.py", line 363, in _clone
    remove_dir=self.git_dir)
  File "/gitcache/git_cache/command_execution.py", line 129, in pretty_call_command_retry
    return_code, stdout_buffer, stderr_buffer = call_command_retry(command, num_retries, **kwargs)
  File "/gitcache/git_cache/command_execution.py", line 83, in call_command_retry
    output_timeout=output_timeout)
  File "/gitcache/git_cache/command_execution_unix.py", line 115, in call_command
    streams_ready = select.select([stdout_r, stderr_r], [], [], 1.0)[0]
KeyboardInterrupt

Question about proper commands to use :)

Hello,

So, I am using gitcache mostly for submodules, since we have more or less 5GB of submodules.

I followed the setup instructions and I see some stats, however how can I be sure that running git submodule (...) is using gitcache?

Normally what I do is gitcache git submodule (...) but I am not sure if this is the right way or just using git clone will use gitcache behind the hood.

However, on stats, I just see stats of some submodules, not all of them.

If someone could tell me what is the proper way to use it I would be very grateful!

Only on HTTPS?

It seems like only HTTPS is supported and not SSH. When I try to clone via SSH, an unhandled repository is cloned.

Check Setup on Each Execution of gitcache

gitcache is a wrapper for git, so it needs to intercept the command line calls of git (resp. git.exe on windows). But to work with git, gitcache itself also has to call the real git command.

To ensure gitcache is called as git we can either

create a symlink from git to gitcache (this is the recommended setup documented in the README.md),
rename gitcache to git or
copy gitcache to git.

And to ensure it is called instead of the real git it must be placed in a directory referenced by the PATH variable before the directory containing the real git command. Actually, the real git command can also be in a directory not referenced by the PATH variable as long as the realgit config setting of gitcache is correct.

Currently, the detection of the real git command works by doing something similar to shutil.which but ignoring symlinks. This can lead to recursive calls of itself if the user did not create a symlink but a copy or rename operation.

So the initial detection of the real git command must be changed. Here, the question is how to identify an executable of git as a real git command. We can't use an option that is handled by git as any wrapper like ourself will simply forward it, but we can add an option to identify whether the candidate is actually gitcache by adding a new git option --identify.

Cannot "Delete directory" on Windows after 5 minutes (outputtimeout)

Hello,

I am encountering an issue when trying to retry the "gitcache git clone mirror -b branch destination_folder" command after a 5 minutes timeout (outputtimeout = 5 mininute at config gitcache).

The log displays the useful information (GITCACHE_LOGLEVEL=Debug): when I using the Jenkin job:

10:11:38 Execute command '['C:\\Git\\git.exe', '-c', 'lfs.url=ssh://git@mydomain:22/repo/repo1.git/info/lfs', '-c', 'lfs.storage=D:\\Je\\gc\\mirrors\\mydomain_22\\repo\\repo1\\lfs', 'clone', 'D:\\Je\\gc\\mirrors\\mydomain_22\\repo\\repo1\\git', '-b', 'my_branch', 'D:\\Je\\wp\\job1_1\\test']' (shell=False, cwd=None) with command timeout of 3600 seconds and output timeout of 300 seconds. 10:11:38 Cloning into 'D:\Je\wp\job1_1\test'... 10:16:39 No stdout/stderr output received within 300 seconds! 10:16:40 Command '['C:\\Git\\git.exe', '-c', 'lfs.url=ssh://git@mydomain:22/repo/repo1.git/info/lfs', '-c', 'lfs.storage=D:\\Je\\gc\\mirrors\\mydomain_22\\repo\\repo1\\lfs', 'clone', 'D:\\Je\\gc\\mirrors\\mydomain_22\\repo\\repo1\\git', '-b', 'my_branch', 'D:\\Je\\wp\\job1_1\\test']' finished with return code -2000. 10:16:40 Delete directory D:\Je\wp\job1_1\test. 10:19:17 Command '['C:\\Git\\git.exe', '-c', 'lfs.url=ssh://git@mydomain:22/repo/repo1.git/info/lfs', '-c', 'lfs.storage=D:\\Je\\gc\\mirrors\\mydomain_22\\repo\\repo1\\lfs', 'clone', 'D:\\Je\\gc\\mirrors\\mydomain_22\\repo\\repo1\\git', '-b', 'my_branch', 'D:\\Je\\wp\\job1_1\\test']' failed with return code -2000. Starting retry 1 of 3. 10:19:17 Execute command '['C:\\Git\\git.exe', '-c', 'lfs.url=ssh://git@mydomain:22/repo/repo1.git/info/lfs', '-c', 'lfs.storage=D:\\Je\\gc\\mirrors\\mydomain_22\\repo\\repo1\\lfs', 'clone', 'D:\\Je\\gc\\mirrors\\mydomain_22\\repo\\repo1\\git', '-b', 'my_branch', 'D:\\Je\\wp\\job1_1\\test']' (shell=False, cwd=None) with command timeout of 3600 seconds and output timeout of 300 seconds. 10:19:17 fatal: destination path 'D:\Je\wp\job1_1\test' already exists and is not an empty directory.

When checking the folder D:\Je\wp\job1_1\test. I see that the presence of the ".git" subfolder prevents cloning again.

and no errors appear in the folder deletion job (I found it here, but don't know if it's right)

https://github.com/seeraven/gitcache/blob/ec99dcf7c175279927aea198753ff7028071f95f/src/git_cache/command_execution.py#L117

The process tries 3 times before receiving an error result with return code 128.

Is a Windows build possible?

Title says it all :)

Git config call is not beeing routed

git config --list

is not beeing routed to git

git will fail after cancel

When using ctrl+c to cancel the operation of cloning from the remote, the following clone will always fail.

I found a db.lock at my GITCACHE_DIR,

Trace back as follows:

Traceback (most recent call last):
  File "/gitcache/git", line 40, in <module>
    handle_git_command(sys.argv[0:1], sys.argv[1:])
  File "/gitcache/git_cache/git_command.py", line 95, in handle_git_command
    sys.exit(git_clone(git_options))
  File "/gitcache/git_cache/commands/clone.py", line 53, in git_clone
    return mirror.clone_from_mirror(git_options)
  File "/gitcache/git_cache/git_mirror.py", line 270, in clone_from_mirror
    if not self.update(ref):
  File "/gitcache/git_cache/git_mirror.py", line 150, in update
    return self._update(ref)
  File "/gitcache/git_cache/git_mirror.py", line 398, in _update
    self.database.save_update_time(self.path)
  File "/gitcache/git_cache/database.py", line 92, in save_update_time
    self.database[path]["last-update-time"] = time.time()
KeyError: '/git_cache/mirrors/github.com/seeraven/gitcache'

I think it may be necessary to handle signal。

Use make4py submodule

With make4py we have a common infrastructure for building python applications. So gitcache should be using it.

git lfs fetch command is not correctly interpreted

A valid git lfs fetch command with arguments like

git lfs fetch origin --include somefile --exclude ""

gives error messages like

2021-10-12 07:33:08 Command 'cd xxx; /usr/bin/git -c lfs.storage=xxx lfs fetch  origin origin' failed. Starting retry 1 of 3.
Invalid ref argument: [origin]

As one can see, the ref is appended although it was already specified.

In addition, we need to clarify what to do with the --include and --exclude arguments. They are currently removed but should IMHO be added as well.

"git clone ... -b branch" causes error

When executing git clone ... -b branch the last argument branch is interpreted as the target directory. Although the man page of git specifies differently, it is apparently allowed to give options after the url or target directory!
To solve this issue, we should think about a general parser for the git commands, or at least get a understanding on how the original git command parser works.

Update mirror repo remote url before each fetch

The remote url of the same repository may slightly differ between gitcache invocations.
The use case: two different CI jobs running on the same machine (not even concurently, or anythign like that), but configured differently.
One is using the form of https://${GITHUB_PAT}@github.com/<repo>.git and the second one is using [email protected]:<repo>.git with ssh-agent setup running.
Because the remote url of the mirror repo is not updated, only the job who initially configured a mirror would work, and the other one would fail, since for instance having an ssh-agent for https:// url is useless, and the PAT token for ssh url is unusable as well.

Regex to match git URLs for caching

We have a setup with multiple submodules (recursive) on an internal gitlab server.
Then those in turn load in submodules from github, usually libraries.
Those don't have to be updated very often while the local ones have to be updated on every build.

So, if you could add a string or regex as a config parameter that must match the URL for the cache to actually do its caching and pass everything else through would be awesome.

Default this parameter could be "" aka don't care.

Add make targets for macOS x86_64/arm64 release builds

git clone fail

I use source and just
./gitcache git clone

Error fetching repos that don't have a master branch

We have a repository with various branches, but none called "master". It looks like that branch name is getting added and causing checkouts to fail:

2023-11-09 09:59:08 Command 'C:\Users\afabian\AppData\Local\Programs\Git\cmd\git.exe -c lfs.storage=c:\gitcache\cache\mirrors\REPO_PATH\lfs lfs fetch origin master' failed with return code 2. Starting retry 2 of 3.
Invalid ref argument: [master]

I think the expected behavior is to check out the ref specified by the parent repository. Come to think of it, "master" would be wrong even if it did exist.

Edit: just realized I said "checkouts" but the actual command was "submodule update --init --recursive"

Bug in `git submodule update --recursive --init`?

Thank you for publishing great software.

I tried to use gitcache for my work and I found git submodule update --recursive --init doesn't work. Here is a reproducing script and its result. Do you have any ideas on this behavior?

> cat gitcache-val.sh
#! /bin/bash -eu

REAL_GIT_DIR1=real_git_dir1
REAL_GIT_DIR2=real_git_dir2
GITCACHE_DIR1=gitcache_dir1
GITCACHE_DIR2=gitcache_dir2

rm -rf ${REAL_GIT_DIR1}
rm -rf ${REAL_GIT_DIR2}
rm -rf ${GITCACHE_DIR1}
rm -rf ${GITCACHE_DIR2}

mkdir ${REAL_GIT_DIR1}
pushd ${REAL_GIT_DIR1}
/usr/bin/git clone https://github.com/aws/aws-sdk-cpp.git --single-branch --branch 1.9.188
pushd aws-sdk-cpp
/usr/bin/git submodule update --recursive --init
popd
popd

mkdir ${REAL_GIT_DIR2}
pushd ${REAL_GIT_DIR2}
/usr/bin/git clone https://github.com/aws/aws-sdk-cpp.git --single-branch --branch 1.9.188
pushd aws-sdk-cpp
/usr/bin/git submodule update --recursive --init
popd
popd

mkdir ${GITCACHE_DIR1}
pushd ${GITCACHE_DIR1}
git clone https://github.com/aws/aws-sdk-cpp.git --single-branch --branch 1.9.188
pushd aws-sdk-cpp
git submodule update --recursive --init
popd
popd

mkdir ${GITCACHE_DIR2}
pushd ${GITCACHE_DIR2}
git clone https://github.com/aws/aws-sdk-cpp.git --single-branch --branch 1.9.188
pushd aws-sdk-cpp
git submodule update --recursive --init
popd
popd

echo ========== Number of diff of real_git_dir1 and real_git_dir2 ==========
diff -rq real_git_dir1/aws-sdk-cpp real_git_dir2/aws-sdk-cpp | grep -v "[.]git" | wc -l
echo ========== Number of diff of real_git_dir1 and gitcache_dir1 ==========
diff -rq real_git_dir1/aws-sdk-cpp gitcache_dir1/aws-sdk-cpp | grep -v "[.]git" | wc -l
echo ========== Number of diff of gitcache_dir1 and gitcache_dir2 ==========
diff -rq gitcache_dir1/aws-sdk-cpp gitcache_dir2/aws-sdk-cpp | grep -v "[.]git" | wc -l

> ./gitcache-val.sh
========== Number of diff of real_git_dir1 and real_git_dir2 ==========
0
========== Number of diff of real_git_dir1 and gitcache_dir1 ==========
218
========== Number of diff of gitcache_dir1 and gitcache_dir2 ==========
0

Extend Functional Tests to Use Recommended Setup

The recommended setup consists of a symlink called git (or git.exe on Windows) targeting the gitcache executable. Such a setup should be covered by the functional tests which are currently calling gitcache resp. gitcache git.

git lfs pull is not handled by gitcache

The git lfs pull command is not handled by gitcache, as it was not necessary with previous versions of git-lfs. However, the
behaviour of git-lfs has changed and leads to errors when the remote (fetch URL) is set to the mirror. When the remote fetch URL is set to the remote repository, git lfs pull works as expected. This happens only if there are files to fetch, e.g., files that are filtered by default but are included using the command line options --include and --exclude "".

So the git lfs pull commands should be handled by gitcache in the following way:

If the specified remote is not origin, proceed with the original git command.
Change the remote fetch URL to the push URL.
Perform the original git command.
Restore the remote fetch URL.

An alternative would be to setup a temporary remote like real-origin and use that remote. This would have the advantage of being more robust in cases where the command is interrupted and leaving the temporary fetch URL.

Terribly underrated

I just want to say this tool works extremely well and I am very grateful for it!

Slow start up

Hi,

I have been using/trying gitcache for a couple of days already and I have noticed that in some cases is slower that plain git.

I'm trying to use gitcache with Terragrunt/Atlantis to avoid cloning the same repo hundreds of times, but since Terragrunt uses git rev-parse --show-toplevel a lot it is being slower than plain git.

The start up time before showing any debug is considerable (In the order of 3-4 seconds) for an already cloned repo:

bash-5.2$ export GITCACHE_UPDATE_INTERVAL=10d
bash-5.2$ time GITCACHE_LOGLEVEL=Debug /usr/local/bin/git clone [email protected]:seeraven/gitcache.git
2024-05-10 10:05:43 Python executable: /usr/local/bin/git
2024-05-10 10:05:43 Called as ['/usr/local/bin/git', 'clone', '[email protected]:seeraven/gitcache.git']
2024-05-10 10:05:43 handle_git_command(['/usr/local/bin/git'], ['clone', '[email protected]:seeraven/gitcache.git']) started
2024-05-10 10:05:43 Found global options [], command 'clone', command options '[]' and arguments ['[email protected]:seeraven/gitcache.git'].
2024-05-10 10:05:43 Loading configuration file /Users/andresmontalban/.gitcache/config.
2024-05-10 10:05:43 Loading configuration file /Users/andresmontalban/.gitcache/config.
2024-05-10 10:05:43 Can't load configuration file /Users/andresmontalban/.gitcache/mirrors/github.com/seeraven/gitcache/gitcache.config as it does not exist!
2024-05-10 10:05:43 Update time of mirror /Users/andresmontalban/.gitcache/mirrors/github.com/seeraven/gitcache not reached yet.
2024-05-10 10:05:43 Starting Clone from mirror /Users/andresmontalban/.gitcache/mirrors/github.com/seeraven/gitcache.
2024-05-10 10:05:43 Retry to execute command '/usr/bin/git -c [email protected]:seeraven/gitcache.git/info/lfs -c lfs.storage=/Users/andresmontalban/.gitcache/mirrors/github.com/seeraven/gitcache/lfs clone /Users/andresmontalban/.gitcache/mirrors/github.com/seeraven/gitcache/git gitcache' up to 3 times.
2024-05-10 10:05:43 Execute command '/usr/bin/git -c [email protected]:seeraven/gitcache.git/info/lfs -c lfs.storage=/Users/andresmontalban/.gitcache/mirrors/github.com/seeraven/gitcache/lfs clone /Users/andresmontalban/.gitcache/mirrors/github.com/seeraven/gitcache/git gitcache' (shell=False, cwd=None) with command timeout of 3600 seconds and output timeout of 300 seconds. stderr_capture=True
Cloning into 'gitcache'...
done.
2024-05-10 10:05:43 Command '/usr/bin/git -c [email protected]:seeraven/gitcache.git/info/lfs -c lfs.storage=/Users/andresmontalban/.gitcache/mirrors/github.com/seeraven/gitcache/lfs clone /Users/andresmontalban/.gitcache/mirrors/github.com/seeraven/gitcache/git gitcache' finished with return code 0.
2024-05-10 10:05:43 Clone from mirror /Users/andresmontalban/.gitcache/mirrors/github.com/seeraven/gitcache was successfully completed within 0.1 seconds.
2024-05-10 10:05:43 Setting push URL to [email protected]:seeraven/gitcache.git and configure LFS.

real	0m5.843s
user	0m0.295s
sys	0m0.168s

If I do not setup GITCACHE_LOGLEVEL=Debug it takes longer because it has to update the clone.

On the other hand if I use plain git I get faster clone:

bash-5.2$ time git clone [email protected]:seeraven/gitcache.git
Cloning into 'gitcache'...
remote: Enumerating objects: 1122, done.
remote: Counting objects: 100% (150/150), done.
remote: Compressing objects: 100% (59/59), done.
remote: Total 1122 (delta 101), reused 114 (delta 84), pack-reused 972
Receiving objects: 100% (1122/1122), 291.03 KiB | 483.00 KiB/s, done.
Resolving deltas: 100% (729/729), done.

real	0m3.956s
user	0m0.160s
sys	0m0.088s

Also simple commands like git rev-parse --show-toplevel are slow to execute:

bash-5.2$ time /usr/local/bin/git rev-parse --show-toplevel
/Users/andresmontalban/code/gitcache

real	0m5.742s
user	0m0.278s
sys	0m0.131s
bash-5.2$ time git rev-parse --show-toplevel
/Users/andresmontalban/code/gitcache

real	0m0.045s
user	0m0.007s
sys	0m0.013s

For these tests I'm running gitcache on a Macbook Pro with M1 Pro CPU, but I'm getting similar results in an AWS instance with 16 cores.

Any idea on how to make it faster?

Thank you very in advance for your help!