seeraven / gitcache Goto Github PK
View Code? Open in Web Editor NEWLocal cache for git repositories to speed up working with large repositories and multiple clones.
License: BSD 3-Clause "New" or "Revised" License
Local cache for git repositories to speed up working with large repositories and multiple clones.
License: BSD 3-Clause "New" or "Revised" License
Hello,
I am trying to use gitcache but it throws me an error.
PS B:\gitlab\0\-software> gitcache git submodule update --init --recursive --depth 1 --recursive
2024-03-14 18:31:37 Save configuration file /.gitcache\config.
2024-03-14 18:31:38 Starting Fetch.
2024-03-14 18:31:40 Fetch was successfully completed within 1.5 seconds.
2024-03-14 18:31:40 Starting Fetch.
2024-03-14 18:31:42 Fetch was successfully completed within 1.5 seconds.
2024-03-14 18:31:43 Starting Fetch.
2024-03-14 18:31:44 Fetch was successfully completed within 1.3 seconds.
2024-03-14 18:31:44 Starting Fetch.
2024-03-14 18:31:46 Fetch was successfully completed within 1.5 seconds.
2024-03-14 18:31:46 Starting Fetch.
2024-03-14 18:31:48 Fetch was successfully completed within 1.3 seconds.
Traceback (most recent call last):
File "\\vboxsvr\vagrant\gitcache", line 47, in <module>
File "git_cache\git_command.py", line 113, in handle_git_command
File "git_cache\commands\submodule_update.py", line 115, in git_submodule_update
IndexError: list index out of range
[2464] Failed to execute script 'gitcache' due to unhandled exception!
PS B:\gitlab\0\-software> gitcache git submodule update
2024-03-14 18:31:58 Starting Fetch.
2024-03-14 18:32:00 Fetch was successfully completed within 1.3 seconds.
2024-03-14 18:32:00 Starting Fetch.
2024-03-14 18:32:02 Fetch was successfully completed within 2.0 seconds.
2024-03-14 18:32:03 Starting Fetch.
2024-03-14 18:32:05 Fetch was successfully completed within 1.8 seconds.
2024-03-14 18:32:05 Starting Fetch.
2024-03-14 18:32:07 Fetch was successfully completed within 1.5 seconds.
2024-03-14 18:32:07 Starting Fetch.
2024-03-14 18:32:08 Fetch was successfully completed within 1.3 seconds.
Traceback (most recent call last):
File "\\vboxsvr\vagrant\gitcache", line 47, in <module>
File "git_cache\git_command.py", line 113, in handle_git_command
File "git_cache\commands\submodule_update.py", line 115, in git_submodule_update
IndexError: list index out of range
[6956] Failed to execute script 'gitcache' due to unhandled exception!
PS B:\gitlab\0\-software> gitcache git submodule update --init
2024-03-14 18:34:30 Starting Fetch.
2024-03-14 18:34:31 Fetch was successfully completed within 1.5 seconds.
2024-03-14 18:34:32 Starting Fetch.
2024-03-14 18:34:33 Fetch was successfully completed within 1.5 seconds.
2024-03-14 18:34:34 Starting Fetch.
2024-03-14 18:34:35 Fetch was successfully completed within 1.5 seconds.
2024-03-14 18:34:36 Starting Fetch.
2024-03-14 18:34:37 Fetch was successfully completed within 1.5 seconds.
2024-03-14 18:34:38 Starting Fetch.
2024-03-14 18:34:39 Fetch was successfully completed within 1.3 seconds.
Traceback (most recent call last):
File "\\vboxsvr\vagrant\gitcache", line 47, in <module>
File "git_cache\git_command.py", line 113, in handle_git_command
File "git_cache\commands\submodule_update.py", line 115, in git_submodule_update
IndexError: list index out of range
[1808] Failed to execute script 'gitcache' due to unhandled exception!
If I run simply git submodule update (...)
it works, but gitcache
shows 0 stats.
I created the symlink and gitcache
is before git
in the PATH
.
Any help is appreciated! Thanks a lot
When I run git clone --recurse-submodules <repo>
, submodules not cached.
When using a password secured ssh key, gitcache
does not ask for a password on Windows but simply fails to clone:
2024-04-25 19:19:44 Starting Initial clone of [email protected]:<my repo>.git into D:\gitcache\mirrors\github.com\<my repo>.
Cloning into bare repository 'D:\gitcache\mirrors\github.com\<my repo>\git'...
[email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
2024-04-25 19:19:46 Command 'C:\Program Files\Git\cmd\git.exe clone --progress --mirror [email protected]:<my repo>.git D:\gitcache\mirrors\github.com\<my repo>\git' failed with return code 128. Starting retry 1 of 3.
On Linux the password prompt is shown as expected.
Discovered and reported by @Youw
I export PATH with gitcache, and I run git clone https://github.com/seeraven/gitcache.git
It output as below:
root@1ffc1000999a:/# git clone https://github.com/seeraven/gitcache.git
2023-11-01 14:41:48 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:48 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:48 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:48 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:48 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:48 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:48 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:48 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:48 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:48 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:49 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:49 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:49 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:49 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:49 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:49 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:49 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:49 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:49 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:49 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:49 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:49 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:49 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:49 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:49 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:49 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:49 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:49 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
2023-11-01 14:41:49 Starting Initial clone of https://github.com/seeraven/gitcache.git into /root/.gitcache/mirrors/github.com/seeraven/gitcache.
^CTraceback (most recent call last):
File "/gitcache/git", line 40, in <module>
handle_git_command(sys.argv[0:1], sys.argv[1:])
File "/gitcache/git_cache/git_command.py", line 95, in handle_git_command
sys.exit(git_clone(git_options))
File "/gitcache/git_cache/commands/clone.py", line 53, in git_clone
return mirror.clone_from_mirror(git_options)
File "/gitcache/git_cache/git_mirror.py", line 272, in clone_from_mirror
if not self.update(ref):
File "/gitcache/git_cache/git_mirror.py", line 149, in update
return self._clone(ref)
File "/gitcache/git_cache/git_mirror.py", line 363, in _clone
remove_dir=self.git_dir)
File "/gitcache/git_cache/command_execution.py", line 129, in pretty_call_command_retry
return_code, stdout_buffer, stderr_buffer = call_command_retry(command, num_retries, **kwargs)
File "/gitcache/git_cache/command_execution.py", line 83, in call_command_retry
output_timeout=output_timeout)
File "/gitcache/git_cache/command_execution_unix.py", line 115, in call_command
streams_ready = select.select([stdout_r, stderr_r], [], [], 1.0)[0]
KeyboardInterrupt
Hello,
So, I am using gitcache mostly for submodules, since we have more or less 5GB of submodules.
I followed the setup instructions and I see some stats, however how can I be sure that running git submodule (...)
is using gitcache?
Normally what I do is gitcache git submodule (...)
but I am not sure if this is the right way or just using git clone
will use gitcache behind the hood.
However, on stats, I just see stats of some submodules, not all of them.
If someone could tell me what is the proper way to use it I would be very grateful!
It seems like only HTTPS is supported and not SSH. When I try to clone via SSH, an unhandled repository is cloned.
gitcache is a wrapper for git, so it needs to intercept the command line calls of git
(resp. git.exe
on windows). But to work with git, gitcache itself also has to call the real git command.
To ensure gitcache is called as git
we can either
git
to gitcache
(this is the recommended setup documented in the README.md),gitcache
to git
orgitcache
to git
.And to ensure it is called instead of the real git it must be placed in a directory referenced by the PATH
variable before the directory containing the real git command. Actually, the real git command can also be in a directory not referenced by the PATH
variable as long as the realgit
config setting of gitcache is correct.
Currently, the detection of the real git command works by doing something similar to shutil.which
but ignoring symlinks. This can lead to recursive calls of itself if the user did not create a symlink but a copy or rename operation.
So the initial detection of the real git command must be changed. Here, the question is how to identify an executable of git
as a real git command. We can't use an option that is handled by git as any wrapper like ourself will simply forward it, but we can add an option to identify whether the candidate is actually gitcache by adding a new git option --identify
.
Hello,
I am encountering an issue when trying to retry the "gitcache git clone mirror -b branch destination_folder" command after a 5 minutes timeout (outputtimeout = 5 mininute at config gitcache).
The log displays the useful information (GITCACHE_LOGLEVEL=Debug): when I using the Jenkin job:
10:11:38 Execute command '['C:\\Git\\git.exe', '-c', 'lfs.url=ssh://git@mydomain:22/repo/repo1.git/info/lfs', '-c', 'lfs.storage=D:\\Je\\gc\\mirrors\\mydomain_22\\repo\\repo1\\lfs', 'clone', 'D:\\Je\\gc\\mirrors\\mydomain_22\\repo\\repo1\\git', '-b', 'my_branch', 'D:\\Je\\wp\\job1_1\\test']' (shell=False, cwd=None) with command timeout of 3600 seconds and output timeout of 300 seconds. 10:11:38 Cloning into 'D:\Je\wp\job1_1\test'... 10:16:39 No stdout/stderr output received within 300 seconds! 10:16:40 Command '['C:\\Git\\git.exe', '-c', 'lfs.url=ssh://git@mydomain:22/repo/repo1.git/info/lfs', '-c', 'lfs.storage=D:\\Je\\gc\\mirrors\\mydomain_22\\repo\\repo1\\lfs', 'clone', 'D:\\Je\\gc\\mirrors\\mydomain_22\\repo\\repo1\\git', '-b', 'my_branch', 'D:\\Je\\wp\\job1_1\\test']' finished with return code -2000. 10:16:40 Delete directory D:\Je\wp\job1_1\test. 10:19:17 Command '['C:\\Git\\git.exe', '-c', 'lfs.url=ssh://git@mydomain:22/repo/repo1.git/info/lfs', '-c', 'lfs.storage=D:\\Je\\gc\\mirrors\\mydomain_22\\repo\\repo1\\lfs', 'clone', 'D:\\Je\\gc\\mirrors\\mydomain_22\\repo\\repo1\\git', '-b', 'my_branch', 'D:\\Je\\wp\\job1_1\\test']' failed with return code -2000. Starting retry 1 of 3. 10:19:17 Execute command '['C:\\Git\\git.exe', '-c', 'lfs.url=ssh://git@mydomain:22/repo/repo1.git/info/lfs', '-c', 'lfs.storage=D:\\Je\\gc\\mirrors\\mydomain_22\\repo\\repo1\\lfs', 'clone', 'D:\\Je\\gc\\mirrors\\mydomain_22\\repo\\repo1\\git', '-b', 'my_branch', 'D:\\Je\\wp\\job1_1\\test']' (shell=False, cwd=None) with command timeout of 3600 seconds and output timeout of 300 seconds. 10:19:17 fatal: destination path 'D:\Je\wp\job1_1\test' already exists and is not an empty directory.
When checking the folder D:\Je\wp\job1_1\test. I see that the presence of the ".git" subfolder prevents cloning again.
and no errors appear in the folder deletion job (I found it here, but don't know if it's right)
https://github.com/seeraven/gitcache/blob/ec99dcf7c175279927aea198753ff7028071f95f/src/git_cache/command_execution.py#L117
The process tries 3 times before receiving an error result with return code 128.
Title says it all :)
git config --list
is not beeing routed to git
When using ctrl+c to cancel the operation of cloning from the remote, the following clone will always fail.
I found a db.lock at my GITCACHE_DIR,
Trace back as follows:
Traceback (most recent call last):
File "/gitcache/git", line 40, in <module>
handle_git_command(sys.argv[0:1], sys.argv[1:])
File "/gitcache/git_cache/git_command.py", line 95, in handle_git_command
sys.exit(git_clone(git_options))
File "/gitcache/git_cache/commands/clone.py", line 53, in git_clone
return mirror.clone_from_mirror(git_options)
File "/gitcache/git_cache/git_mirror.py", line 270, in clone_from_mirror
if not self.update(ref):
File "/gitcache/git_cache/git_mirror.py", line 150, in update
return self._update(ref)
File "/gitcache/git_cache/git_mirror.py", line 398, in _update
self.database.save_update_time(self.path)
File "/gitcache/git_cache/database.py", line 92, in save_update_time
self.database[path]["last-update-time"] = time.time()
KeyError: '/git_cache/mirrors/github.com/seeraven/gitcache'
I think it may be necessary to handle signal。
With make4py we have a common infrastructure for building python applications. So gitcache should be using it.
A valid git lfs fetch
command with arguments like
git lfs fetch origin --include somefile --exclude ""
gives error messages like
2021-10-12 07:33:08 Command 'cd xxx; /usr/bin/git -c lfs.storage=xxx lfs fetch origin origin' failed. Starting retry 1 of 3.
Invalid ref argument: [origin]
As one can see, the ref is appended although it was already specified.
In addition, we need to clarify what to do with the --include
and --exclude
arguments. They are currently removed but should IMHO be added as well.
When executing git clone ... -b branch
the last argument branch
is interpreted as the target directory. Although the man page of git specifies differently, it is apparently allowed to give options after the url or target directory!
To solve this issue, we should think about a general parser for the git commands, or at least get a understanding on how the original git command parser works.
The remote url of the same repository may slightly differ between gitcache invocations.
The use case: two different CI jobs running on the same machine (not even concurently, or anythign like that), but configured differently.
One is using the form of https://${GITHUB_PAT}@github.com/<repo>.git
and the second one is using [email protected]:<repo>.git
with ssh-agent setup running.
Because the remote url of the mirror repo is not updated, only the job who initially configured a mirror would work, and the other one would fail, since for instance having an ssh-agent for https://
url is useless, and the PAT token for ssh url is unusable as well.
We have a setup with multiple submodules (recursive) on an internal gitlab server.
Then those in turn load in submodules from github, usually libraries.
Those don't have to be updated very often while the local ones have to be updated on every build.
So, if you could add a string or regex as a config parameter that must match the URL for the cache to actually do its caching and pass everything else through would be awesome.
Default this parameter could be "" aka don't care.
We have a repository with various branches, but none called "master". It looks like that branch name is getting added and causing checkouts to fail:
2023-11-09 09:59:08 Command 'C:\Users\afabian\AppData\Local\Programs\Git\cmd\git.exe -c lfs.storage=c:\gitcache\cache\mirrors\REPO_PATH\lfs lfs fetch origin master' failed with return code 2. Starting retry 2 of 3.
Invalid ref argument: [master]
I think the expected behavior is to check out the ref specified by the parent repository. Come to think of it, "master" would be wrong even if it did exist.
Edit: just realized I said "checkouts" but the actual command was "submodule update --init --recursive"
Thank you for publishing great software.
I tried to use gitcache
for my work and I found git submodule update --recursive --init
doesn't work. Here is a reproducing script and its result. Do you have any ideas on this behavior?
> cat gitcache-val.sh
#! /bin/bash -eu
REAL_GIT_DIR1=real_git_dir1
REAL_GIT_DIR2=real_git_dir2
GITCACHE_DIR1=gitcache_dir1
GITCACHE_DIR2=gitcache_dir2
rm -rf ${REAL_GIT_DIR1}
rm -rf ${REAL_GIT_DIR2}
rm -rf ${GITCACHE_DIR1}
rm -rf ${GITCACHE_DIR2}
mkdir ${REAL_GIT_DIR1}
pushd ${REAL_GIT_DIR1}
/usr/bin/git clone https://github.com/aws/aws-sdk-cpp.git --single-branch --branch 1.9.188
pushd aws-sdk-cpp
/usr/bin/git submodule update --recursive --init
popd
popd
mkdir ${REAL_GIT_DIR2}
pushd ${REAL_GIT_DIR2}
/usr/bin/git clone https://github.com/aws/aws-sdk-cpp.git --single-branch --branch 1.9.188
pushd aws-sdk-cpp
/usr/bin/git submodule update --recursive --init
popd
popd
mkdir ${GITCACHE_DIR1}
pushd ${GITCACHE_DIR1}
git clone https://github.com/aws/aws-sdk-cpp.git --single-branch --branch 1.9.188
pushd aws-sdk-cpp
git submodule update --recursive --init
popd
popd
mkdir ${GITCACHE_DIR2}
pushd ${GITCACHE_DIR2}
git clone https://github.com/aws/aws-sdk-cpp.git --single-branch --branch 1.9.188
pushd aws-sdk-cpp
git submodule update --recursive --init
popd
popd
echo ========== Number of diff of real_git_dir1 and real_git_dir2 ==========
diff -rq real_git_dir1/aws-sdk-cpp real_git_dir2/aws-sdk-cpp | grep -v "[.]git" | wc -l
echo ========== Number of diff of real_git_dir1 and gitcache_dir1 ==========
diff -rq real_git_dir1/aws-sdk-cpp gitcache_dir1/aws-sdk-cpp | grep -v "[.]git" | wc -l
echo ========== Number of diff of gitcache_dir1 and gitcache_dir2 ==========
diff -rq gitcache_dir1/aws-sdk-cpp gitcache_dir2/aws-sdk-cpp | grep -v "[.]git" | wc -l
> ./gitcache-val.sh
========== Number of diff of real_git_dir1 and real_git_dir2 ==========
0
========== Number of diff of real_git_dir1 and gitcache_dir1 ==========
218
========== Number of diff of gitcache_dir1 and gitcache_dir2 ==========
0
The recommended setup consists of a symlink called git
(or git.exe
on Windows) targeting the gitcache
executable. Such a setup should be covered by the functional tests which are currently calling gitcache
resp. gitcache git
.
The git lfs pull
command is not handled by gitcache, as it was not necessary with previous versions of git-lfs. However, the
behaviour of git-lfs has changed and leads to errors when the remote (fetch URL) is set to the mirror. When the remote fetch URL is set to the remote repository, git lfs pull
works as expected. This happens only if there are files to fetch, e.g., files that are filtered by default but are included using the command line options --include
and --exclude ""
.
So the git lfs pull
commands should be handled by gitcache in the following way:
origin
, proceed with the original git command.An alternative would be to setup a temporary remote like real-origin
and use that remote. This would have the advantage of being more robust in cases where the command is interrupted and leaving the temporary fetch URL.
I just want to say this tool works extremely well and I am very grateful for it!
Hi,
I have been using/trying gitcache
for a couple of days already and I have noticed that in some cases is slower that plain git
.
I'm trying to use gitcache
with Terragrunt/Atlantis to avoid cloning the same repo hundreds of times, but since Terragrunt uses git rev-parse --show-toplevel
a lot it is being slower than plain git.
The start up time before showing any debug is considerable (In the order of 3-4 seconds) for an already cloned repo:
bash-5.2$ export GITCACHE_UPDATE_INTERVAL=10d
bash-5.2$ time GITCACHE_LOGLEVEL=Debug /usr/local/bin/git clone [email protected]:seeraven/gitcache.git
2024-05-10 10:05:43 Python executable: /usr/local/bin/git
2024-05-10 10:05:43 Called as ['/usr/local/bin/git', 'clone', '[email protected]:seeraven/gitcache.git']
2024-05-10 10:05:43 handle_git_command(['/usr/local/bin/git'], ['clone', '[email protected]:seeraven/gitcache.git']) started
2024-05-10 10:05:43 Found global options [], command 'clone', command options '[]' and arguments ['[email protected]:seeraven/gitcache.git'].
2024-05-10 10:05:43 Loading configuration file /Users/andresmontalban/.gitcache/config.
2024-05-10 10:05:43 Loading configuration file /Users/andresmontalban/.gitcache/config.
2024-05-10 10:05:43 Can't load configuration file /Users/andresmontalban/.gitcache/mirrors/github.com/seeraven/gitcache/gitcache.config as it does not exist!
2024-05-10 10:05:43 Update time of mirror /Users/andresmontalban/.gitcache/mirrors/github.com/seeraven/gitcache not reached yet.
2024-05-10 10:05:43 Starting Clone from mirror /Users/andresmontalban/.gitcache/mirrors/github.com/seeraven/gitcache.
2024-05-10 10:05:43 Retry to execute command '/usr/bin/git -c [email protected]:seeraven/gitcache.git/info/lfs -c lfs.storage=/Users/andresmontalban/.gitcache/mirrors/github.com/seeraven/gitcache/lfs clone /Users/andresmontalban/.gitcache/mirrors/github.com/seeraven/gitcache/git gitcache' up to 3 times.
2024-05-10 10:05:43 Execute command '/usr/bin/git -c [email protected]:seeraven/gitcache.git/info/lfs -c lfs.storage=/Users/andresmontalban/.gitcache/mirrors/github.com/seeraven/gitcache/lfs clone /Users/andresmontalban/.gitcache/mirrors/github.com/seeraven/gitcache/git gitcache' (shell=False, cwd=None) with command timeout of 3600 seconds and output timeout of 300 seconds. stderr_capture=True
Cloning into 'gitcache'...
done.
2024-05-10 10:05:43 Command '/usr/bin/git -c [email protected]:seeraven/gitcache.git/info/lfs -c lfs.storage=/Users/andresmontalban/.gitcache/mirrors/github.com/seeraven/gitcache/lfs clone /Users/andresmontalban/.gitcache/mirrors/github.com/seeraven/gitcache/git gitcache' finished with return code 0.
2024-05-10 10:05:43 Clone from mirror /Users/andresmontalban/.gitcache/mirrors/github.com/seeraven/gitcache was successfully completed within 0.1 seconds.
2024-05-10 10:05:43 Setting push URL to [email protected]:seeraven/gitcache.git and configure LFS.
real 0m5.843s
user 0m0.295s
sys 0m0.168s
If I do not setup GITCACHE_LOGLEVEL=Debug
it takes longer because it has to update the clone.
On the other hand if I use plain git I get faster clone:
bash-5.2$ time git clone [email protected]:seeraven/gitcache.git
Cloning into 'gitcache'...
remote: Enumerating objects: 1122, done.
remote: Counting objects: 100% (150/150), done.
remote: Compressing objects: 100% (59/59), done.
remote: Total 1122 (delta 101), reused 114 (delta 84), pack-reused 972
Receiving objects: 100% (1122/1122), 291.03 KiB | 483.00 KiB/s, done.
Resolving deltas: 100% (729/729), done.
real 0m3.956s
user 0m0.160s
sys 0m0.088s
Also simple commands like git rev-parse --show-toplevel
are slow to execute:
bash-5.2$ time /usr/local/bin/git rev-parse --show-toplevel
/Users/andresmontalban/code/gitcache
real 0m5.742s
user 0m0.278s
sys 0m0.131s
bash-5.2$ time git rev-parse --show-toplevel
/Users/andresmontalban/code/gitcache
real 0m0.045s
user 0m0.007s
sys 0m0.013s
For these tests I'm running gitcache
on a Macbook Pro with M1 Pro CPU, but I'm getting similar results in an AWS instance with 16 cores.
Any idea on how to make it faster?
Thank you very in advance for your help!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.