Comments (7)
Sry if I misunderstand - I am still new to ClearML 🙂 . Though is the commit you referenced @bmartinn really addressing the issue that @milongo mentioned?
If I see correctly, the referenced commit rewrites how https git repo urls are translated into ssh repo urls. But the issue raised by you @milongo is about 'userless' ssh clone links, right?
At least we have the issue that running ClearML in our git repo creates links like:
ssh://my_server_url:<ssh_port>/<user>/<repo>.git
and we do not want to fix them all by hand. Alternative solutions could be changing what repo url is written when a Task
is initialized, but solving this 'agent-side' would be better.
from clearml-agent.
OK I found two workarounds, both rather hacky. If you need a solution, you can use it, but I personally would prefer a cleaner way.
Both rely on having a ~/.ssh/config
file like the one below:
Host git.mycompany.com
HostName 123.123.123.123
IdentityFile ~/.ssh/id_rsa
IdentitiesOnly yes
User git
Port 2022
One gotcha is that you need to spell out the domain as in the ssh url. People often use abbreviations like Host gitserver
and then state the domain as HostName
. However in this case, you need the full domain as in the url as Host
and the IP of the server (might work with DNS, have not tested) as HostName
.
You can test it outside of the docker container by running e.g. git clone ssh://git.mycompany.com:2022/myuser/repo.git
. This should work outside of the container.
For making this available in the docker container, we have two options.
Option 1: Own config/key with root, mount it manually
The docker containers executed by the clearML agent run as root
. Without any changes, ssh
will complain about a bad owner of the config file, even when file permissions are 600
. To circumvent this, you can own the config and key as root
. Outside of the docker container:
sudo chown root:root ~/.ssh/config && sudo chown root:root ~/.ssh/id_rsa
Though now you will get an error (when running the clearML agent with --foreground
) because your user can no longer copy the whole .ssh
folder:
Failed creating temporary copy of ~/.ssh for git credential
So in your clearml.conf
, you need to mount the credentials manually:
agent: {
...
extra_docker_arguments: ["-v", "/home/datapipelineuser/.ssh:/root/.ssh"]
...
}
Option 2: Own the ssh credentials in the container
Instead of owning the credentials outside and mounting them manually, you can add a command to own the credentials in the container. Adding
agent: {
...
extra_docker_shell_script: ["chown -R root:root /root/.ssh"]
...
}
to your clearml.conf
should do the trick. I prefer the second solution, but overall, the best would be if we could either have a user set when creating the task or overriding the user even when a ssh://
domain is given.
from clearml-agent.
Thanks @milongo, you are correct there is currently no way to configure the git user the SSH clone is using
(well other than changing the original repository link in the Task, but that will scale badly)
I think we should add a configuration argument for the agent
section in the ~/clearml.conf
file
- Since this issue is actually a feature request for
clearml-agent
, I'll make sure we move it to the correct repository - I'll update once we have an RC with a fix
from clearml-agent.
Hi @milongo
Good news the new RC includes this feature.
Add to your ~/clearml.conf
the following line:
agent.force_git_ssh_user = "git"
And upgrade the clearml-agent
to the latest RC:
pip install clearml-agent==0.17.2rc2
from clearml-agent.
@bmartinn should this work agent-side in clearml-agent==1.0.0
?
I have:
$ pip freeze | grep clearml
clearml==1.0.4
clearml-agent==1.0.0
and in clearml.conf
:
...
agent: {
force_git_ssh_user: "git"
force_git_ssh_protocol: true
default_docker {
image: "my_image:latest"
}
...
}
Now after running Task.init
locally, I see in the UI this URL pattern:
ssh://my_server_url:<ssh_port>/<user>/<repo>.git
and the execution fails with:
...
cloning: ssh://my_server_url:<ssh_port>/<user>/<repo>.git
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
...
Should this work? After adding force_git_ssh_user: "git"
, should I instead see cloning: ssh://git@my_server_url:<ssh_port>/<user>/<repo>.git
, or is that handled somewhere in the backend?
EDIT/Sidenote: When updating the repo URL with the user in the UI, it works. So I do not think it is a general permission/misconfiguration issue.
from clearml-agent.
@milongo can we close this?
from clearml-agent.
Hi @jkhenning, we can! I'm closing it now.
from clearml-agent.
Related Issues (20)
- Warning: could not locate requested Python version 3.11, reverting to version 3.11 HOT 2
- Feature request: docker pruning/post execution action
- Environment variables not applied HOT 5
- Clearml Task: pip install order HOT 5
- clearml_agent: ERROR: 'utf-8' codec can't decode byte 0xfc in position 38: invalid start byte HOT 1
- Update docker bases to ubuntu 20 (or 22), and newer python HOT 7
- Cloned experiment ignores arguments HOT 5
- Modify clearml-agent to accept urlib>=2 as a dependency.
- poetry_install_extra_args passes arguments to poetry config HOT 1
- environment variables in default_docker arguments of clearml.conf not passed to container on first run HOT 2
- Yolo Execution with GPU HOT 4
- RAM / CPU cores partitioning for multiple agents on the same machine HOT 1
- Issue of checkout PR commit by sha HOT 1
- Image on Docker Hub is out of date HOT 12
- no module named "virtualenv" with execute_remotely HOT 5
- clearml-agent build not building a docker image HOT 10
- shh to http conversion fails with dev.azure HOT 2
- Run in a docker mode not passing envs (DIND) HOT 2
- gnutls_handshake() failed: An unexpected TLS HOT 4
- The cmd clearml-agent daemon stop marked ongoing Task as completed
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from clearml-agent.