Comments (13)
As update on the IRC discussion Sep 12 it is important to consider that ssh could have different styles on different remotes, as example:
Consider a company who pulls code from both the internal git servers, as well as different remote sources, could have an internal manifest similar to:
remotes:
- name: zephyrproject-rtos
url: https://github.com/zephyrproject-rtos
ssh: ssh://[email protected]/zephyrproject-rtos
-name: remote-gerrit
url: https://remote-gerrit.somewhere.com/gerrit
ssh: ssh://[email protected]:29418
-name: mycompany-bitbucket
url: https://mycomp/bitbucket
ssh: ssh://[email protected]:1234/
projects:
-name: proj1.git
remote: zephyrproject-rtos
-name: proj2.git
remote: remote-gerrit
-name: proj3.git
remote: mycompany-bitbucket
As there can be multiple remotes in the same company, and those can use different user@server syntax, e.g. git and bitbucket seems to use git@server, whereas if I remember correctly from when I worked with gerrit, I think the default syntax there is ssh@server.
But this is most likely configurable.
Also git-servers tends to use different ports.
All of this this may make it trickier to make the solution with just an overlay on command line.
So splitting to allow to specify bort a url and a ssh on remotes that supports both could be a clear way/
Then on west fetch
it can be specified what is preferred.
from west.
As discussed on IRC, I think the URL specifying the transport is a feature, not a bug, and that a potentially better way to handle this would be through adding support for "manifest overlays" similar to device tree overlays; these would be files in west/manifest-overlays/xxx.yml
which could be used to override the behavior in the main manifest.
With that feature, you could write a file west/manifest-overlays/remotes.yml
containing this:
manifest:
remotes:
- name: zephyrproject
- url: [email protected]:zephyrproject-rtos
The main manifest entry fragment from west/manifest/default.yml
:
remotes:
- name: zephyrproject
url: https://github.com/zephyrproject-rtos
Would then have its URL overridden from the https:// to the git+ssh version.
To ensure determinism, overlays would be applied in sorted order, allowing users to name them 00-first.yml, 01-second.yml, etc. as desired.
Seem reasonable?
from west.
sounds reasonable so far, although I need to see the full picture.
To avoid everyone having to write their own overlays, I can then create a repo, containing both:
default.yml
remotes:
- name: zephyrproject
url: https://github.com/zephyrproject-rtos
and an overlay: 00-ssh-remote.yml:
manifest:
remotes:
- name: zephyrproject
- url: [email protected]:zephyrproject-rtos
then how does the user of west
decides that the overlay should / should not be used ?
I assume, as an argument on the command line, like:
west --overlay 00-ssh-remote.yml
from west.
To avoid everyone having to write their own overlays, I can then create a repo, containing both:
If you're creating your own repo, why not just update the remote URLs in the main manifest? You are already committing to maintaining a copy at that point.
from west.
The point that I'm trying to make, is that Github itself provides to possibility to choose between:
- https
- ssh
Therefore there is a possibility for any github developer to use any of the two (and they might have a personal preference for one of the methods).
As seen in just our little reference group, there is already two who default seems to be using SSH.
Therefore, I believe it would make sense that both the https
and ssh
where described in the manifest for Zephyr so that developers can easily choose their preferred method.
(defaulting to https version is fine with me)
As well as other companies forking Zephyr has the ability to described both a https
and ssh
version in their manifest.
The overlay proposal as I understand, means that everyone who prefers ssh
would need to write that overlay, correct ?
from west.
CC @nashif
from west.
@mbolivar did some thinking on this.
remotes:
- name: <name-of-remote>
url: <default-url-of-remote>[;<optional-url>]
as example:
remotes:
- name: remotes:
url: https://github.com/zephyrproject-rtos;ssh://[email protected]/zephyrproject-rtos
then default url and transport will be first in list, but user can request other preference, e.g.
west pull --transport ssh
which will use ssh is described.
If not available, it will just default to first in list.
from west.
url: <default-url-of-remote>[;<optional-url>]
Hm, I see what you mean.
I think it would be better to use a real YAML list for this, though. Semicolons are reserved sub-delimiters in URLs. Using the native list type also lets us keep pykwalify schema checking, and add more metadata.
Something like:
remotes:
- name: my-cool-remote
urls:
- transport: git
url-base: git://my-cool-remote.com/git
- transport: git+ssh
url-base: [email protected]:git
Commentary:
- The
urls
key in a remote entry would be in addition to the currenturl
, not a replacement. - If the user chooses to use
url
, the value would be either a simple string like today or atransport: x/ base: y
dictionary like an entry inurls
. - It would be an error to specify both
urls
andurl
for a single remote. - If
url-base
is too verbose, we could shorten it tourl
orbase
. - I think the
transport
key has some problems (it mixes the VCS, like git, with the protocol used by the VCS, like SSH) and deserves more thought, especially since we've been asked to support non-git systems.
To clarify, the following url
fragments would both be valid after this change:
url: https://my-cool-remote.com/git
url:
transport: git+https
url-base: https://my-cool-remote.com/git
We would need to come up with some default assumptions about the transport
if not given in a url
. This is another reason I think it is not quite right.
west pull --transport ssh
which will use ssh is described.
Perhaps this could be a configuration option in addition to a manifest defaults
entry. I think this is more of a permanent setting than something to specify on an invocation-by-invocation basis.
In defaults
:
defaults:
- transport: git+https
In west/config
, something like:
[manifest.defaults]
transport = git+https
from west.
url: <default-url-of-remote>[;<optional-url>]
Hm, I see what you mean.
I think it would be better to use a real YAML list for this, though. Semicolons are reserved sub-delimiters in URLs.
Correct, but that could be handled by requiring such characters to be escaped. Just as you would need to do if trying to use a url containing ;
on command line in a shell.
In a shell the semi-colon would be treated as separation of commands unless escaped.
Using the native list type also lets us keep pykwalify schema checking, and add more metadata.
Now, that is an argument I accept ;)
Sticking to default yaml might be best option.
Been reading up on yaml, and seems this is an allowed sequence, close to your example:
remotes:
- name: my-cool-remote
urls:
- git://my-cool-remote.com/git
- [email protected]:git
- ssh://[email protected]/git
- https://my-cool-remote.com/git
and we don't need to specify transport, as it is given by the URL:
git://
and [email protected]:git
defines git transport (technically ssh without encryption)
ssh://
defines ssh
https://
defines https
west pull --transport ssh which will use ssh is described.
Perhaps this could be a configuration option in addition to a manifest defaults entry. I think this is more of a permanent setting than something to specify on an invocation-by-invocation basis.
it could be provided west init
, like west init --transport git;https;ssh
and then written to west/config
Runing re-init or writing to west/config
can thus be used to update the setting.
Question: should we consider a west config
command for the future, as I personally don't like to put this with west init
If no setting is given or the transport is not in urls
, the first item in urls
sequence is picked.
from west.
and we don't need to specify transport, as it is given by the URL
Sorry again for choosing such a poor name as "transport". I'm trying with this field to specify both the VCS (git, for now) and the actual network transport. So I'm mixing application layer and L4 in the OSI sense in my above proposal, which is why I think it's not quite right.
My point is that if we are going to make this change, we should do so in a way that allows the schema to encode non-git source access methods, even if west itself cannot handle anything other than git, in order to avoid a schema compatibility break later.
It should also be noted that /some/path and file:///some/path are treated differently by Git - the former is more efficient for remotes on systems which support hard links. So some non-URL encodings may be desirable.
from west.
@tejlmand @carlescufi I found a workaround you might be interested in:
https://wiki.archlinux.org/index.php/git#Protocol_defaults
from west.
@tejlmand I am going to close this since west has been available for years and we still haven't discovered a compelling use case for it.
from west.
#615 looks similar.
from west.
Related Issues (20)
- Support `west manifest --resolve` and `--freeze` with `manifest.project-filter` HOT 1
- Follow up work for `manifest.project-filter` implementation
- west init: CLI argument to automatically setup new workspace from template HOT 11
- `already defined as extension command`-error prints wrong spec HOT 1
- Introduce absolute path variant of zephyr.base HOT 13
- west cannot process a git branch name containing a single quote ' HOT 8
- west init access denied on windows HOT 2
- git: Add support for sparse checkout HOT 5
- Investigate if the performance of submodule update could be improved HOT 7
- west update loops infinitely over the first repository HOT 14
- West Re-Implementation - git ws
- west update - AttributeError: 'NoneType' object has no attribute 'err' HOT 1
- menuconfig aborting due to Kconfig warnings HOT 4
- Allow using reference repositories to share objects HOT 8
- Allow import of optional projects from manifest file HOT 7
- Moving from Zephyr 3.2.99 to 3.5.99: `west` cannot find its configuration file HOT 3
- --mr documents argument to be a revision HOT 1
- `pip install` fails on MSYS2 HOT 3
- Infinite loop when building in a moved directory HOT 4
- `die_if_no_git` not listed under west API HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from west.