Comments (3)
@sgotti, I would appreciate if you will have time to take a look. Thanks!
from agola.
@kvaster Thanks.
you need to create separate pod and save cloned repository to workspace - wasting resources.
This is true only if you have a single task. If you have multiple tasks (like a build matrix) it'll be better to clone one time only, save to workspace and restore in the other tasks.
So to summarize and to split it in multiple parts:
- Add ability to have multiple "steps" container and reference them in the step definition. If no container is referenced in the step then the primary one is used.
There're multiple issues to face:
- It'll require having a shared volume between "step containers". Where will this shared volume should be mounted? There're some things to consider:
- Currently we can define a working dir (defaulting to ~/project if not defined) in different places (task, run step). The working dir is then expanded by the executor at the step execution time to resolve possible
~
(usingtoolbox expanddir
) . So the working dir full path depends on the current step user. We cannot assume it'll be root since it could be defined in different places (container, task) - The working dir is created by the executor using the task user and the task working dir (using
toolbox mkdir
) - So we cannot by default choose a "shared dir" related to the working dir also if it's the current default (~/project) since it's not fixed but depends by multiple variables.
- For the same reason, since we have to mount volume at the container creation time we cannot know the working_dir at that time.
- Currently we can define a working dir (defaulting to ~/project if not defined) in different places (task, run step). The working dir is then expanded by the executor at the step execution time to resolve possible
- We cannot assume the step executing user will be root.
- Now the executor determines the step user by looking at the task user (username or uid), then the container user (username or uid) or it'll be passed as empty to the executor that will use the image default user. If we have multiple "step" container we should explicitly document that we are using the primary step container user and that it must exist on all step containers and have the same uid
Possible solution:
-
For single "step" container configs do nothing (to not impact existing configs).
-
For multiple "step" container configs:
- Change the default working_dir to a fixed full path (like /opt/project).
- Add an option to define the task shared volume mount path with a default as the default working_dir (like /opt/project).
- The executor will create a volume and tell the executor driver to mount it to the shared volume path in all the "step" containers.
- One thing to check is which permission the volume will have (may depend on executor type) and make possible that it'll be accessible by the task user (we could probably need to find a way to chown the mounted dir to the task user of the primary container)
-
Add a lot of tests to check all the possible cases.
- Use 'clone' container by default for 'clone' step.
- Use 'alpine-git' as default 'clone' container if there is no explicitly defined 'clone' container.
I'll avoid adding a default definition of a "clone" container and making it the default for the clone step since it'll break current configs (a user can have its custom image with its binaries and this will change it)
from agola.
Thanks. I'll try to prepare changes.
from agola.
Related Issues (20)
- What happened to the DockerHub images? HOT 1
- Optional authentication for internal services HOT 1
- Clone step: use internal git HOT 1
- notification service: implement logic to start from the last run event triggered HOT 2
- notification service: save webhook and deliverys HOT 2
- Optimize run scheduling algorithm for improved CI/CD efficiency
- gateway service: order projects by name
- Add pagination to list APIs. HOT 10
- notification: add apis to manage run webhooks retry logic
- Organization rules for runs start/stop HOT 10
- notification service: save commit status and deliveries
- Global view of running runs HOT 3
- Support obtaining OIDC ID token from Git server
- variables and secrets are not loaded in building environment HOT 8
- More convenient to use HOT 1
- Agola CPU spikes to 120% and goes into tight futex/nanosleep loop after some time HOT 7
- Couldnt get examples/agolademo to pull from docker HOT 1
- Checkout git repo fully, without switching to specific commit HOT 1
- External git repos in workflows HOT 1
- Add support for colored text (ANSI escape sequences) within the logs HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from agola.