devfile / api Goto Github PK
View Code? Open in Web Editor NEWKube-native API for cloud development workspaces specification
License: Apache License 2.0
Kube-native API for cloud development workspaces specification
License: Apache License 2.0
In a devfile that is being used to define a stack, there are certain files that the stack may be looking for to decide if a given project can be run by a given stack or not, e.g. a maven stack may look for the pom.xml and a node.js stack may look for package.json.
If we can provide a way to allow a given stack to define those required resources, we can use it in tools like Che or odo to do quick check on project to suggest the stacks on a devfile repo that may support the a given application. An example can be:
parent.yaml
---
schemaVersion: 2.0.0
metadata:
publisher: my-org
name: maven-stack
version: x.y.z
matchingRules:
existingFiles: ['pom.xml', 'aaa.xml']
devfile.yaml
---
schemaVersion: 2.0.0
metadata:
name: my-maven-project
parent:
id: organization/maven-stack/x.y.z
...
There may be other types other than existingFiles
that we can support in the future.
We need to agree on a convention to specify the step(s) of the development lifecycle a command belongs to. For example a given command will be used during the build
step and another one is used for the run
step. Tools like odo
and che
will then leverage this information to infer the sequence of the commands that need to be run.
A possible convention we can agree on is to specify the lifecycleStep
as command annotation:
commands:
- exec:
annotations:
lifecycleStep: build
commandLine: "./build.sh"
component: build-container
- exec:
annotations:
lifecycleStep: run
commandLine: "./run.sh"
component: runtime-container
In previous version we used to have a apiVersion
attribute. We would like to rename it schemaVersion
. The main reason is because the devfile format versioning doesn't adhere to the Kubernetes versioning scheme and calling it schemaVersion
will avoid confusions.
Handling of resources that comes from the parent devfile. Things to do:
Could we support the ${STACK_ROOT} variable to be able to express that a specific artifact comes with the stack and not necessarily with the source code/project ?
Example, I would like to specify that a Dockerfile build be done using the ${STACK_ROOT}/Dockerfile
and not the ${PROJECTS_ROOT}/Dockerfile.build .
Such artifacts/files would be packaged and served off the devfile directory, example
EDIT:
This is how tools could implement it : #55
Target Date: 09-09-2022
When running exec commands from a devFile within a container we may need to run the commands as a different user to the user that is the default user within the container. I am therefore proposing we have a runAsUser
within the exec
, for example:
commands:
- exec:
id: init-deps
name: init image dependancies
component: java-openliberty-build
commandLine: /project/bin/init-stack.sh
runAsUser: root
- exec:
id: stack-prep
name: stack-prep
component: java-openliberty-build
commandLine: /project/bin/run-stack.sh prep
runAsUser: java_user
- composite:
id: setupContainer
commands:
- init-deps
- stack-prep
parallel: false
group:
name: init
isDefault: true
So in this example there is a need to run some setup that requires to be run as root
, and then some other setup that needs to be run as java_user
.
It is not easy to switch userids within a shell script (bash) then different shell scripts need to be run and need to be run as different userids.
The outer-loop guidance for tools would be added in devfile 2.2.0. This is a work-in-progress epic.
Examples in the doc
It should be published on https://github.com/redhat-developer/devfile/tree/master/docs on 2.0.0
branch
The traditional way of customizing an existing container image is to create a new container by extending that existing container as a parent image. For example, the java-maven dev file uses quay.io/eclipse/che-java11-maven:nightly to pull in Maven, rather than using the standard Maven docker image (hub.docker.com/_/maven).
However, this approach requires creating a ‘child’ image for every parent image that is to be used in Che (or at least those requiring customization).
This ‘nice to have’ feature proposes a second option for customizing containers: when a container is initially created in Che (from a devfile), files that are specified by an environment value can be automatically copied (on startup) into the container at specific locations using an initContainer
component.
As proposed, a devfile writer is free to either create parent image, or to customize an existing image using an initContainer
. This feature is not a replacement, it is just another tool in the devfile-writers arsenal, to make it easier to customize individual files/directories in container without fully extending them with a new image.
Example
components:
- container:
image: maven:3.6.3-jdk-11
volumeMounts:
- name: scripts
mountPath: "/home/user/scripts"
- initContainer:
name: cp
image: quay.io/devfile/cp"
volumeMounts:
- name: scripts
mountPath: "/scripts"
env:
- name: SRC_PATHS
value: "/src/resources/scripts/build.sh, /scripts2"
- name: DEST_PATHS
value: "/scripts, /scripts"
- volume:
name: scripts
emptyDir: {}
In this example, the container will be based on the standard maven image, but additional an additional script file (build.sh) and an additional directory (/scripts2) will be copied into the container at startup.
The source for both of these files would be the root of the project sources (where the devfile is located).
For example: If the dev file were hosted here then build.sh would be hosted here
<repo root>
├── devfile.yaml
├── scripts2
└── resources
└── scripts
└── build.sh
With this feature, it is now possible to use an existing container image in Che, while still retaining the ability to add additional required files.
I was wondering rather than specifying a custom image in component container like this:
commands:
- exec:
id: buildSchema
label: Build the schema
commandLine: "./buildSchema.sh"
component: build-tools
group:
kind: build
isDefault: true
components:
- container:
image: "maysunfaisal/myImageWithBuildSchema.sh"
mountSources: true
sourceMapping: /home/src
name: "build-tools"
if we should support syncing of build artifact(s) buildSchema.sh
to the component container with a base image, for example maven:latest
rather than providing maysunfaisal/myImageWithBuildSchema.sh
.
This way, a devfile creator would not require the need to build a custom image with these artifacts burned in.
Right now, there isn’t a way to specify the size of a component’s volume in a devfile, and Che will use the size specified in che.properties (default 1Gi). As different component-types may have different volume requirements (e.g. the issue seen in redhat-developer/odo#2455), Odo will have a need to specify the size of volumes it interacts with, and rather than having an odo-specific setting, it would be nice if we could store this setting in the devfile,
This change might also be nice for Che, as it would allow components to configure the size of volumes they need (such as a maven cache volume that requires more than 1Gi), rather than relying on global Che defaults.
Something like:
---
components:
- container:
image: "maven"
name: "maven"
volumesMount:
- name: my-data-volume
- volume:
name: my-data-volume
size: 5Gi
Example with proposed changes:
commands:
- exec:
commandLine: "./buildSchema.sh"
workingDir: "~/schemaScripts"
component: build-tools
alias: buildSchema
Adding optional metadata version
and name
:
schemaVersion: 2.0.0
metadata:
name: petclinic-dev-environment # optional
version: x.y.z # optional
projects:
...
components:
...
commands:
...
Since in the schema the metadata are optional they can even be extracted in a meta.yaml as it is today in the devfile-registry but that's not covered in the spec.
schemaVersion
. And the semantic is the semVer one.
When you use a devfile in an IDE, the IDE might want to be able to display a devfile-specific welcome page. Likewise, a command line tool might want to link to information on how to use the devfile. In both of these cases we want the devfile to provide the link and not pull other files into the user's project, which could be easily removed or annoying to clean up.
For cases like these it would be nice to have an optional doc url in the devfile metadata.
The devworkspace status field is currently lacking for many use cases. I'd like to propose adding a few fields that are common to other k8s objects in order to support management better.
New fields:
.status.conditions
- Type corev1.Condition[]
, stores information about various aspects of workspace state (c.f. .status.conditions
on Pods.).status.phase
- String valued, stores current "phase" of workspace (e.g. running
, started
, stopped
, failed
)The struct for conditions would be the common Kubernetes condition struct type.
Conditions can be used to share information about progress in a workspace start process -- e.g. "storage ready", "deployment created", "deployment ready". They can also be used to give more detailed information about the last observed state of a workspace (e.g. the workspace deployment isn't ready because we're pulling images)
Phase is useful to track the status of the workspace -- if e.g. a routing class that is unsupported is specified, a workspace can be marked as "failed". If another service is waiting to show mainIdeUrl
to the user, it needs to know the workspace is in a running state.
status:
condition:
- lastTransitionTime: "2020-05-06T23:21:55Z"
status: "True"
type: ServiceAccountReady
- lastTransitionTime: "2020-05-06T23:21:55Z"
status: "True"
type: RoutingReady
- lastTransitionTime: "2020-05-06T23:21:55Z"
status: "True"
type: DeploymentReady
reason: "OneWordReasonForTransition"
message: "User-readable message about transition"
- lastTransitionTime: "2020-05-06T23:21:55Z"
status: "True"
type: ComponentsReady
phase: Running
mainIdeUrl: https://my-workspace.my-cluster.com
workspaceId: workspace-id
additionalInfo: {}
Supported Phases:
Running
: Workspace is fully operational and can be accessedStarting
: Workspace is currently being startedStopped
: Workspace exists but is not runningStopping
: Workspace is currently stopping (potentially waiting on preStop actions, e.g.Failed
: There is an issue preventing this workspace from entering the Running
phase.For conditions, I don't have a concrete proposal for categories. Currently, in the che-workspace-operator (which would adopt this API), we use
const (
WorkspaceComponentsReady WorkspaceConditionType = "ComponentsReady"
WorkspaceRoutingReady WorkspaceConditionType = "RoutingReady"
WorkspaceServiceAccountReady WorkspaceConditionType = "ServiceAccountReady"
WorkspaceDeploymentReady WorkspaceConditionType = "DeploymentReady"
)
but I don't view this list as complete.
Feature request: A mechanism to define/access external dependencies in the workspace. This would consist of links to external dependencies so that a developer could easily access those dependencies directly from the context of the workspace.
External dependencies could be url links to any resouce on the web. Common uses that I foresee are:
Proposed externalDependency
component definition in the defile.yaml
:
apiVersion: 1.0.0
metadata:
name: My-Workspace
projects:
- name: My-App
source:
location: 'https://github.com/triceam/my-sample-app-repo'
type: git
components:
- id: che-incubator/typescript/latest
memoryLimit: 512Mi
type: chePlugin
- mountSources: true
endpoints:
- name: nodejs
port: 3000
memoryLimit: 512Mi
type: dockerimage
alias: nodejs
image: 'quay.io/eclipse/che-nodejs10-ubi:7.8.0'
- name: IBM Watson Visual Recognition
url: http://cloud.ibm.com/watson/instance
type: externalDependency
- name: IBM Cloud Continuous Delivery
url: http://cloud.ibm.com/devops/instance
type: externalDependency
- name: My on-prem Database
url: http://url.to.my.database/foo
type: externalDependency
- name: My on-prem OpenShift cluster
url: http://url.to.my.openshift/dashboard
type: externalDependency
Proposed visualization in the Eclipse Che workspace:
Clicking on the external dependency link would open a new tab or window to access the external dependency.
As a stack author, I wish to provide outer-loop deployment guidance in the form of a podTemplate. The tools may offer different choices based on that : pod, Deployment, KnativeService, StatefulSet, DeploymentConfig, etc.
---
components:
- id: deploy-node-app
podTemplate:
containers:
- command: ['sh', '-c', 'echo "Hello, Kubernetes!" && sleep 3600']
readinessProbe:
httpGet: ""
path: /api/health/readiness
port: 8080
scheme: HTTP
When associated with the corresponding build guidance ( example #51 ), che/odo must make a smart kustomize-styled replacement of the image field with built image from the 'dockerfile' component.
- composite:
id: build-and-deploy
label: Build image and deploy nodejs app
commands:
- build-node-app
- deploy-node-image
parallel: false
group:
kind: deploy
(More of a question than a user story)
In devfile 1.0, we were able to set a dockerimage
component entrypoint command and args with the command
and args
field. A simple example might be:
-
type: dockerimage
image: docker.io/someimage:latest
alias: build
memoryLimit: 512Mi
command: ['tail']
args: [ '-f', '/dev/null']
While working on the Devfile 2.0 integration in odo, @adisky noticed there weren't equivalent fields for command and args in the container component, and I couldn't seem to find anything in the devfile 2.0 spec either:
Do we need those fields in devfile 2.0 (and therefore don't allow people to override a container's entrypoint) or was that an omission from the spec?
As a stack/devfile author, I wish to convey outer-loop deployment guidance in the form of a list of parameterized kubernetes objects.
The format draws inspiration from Templates https://github.com/sbose78/nodejs-rest-http-crud/blob/master/.openshiftio/application.yaml#L38
---
components:
- id: deploy-node-app
parameters:
- name: ${image}
value: quay.io/quarkus-quickstarts/getting-started-knative ## Default
objects:
- apiVersion: serving.knative.dev/v1alpha1
kind: Service
metadata:
name: greeter
spec:
template:
spec:
containers:
- image: ${image}
livenessProbe:
httpGet:
path: /health/live
readinessProbe:
httpGet:
path: /health/ready
If the objects
component is associated with a dockerfile
component (image build guidance) in a composite
command, #51 , che/odo must associate the built image from the 'build' guidance using the image
parameter.
- composite:
id: build-and-deploy
label: Build image and deploy nodejs app
commands:
- build-node-app
- deploy-node-app
parallel: false
group:
kind: deploy
Odo will have a need for multiple components to mount the same volume. There isn’t an option in devfiles right now to define a volume once, and have multiple components mount that same volume.
To work around this, we’re proposing adding a way to the devfile spec that allows you to define a volume once (a top level ‘volumes’ field), then easily add it to other components via a ‘volumeMount’ field. Like how Kubernetes handles volumes in containers, there'd be a top level volumes
field and a volumeMounts
field in the component:
projects:
- name: "my-project"
volumes:
- name: my-data-volume
size: 5Gi
components:
- container:
image: "busybox"
name: "mycontainer"
memoryLimit: "128M"
mountSources: true
volumeMount:
- name: my-data-volume
path: /some/folder
- container:
image: "busybox"
name: "mycontainer"
memoryLimit: "128M"
mountSources: true
volumeMount:
- name: my-data-volume
path: /some/other/folder
Current devfile 1.0 container.endpoints syntax (link to the doc):
components:
- alias: maven
type: dockerimage
image: eclipe/maven-jdk8:latest
endpoints:
- name: maven-server
port: 3101
attributes:
protocol: http
secure: 'true'
public: 'true'
discoverable: 'false'
Current syntax of che plugins (link to the doc):
spec:
endpoints:
- name: "theia"
public: true
targetPort: 3100
attributes:
protocol: http
type: ide
secure: true
cookiesAuthEnabled: true
discoverable: false
containers:
- name: theia-ide
image: "quay.io/eclipse/che-theia:7.10.0"
ports:
- exposedPort: 3100
The new type is container
:
components:
- container:
image: busybox
endpoints:
- name: petclinic-web-server
Possible alternative may be image
, containerImage
or ociImage
...
Currently, project source is mounted to either CHE_PROJECT_ROOT or /projects when mountSources is true. The CHE_PROJECT_ROOT value is configurable during Che setup via the che.properties file
If odo is to consume the devfiles, we would like it to have an alternate way to set the source path value in the component container, rather than having a fixed single value via either CHE_PROJECT_ROOT or /projects for all the devfiles.
This can be achieved by having sourceMapping when mountSources is true, which transfers/mounts the project to the specified path in the container.
Che can also use the property i.e.; when sourceMapping is present, it overrides the CHE_PROJECT_ROOT value if present and mounts the source to the path defined by sourceMapping
Example:
---
components:
- container:
image: "busybox"
name: "mycontainer"
mountSources: true
sourceMapping: "/home/src"
Currently the following syntax are used to reference the spec in a che-plugin registry or in a yaml file published on the internet respectively:
components:
- chePlugin:
registry:
id: redhat/vscode-yaml/latest
registryUrl: "external-registry-url"
- chePlugin:
url: https://gist.../meta.yaml
That may be confusing. We should find a better name.
In a devfile that is being used to define a stack, there could be multiple starter projects, each providing a separate starter application (simple app, app integrated with Kafka, reactive app etc.)
Today we only have a name
field to distinguish between them. But for developer to make a choice between these starter applications, having more details information on the capabilities you get with each project will be necessary.
If we can add starterProjects
section, devfile creators will be able to use it for starter projects (only one should be selected by the user).
starterProjects:
- name: "kafka-project"
description: "Use this app to get a nodejs application for working with kafka"
- name: "simple-project"
description: "Use this app to get a simple "hello world" nodejs application"
starterProjects
would have the same syntax as projects
but a slightly different behavior: instead of cloning the git repo, only the source code would be fetched, the git metadata would not.
Moreover we would like to add the description
field to both project
and starterProject
to help developers understand the content of the project.
---
components:
- container:
image: maven
terminating: true
- kubernetes:
reference: https://.../mongo.yaml
terminating: true
- plugin:
id: java # <-- terminating is not supported for plugins components (terminating=true/false can be specified in the plugin definition)
Opening this issue to have a discussion about where devfile registries can/should be hosted, and whether we want to have any guidance or recommendations about this. IMHO neither devfile registry providers nor clients need to all support the same mechanisms, but it would probably be worth defining what access methods are core (assuming only url) or what kind of https/auth mechanisms are expected as a base. e.g. do we expect registries to be hosted with no auth, basic auth, on GitHub (with or w/o auth), GHE, or Kube? We don't need to list these, just generally what should be expected.
We want to replace chePlugin
and cheEditor
with a new plugin
component type that:
meta.yaml
, a devfile-like format)odo
or even desktop IDEsThese are the use cases of devfile plugins (basically Che use cases):
devfile.yaml
---
components:
- plugin:
name: 'java language server'
id: redhat/dependency-analytics/latest # <= reference plugins published
# in a plugin registry
registry: https://plugin-registry.io/
- plugin:
name: 'k8s workspace plugin'
kubernetes:
namespace: mynamespace
selector: mykey=myvalue # <== reference a kubernetes
# devworkspacetemplate resource
- plugin:
name: ...
uri: https://..../java8-plugin.yaml # <== reference external raw devfile
components: # <== customise plugin's configuration
- kubernetes:
name: ...
reference: ...
- container:
name: vscode-java
memoryLimit: 2Gi
- volume:
name: m2
size: 2G
java8-plugin.yaml
---
schemaVersion: 2.0.0
metadata:
publisher: redhat
name: java8
version: 0.57.0
displayName: Language Support for Java 8
title: Language Support for Java(TM) by Red Hat
description: Java Linting, Intellisense, (...)
icon: https://(..)/logo.svg
repository: https://github.com/redhat-developer/vscode-java
category: Language
firstPublicationDate: "2020-02-20"
pluginType: che-theia-vsx # <== mandatory for plugins. Valid types
# are che-theia-vsx, che-editor, che-theia-plugin,
# che-theia-extension, generic-service and
# generic-ui
parent:
id: redhat/theia-vsx-stack/latest
components:
- container:
name: vsx-installer
env:
- name: VSX_LIST
value: java-dbg.vsix,java.vsix
components:
- kubernetes:
name: ...
reference: ...
- container:
image: ...che-sidecar-java
name: vscode-java
memoryLimit: "1500Mi"
volumeMounts:
- path: "/home/theia/.m2"
name: m2
- volume:
name: m2
chePlugin
is replaced by non Che specific plugin
cheEditor
is a plugin
with pluginType: che-editor
)pluginType
metadata that allow tools to figure out if they supported the plugin or not. The list of supported types are che-theia-vsx/che-editor/che-theia-plugin/che-theia-extension/generic-service/generic-ui.Plugins are devfile too now. Hence we should rename the devfile registry as stack registry.
As a consequence there will be 2 registries (as before) but name differently:
We may consider to eliminate the plugin concept from the devfile and instead use devfile composition (a.k.a. parents).
For consistency with the Kubernete API spec we may want to use polymorfic plugin type (as we do for components):
- plugin:
name: ...
registryReference:
id: ...
- plugin:
name: ...
kubernetes:
...
- plugin:
name: ...
uriReference:
uri: ...
instead of
- plugin:
name: ...
id: ...
- plugin:
name: ...
kubernetes:
...
- plugin:
name: ...
uri: ...
Instead of the simple customisation of a plugin seen above (a la kustomize) we may use parameters to customize some some attributes of a plugin:
parent:
id: redhat/theia-vsx-stack/latest
parameters:
- VSX_LIST: java-dbg.vsix,java.vsix
instead of
parent:
id: redhat/theia-vsx-template/latest
components:
- container:
name: vsx-installer
env:
- name: VSX_LIST
value: java-dbg.vsix,java.vsix
We want to experiment how the Devfile
format could be used to define a stack.
1. Parent devfile
A parent devfile is a devfile published somewhere where odo
or Che can read
it. Like a gist, a registry or a kubernetes resource (see referencing parents
below).
A parent devfile is a regular devfile and includes components (for build and runtime), one or more code sample projects, events and commands to build and run the sample applications.
Any devfile that references a parent devfile will inherit its components, events and commands but won't inherit the code sample.
2. Referencing parents: uri, id or kubernetes
A parent devfile can be referenced in 3 different ways. Using its id
if it has been published in a registry:
parent:
id: redhat/nodejs/11.6 # <-- the id format is <publisher>/<stack-name>/<version>
registry: https://devfile-registry.io/
Using the URI if it has been published on a static http server (like gist or pastebin):
parent:
uri: https://raw.githubusercontent.com/eclipse/che-devfile-
Using a Kubernetes resource name, namespace if it has been deployed on a Kubernete cluster as a DevWorkspaceTemplate
:
parent:
kubernetes:
name: mydevworkspacetemplate
namespace: mynamespace
3. Customizing parent configuration
When referencing a parent, a devfile should be able to customize its configuration. The syntax to customize a parent configuration is similar to kustomize:
parent:
id: redhat/theia-vsx-stack/latest # <--- Parent referenced by id
components: # <--- Parent configuration can be customized
- name: vsx-installer # <--- Should match the name of an existing parent component
container:
env:
- name: VSX_LIST
value: java-dbg.vsix,java.vsix
commands:
(...)
components: # The main Devfile body provides new elements, not inherited from the parent
- name: tooling # <--- Should *not* match the name of a parent component
container: # <--- Components are added to parent's components
image: busybox
events: # <--- Events are only allowed in the main Devfile body,
(...) # and the resulting events is the merged of the these events with the events of the parent.
The place where an customizable element (component, command or project) should be defined is clearly driven by the following rule. It is either:
parent
element itself if it is an override of an existing parent element,As a consequence;
As for the events
element it is not really customizable: Due to the structure of the Events
object, we can only add command bindings to some event. Overriding an existing command binding doesn't really makes sense. That's why we only allow events in the main devfile body, and not in the overrides. And the resulting events are the merge of the event objects coming from the parent, the plugins, and the main devfile body.
The parent devfile is published on github repo: https://raw.githubusercontent.com/eclipse/che-devfile-registry/master/devfiles/nodejs/devfile.yaml
The following devfile references it:
schemaVersion: 2.0.0
metadata:
name: nodejs-app
parent:
uri: https://(...)/nodejs/devfile.yaml # <--- Parent referenced by `uri`, registry `id`
# or `kubernetes` devworkspace
components: # <--- Parent configuration can be customized
- name: vsx-installer
container:
env:
- name: VSX_LIST
value: java-dbg.vsix,java.vsix
components: # <--- components are added to parent's components
- name: tooling # <--- should not match the name of a parent component
container:
image: busybox
commands: # <--- commands are added to parent's commands
(...)
(EDITED - check edit history for prior revisions and discussion of the opening post)
For devfile 2.2.0. Check the proposal at https://github.com/devfile/api/blob/main/docs/proposals/devfile/outer-loop-build-and-deploy.md
A stack may require expressing the outer-loop build guidance ( source code to image ) in the form of a Dockerfile build, s2i build, etc.
variables:
myimage: myimagename
components:
- name: mydockerfileimage
image:
imageName: {{myimage}}
dockerfile:
buildContext: ${PROJECTS_ROOT}/build
uri: Dockerfile
args: [ "arg1", "arg2", "arg3" ]
rootRequired: false
The other dockerfile src are available in the proposal doc.
Once the runtime image has been built, we can deploy it via the deploy group command:
components:
- name: outerloop-deploy
kubernetes:
uri: deployment-manifest.yaml
commands:
- id: deployk8s
apply:
component: outerloop-deploy
group:
kind: deploy
isDefault: true
The build tool mechanism like kaniko
or buildah
will be up to the tools and hence will not be a part of the spec.
Should it be possible to set environment variables that apply for a specific command? The current way to handle this would be to set the environment variables inline with the command but this can be cumbersome and reduce readability.
Sample of current devfile snippet to show example of usage/format:
- exec:
commandLine: "MAVEN_CONFIG=~/mvn/config JAVA_OPTS="-XX:MaxRAMPercentage=50.0 -XX:+UseParallelGC -XX:MinHeapFreeRatio=10
-XX:MaxHeapFreeRatio=20 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90
-Dsun.zip.disableMemoryMapping=true -Xms20m -Djava.security.egd=file:/dev/./urandom
-Duser.home=/home/user" ./buildSchema.sh"
component: build-tools
alias: buildSchema
Example with proposed changes:
commands:
- exec:
commandLine: "./buildSchema.sh"
env:
- name: MAVEN_CONFIG
value: "~/mvn/config"
- name: JAVA_OPTS
value: "-XX:MaxRAMPercentage=50.0 -XX:+UseParallelGC -XX:MinHeapFreeRatio=10
-XX:MaxHeapFreeRatio=20 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90
-Dsun.zip.disableMemoryMapping=true -Xms20m -Djava.security.egd=file:/dev/./urandom
-Duser.home=/home/user"
component: build-tools
alias: buildSchema
As odo is working on consuming the v2 spec, i noticed that the command group kind is called command group type https://github.com/devfile/kubernetes-api/blob/master/pkg/apis/workspaces/v1alpha1/commands.go#L20
// CommandGroupType describes the kind of command group.
// +kubebuilder:validation:Enum=build;run;test;debug
type CommandGroupType string
Can we please rename this to CommandGroupKind
to be consistent with the discussion here in #27 and also devfile actually uses kind
instead of type
:
commands:
- exec:
id: devBuild
component: runtime
commandLine: npm install
workingDir: ${CHE_PROJECTS_ROOT}/nodejs-web-app/app
group:
kind: build
isDefault: true
In current devfile spec there it is not possible to specify if a container will be run as part of the main dev environment pod[1] or if it should run in its own separated pod. In the v2.0.0 spec we would like to add a new field (dedicatedPod
) that can be applied to container components.
components:
- container:
image: maven
dedicatedPod: true # <-- Optional. Default value is false
[1] In Che the main pod is the pod where the editor and its plugins run.
Commands should have a group
field that will specify in what point of application lifecycle the command should be executed (or type of the command like, build, incremental-build, run etc..).
group
field is optionalgroup
build
, run
, test
, debug
commands:
- exec:
name: "package"
commandLine: "mvn package"
component: build-tools
group:
kind: build
- exec:
name: "install"
commandLine: "mvn install"
component: build-tools
group:
kind: build
isDefault: true
This is something that I've been thinking about for a while, and haven't really been able to find concrete answers on. As there will be multiple consumers of devfile registries (Che, Odo, Codewind), we need to standardize/document what requirements we impose on both a devfile registry, as well as its consumers, so that a user could point Che, Odo, etc at one, and have it properly work across all of them.
Generally, a devfile registry consists of a simple HTTP server, devfiles, and an index.json linking to those devfiles. But there's some unanswered things that need to be documented:
/devfiles
meta.yaml
for each devfile. What requirements are imposed there?
https://github.com/kubernetes/api/blob/master/core/v1/types.go#L2116
Could we support the Kubernetes Container API in the components.container ?
We might want to introduce a syntax that would allow binding commands to a given workspace event.
The lifecycle binding could be provided by a new top-level construct that could look like the following:
events:
<lifecycleBinding>:
- <commandName>
where can be one of the following:
lifecycleBindings
are processed sequentially. For example preStart
commands will be invoked after every postCreate
command has completed.
postStart
bindings should happen after all plugins and extensions have started, including project cloning. This means that those commands are not triggered until the user opens the IDE in his browser.
preStart
commands may need execution of a workspace pod init container. That means that the component associated to the preStart
command should not be part of the workspace pod. The same applies to postStop
bindings.
lifecycleBinding
should provide a mechanism to figure out when they have completed. That means that the command should be terminating or having a readiness probe.
lifecycleBinding
are not guarantee to be executed sequentially or in any order. To run commands in a given order a user should use a composite.
components:
- container: # <== This component is associated with a **preStart** event only.
name: "copier" # That means it won't be included in the workspace pod but
image: '' # will be run before it (as an initContainer for example).
env:
- name: ""
value: ""
volume: ....
- container:
name: "maven"
image: ''
env:
- name: ""
value: ""
volume: ....
- plugin:
id: theia
commands:
- exec:
name: "copyNeededFiles"
component: "copier"
commandLine: "cp somefile"
- exec:
name: "buildAll"
component: "maven"
commandLine: "mvn ..."
- vsCodeTask:
name: "openFile"
component: "theia"
inline: |
{
xxxx
}
events:
preStart:
- "copyNeededFiles" # <== Start of the workspace happens
# after this command is completed
postStart:
- "buildAll" # <== The Order of execution of the commands
- "openFile" # under a same binding is not guarantee
lifecycleBindings
of componentsalias
should not be used anymorename
should be mandatory for kubernetes/openshift/containers componentsname
should be optional for plugins componentsname
is not specified for a component of type plugin it should be inferred from its id
/url
field (id
/url
are used as name
if its not specified explicitly)### a Devfile spec
- container:
name: tooling-container # <-- mandatory
image: busybox
mountSources: true
endpoints:
- name: petclinic-web-server
- kubernetes:
name: mongodb # <-- mandatory
reference: .../deployment.yaml # <-- already have a metadata.name: 'database1'
- plugin:
name: vscode-java # <-- optional
id: ms-vscode/vscode-java/1.1.1
Composite currently only has a label
. But that appears to be something that would be visible in the Che UI. It should have an id
as well so that tools like odo can reference composite commands with the following requirements:
label
and id
id
is mandatoryid
used to reference parent devfiles commands (not label
)Sample devfile snippet to show example of usage/format:
projects:
- name: "devworkspace-spec"
git:
location: "https://github.com/che-incubator/devworkspace-api"
branch: "master"
commands:
- exec:
commandLine: "./buildSchema.sh"
component: build-tools
id: buildSchema
- vscodeTask:
id: openDevfile
inlined:
json
- composite:
id: buildAndOpen
label: Build schema and open devfile
commands:
- buildSchema
- openDevfile
parallel: false
components:
- container:
image: some container image with required build tools
name: "build-tools"
On each Pull Requests, add a validation to check that the json schemas, and other generated files, have correctly been generated and committed back to the repository.
If not we should not allow merging the PR
Polymorphic types makes devWorkspaces
objects easier to validate and more close as a Kubernetes custom resources. But are we sacrificing the UX of the devfile? Was the 1.0.0 devfile spec easier to read and modify for humans?
components:
- type: container
image: maven
...
- type: container
image: nodejs
...
- type: kubernetes
reference: https://gist.../mongodb.yaml
...
components:
- container:
image: maven
...
- container:
image: nodejs
...
- kubernetes:
reference: https://gist.../mongodb.yaml
...
components:
containers:
- image: maven
...
- image: nodejs
...
kubernetes:
- reference: https://gist.../mongodb.yaml
...
Here is a list of proposals to include in the next version of the Devfile
format. Proposals need to be reviewed and approved. We are flagging issues with 👍once they have been reviewed and approved.
We would like to split Devfile
related source code in 2 distinct repositories:
The Devfile
schema is directly generated from the DevWorkspace
API. PRs to change the Devfile
format spec (documentation included) should be done here. Issues related to the format should be also created here.
The Devfile
specification doc and examples are published here. Only PRs to add new examples should be done here.
Subtasks related to the repositories setup:
schemaVersion
attribute 👍apply
a component 👍If other go projects want to add the devworkspace as a dependency they should be able to import only the core API. For instance chePlugins
or cheEditors
components should not be part of it.
We should publish it to https://github.com/redhat-developer/devfile on branch 2.0.0
For now the only type of command that allows running code in a container is the exec
command.
Its semantic is really:
execute a command line in a already-running container.
That doesn't seem to fit with the case when we want to just start a terminating container (as defined in a container
component), with its default entry-point, possibly before the start of the Main workspace POD.
Similarly, we could need to apply a Job defined in a kubernetes
component at some point in the workspace lifecycle (workspace stop for example)
So we should probably introduce a new type of command, for example, named apply
, that would allow applying a component at some point in the workspace history.
commands:
- apply:
id: aCommandToStartAnInitContainer
component: someContainerComponent
- apply:
id: cleanupJobCommand
component: AKubernetesComponentWithAJob
events:
preStart:
- "aCommandToStartAnInitContainer" # <== Start of the workspace happens
# after this terminating container is completed
postStop:
- "cleanupJobCommand"
Odo will have a need for multiple components to mount the same volume. There isn’t an option in devfiles right now to define a volume once, and have multiple components mount that same volume.
To work around this, we’re proposing adding a way to the devfile spec that allows you to define a volume once (a top level ‘volumes’ field), then easily add it to other components via a ‘volumeMount’ field. This is similar to how Kubernetes pods handle volumes.
Example:
---
projects:
- name: "my-project"
components:
- container:
image: "busybox"
name: "mycontainer"
memoryLimit: "128M"
mountSources: true
volumeMount:
- name: my-data-volume
path: /some/folder
- kubernetes:
referenceContent: |
kind: Pod
metadata:
name: sample-pod
spec:
containers:
image: docker.io/my-sample-image:latest
volumeMounts:
- mountPath: /some/path
name: my-data-volume
- volume:
name: my-data-volume
size: 5Gi
The goal of introducing this component is to ONLY let the user specify whether the app is resilient as a Deployment / DeploymentConfig / Stateful Set / Knative Service / any pod-spec-able workload
The tool and user should pick up the hint and do the needful.
---
components:
- workload:
apiVersion: service.knative.dev/v1
type: Service
part of #49
An application stack (represented by a devfile) might need to provide additional files to support certain use cases. Some examples that come to mind are:
Today, we can use $PROJECTS_ROOT as the location where the user's application code is made available in my container using mountSources: true
definition on a component/container.
However, there is no way to express this for files that we need are not coming form user's application:
The current options are:
Both these approach can work but cause additional overhead for stack maintainer. I feel almost every stack will need this capability and it will be a big improvement in the experience if we can simplify access to stack code.
It would be good to have all files from the stack (devfile's location or explicitly declared in the devfile) made available as a volume mount in the container at location $STACK_ROOT.
We can use the mountSources: true
flag or create a new one mountStack:true
(or a similar name) to trigger this.
/kind user-story
/kind epic
/area api
/area library
/area integration-tests
/area test-automation
As a stack/devfile author, I wish to provide deployment guidance in the form of a Helm Chart
---
components:
- id: deploy-node-app
helm:
chart: http://helm-chart-url
values:
image: quay.io/sample/hello-world # A chart may expose a well-known values.yaml parameter called "image".
replicas: 3
If the helm
component is associated with a dockerfile
component (image build guidance) in a composite
command, #51 , che/odo must associate the built image from the 'build' guidance using the image
parameter in the chart's values.yaml .
- composite:
id: build-and-deploy
label: Build image and deploy nodejs app
commands:
- build-node-app
- deploy-node-app
parallel: false
group:
kind: deploy
refined by: @michael-valdron
This a simple proposal to use the name manifest
instead of kubernetes
or openshift
in the devfile components.
---
components:
- container:
name: component1
image: maven
- manifest:
name: component2
reference: https://.../mongo.yaml
- plugin:
id: java
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.