Just a simple jinja or python syntax error yields:

$ dm --service=http://localhost:8080 deploy types/redis/v1/redis.yaml
2015/11/11 15:40:27 cannot deploy template named redis.yaml: status code: 400 status: 400 Bad Request : cannot expand template named redis.yaml (template expansion:expander service response:&{400 Bad Request 400 HTTP/1.1 1 1 map[Date:[Wed, 11 Nov 2015 23:40:27 GMT] Content-Length:[642] Content-Type:[text/plain; charset=utf-8]] 0xc8204c20c0 642 [] false map[] 0xc82040a0e0 }
):
imports:

• path: redis.jinja

resources:

• name: redis
  type: redis.jinja
  properties: null

When using an external template, the manifest is not populated with the contents of the template, or its schema or sub-imports.

There are some useful examples for DM on GCP. We should migrate them to the templates registry for Kubernetes.

There is useful documentation for DM on GCP that we don't yet have for Kubernetes. We should migrate it to this project.

Currently, there's no quality assurance for types pushed to the registry, other than review by the registry owners. We've talked about putting a test framework in place, to check for things like:

• Schema file is present and well formed
• Schema file metadata is complete
• Type definition file is present and validates against the schema
• Referenced types/versions exist and are valid
• Referenced container images/tags exist and are publicly accessible
• Type expands and deploys successfully
• Type specific test suites supplied with the type pass

Tests should run when a type is created or updated. There needs to be a way to add type specific test suites to the type/version directory.

It's in the wrong place in types/replicatedservice/v1/replicatedservice.py

A deployment needs to have information about the resources created:

• status of resource (created, failed, aborted)
• error messages
• RESTful identifier/link to the actual resource created
• name, type, properties

Currently, we require an exact match of the version to resolve chart reference. Instead, we should use semver semantics, meaning that:

• We should prevent breaking changes. For example, a reference to redis:v1,0,0 cannot be satisfied by redis:v2.0.0.
• We should resolve chart references to compatible versions, defined as versions with the same major and minor number, plus higher patch number . For example, a reference to redis:v1.0.0 should resolve to redis:v1.0.1.
• We should take the latest compatible version. For example, if redis:v1.0.1 and redis:v1.0.2 are available, a reference to redis:v1.0.0 should resolve to redis:v1.0.2.

It should be possible to indicate a version for a type, other than through its name, and a template that uses a type should be able to indicate the version range it will accept, and/or we should support semver based version alignment.

Automated tests should assert invariants about the chart version associated with a pull request in kubernetes/charts.

• Automated tests should fail a pull request that doesn't satisfy semver semantics. This requriement ensures that only valid builds can be used to satisfy references.
• Automated tests should fail a pull request that changes an existing version. In other words, it should ensure that versions are immutable. For example, if you have already committed v1.2.0, you can commit v1.2.1, v1.2.2, v1.3.0 or v2.0.0, but you can't change v1.2.0. This requirement guarantees that a dependency on a fully specified three part version number can't break you by changing out from under you.

Currently, we pull type definitions from URLs. We then use the URL as the type name in DM, which makes for awkward REST queries, since the URL must be encoded.

A simpler approach would be to do what the go command does, and let the user specify well known registries by just their prefix (i.e., github.com, bitbucket.org, code.google.com, etc.). See go help importpath for details.

Currently, Kubernetes users have to install DM onto their clusters before they can use it. Make DM a cluster add-on, so that they can enable it with a command line switch.

https://github.com/kubernetes/kubernetes/blob/master/hack/verify-all.sh

The vocabulary currently used by DM is confusing:

• Configuration files that contain no substitution points are called templates.
• Python or Jinja files that do contain substitution points are called types.
• Kubernetes primitives like Pod, Service and Replication Controller, are called configurable resource types. There are no other uses of this term, unlike in DM for GCP.

On top of this confusion, the term type is opaque. Users looking for simple templates might not think to look for types. Everyone knows what a template is. It's a piece of content that contains substitution points. However, a type could be just about anything.

We need to clean up the terminology, and focus more on making it intuitive to the Kubernetes community than on following the precedents set by DM for GCP.

Several users have stumbled trying to bootstrap DM. Typically, they run afoul of getting python configured correctly. However, bootstrapping is unnecessary. It's a novelty and fundamentally, we don't need to modify the user's machine to deploy DM. We built the bootstrap initially to get a feeling for how things would work, but it's just as valid, much faster and much less error prone to create the resources for DM in the cluster using a flat kubectl configuration file. So, instead of subjecting first time users to python package installs and running processes locally, we should provide a configuration file to streamline getting started. We should also preserve the bootstrap file and the accompanying instructions as an example.

Rather than just deploying to default, consider deploying to a unique namespace, e.g. deployment-manager, so that DM resources don't clutter the default namespace.

Currently, our CLI has the generic name client and takes its action from a flag. We could make it friendlier by changing the name to dm and taking the action as the first argument. With those changes, the user experience would change from this:

client --action=deploy <template_files>

to something like this:

dm deploy <template_files>

Currently, expandybird and resourcifier are called synchronously by manager. These calls should be asynchronous, so that we can parallelize execution where permitted by dependencies.

Make expansion run in a sandbox to guarantee that it's well behaved.

Assuming #22, the CLI should be able to:

• add and remove type registries for the user
• set the default type registry for other commands
• list the types and versions in a type registry
• print the metadata for a specific type version

We do not need a way to fetch a type definition or schema, a place to put type definitions or schemas on the localhost, or a way to upload changes to a type definition or schema, since git provides all of that functionality for us, with many supporting features, such as issues, pull requests, etc.

This approach is consistent with our philosophy that config is code, and should be treated as such.

The type registry in this repository is useful for pedagogical purposes, but it doesn't serve well as a destination for publishers. We need a standalone type registry, also owned by CNCF, as described by this comment.

When you try to delete a deployment but it cannot find certain resources the deployment is not deleted.

To reproduce:
Deploy the Guestbook example:
dm deploy guestbook.yaml
Manually delete a replication controller:
kubectl delete rc frontend-rc
Now try to delete the Guestbook deployment
dm delete guestbook.yaml
This returns an error:
cannot delete configuration: kubetcl failed for resource: frontend-rc: exit status 1: Error from server: error when stopping "STDIN": replicationControllers "frontend-rc" not found
And deployment manager still contains the guestbook.yaml deployment:
dm list
includes guestbook.yaml

$ dm --service=http://localhost:8080 deploy types/redis/v1/redis.yaml
2015/11/11 15:37:20 cannot deploy template named redis.yaml: status code: 400 status: 400 Bad Request : cannot create configuration (deployer service response:
&{400 Bad Request 400 HTTP/1.1 1 1 map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff] Date:[Wed, 11 Nov 2015 23:37:20 GMT] Content-Length:[618]] 0xc82000e360 618 [] false map[] 0xc82040a000 }
)

$ cat log/manager.log
2015/11/11 15:37:14 Version: 0.0.1
2015/11/11 15:37:14 Listening on port 8080...
2015/11/11 15:37:16 manager: create deployment: handling request:POST /deployments
2015/11/11 15:37:16 manager: create deployment: handling request:POST /deployments
2015/11/11 15:37:16 Creating deployment: redis.yaml
2015/11/11 15:37:17 created deployment: &{redis.yaml 1 2015-11-11 15:37:17.613537833 -0800 PST 2015-11-11 15:37:17.613538146 -0800 PST 0001-01-01 00:00:00 +0000 UTC 0001-01-01 00:00:00 +0000 UTC Creat t/r/v/redis.yaml d/dm.go X
ed map[]}
2015/11/11 15:37:17 Added manifest manifest-1447285037613578429 to deployment: redis.yaml
2015/11/11 15:37:20 deleted deployment: &{redis.yaml 1 2015-11-11 15:37:17.613537833 -0800 PST 2015-11-11 15:37:17.613538146 -0800 PST 0001-01-01 00:00:00 +0000 UTC 0001-01-01 00:00:00 +0000 UTC Created map[manifest-1447285037613578429:0xc82049ab00]}
2015/11/11 15:37:20 manager: create deployment: returning response: status code:400, status:cannot create configuration (deployer service response:
&{400 Bad Request 400 HTTP/1.1 1 1 map[Content-Type:[text/plain; charset=utf-8] X-Content-Type-Options:[nosniff] Date:[Wed, 11 Nov 2015 23:37:20 GMT] Content-Length:[618]] 0xc82000e360 618 [] false map[] 0xc82040a000 }
)
::1 - - [11/Nov/2015:15:37:16 -0800] "POST /deployments HTTP/1.1" 400 282 "" "Go-http-client/1.1"

$ cat log/resourcifier.log
2015/11/11 15:37:14 Version: 0.0.1
2015/11/11 15:37:14 Listening on port 8082...
2015/11/11 15:37:17 resourcifier: create configuration: handling request:POST /configurations
2015/11/11 15:37:18 kubetcl failed for resource: redis-master: exit status 1: Error from server: error when creating "STDIN": service "redis-master" already exists

2015/11/11 15:37:18 kubetcl failed for resource: redis-master-rc: exit status 1: Error from server: error when creating "STDIN": replicationControllers "redis-master-rc" already exists

2015/11/11 15:37:19 kubetcl failed for resource: redis-slave: exit status 1: Error from server: error when creating "STDIN": service "redis-slave" already exists

2015/11/11 15:37:20 kubetcl failed for resource: redis-slave-rc: exit status 1: Error from server: error when creating "STDIN": replicationControllers "redis-slave-rc" already exists

2015/11/11 15:37:20 resourcifier: create configuration: returning response: status code:400, status:kubetcl failed for resource: redis-master: exit status 1: Error from server: error when creating "STDIN": service "redis-master" already exists

kubetcl failed for resource: redis-master-rc: exit status 1: Error from server: error when creating "STDIN": replicationControllers "redis-master-rc" already exists

kubetcl failed for resource: redis-slave: exit status 1: Error from server: error when creating "STDIN": service "redis-slave" already exists

kubetcl failed for resource: redis-slave-rc: exit status 1: Error from server: error when creating "STDIN": replicationControllers "redis-slave-rc" already exists

::1 - - [11/Nov/2015:15:37:17 -0800] "POST /configurations HTTP/1.1" 400 618 "" "Go-http-client/1.1"

Use https://github.com/kubernetes/kubernetes/blob/master/.travis.yml as a starting point. @vaikas-google: allowed travis access to this repo on 10/22. Need to find out if travis is still the recommended solution for build and test automation.

As a Kubernetes user, I can write DM templates that bind output parameters from a resource into input parameters in dependent resources.

Resource definitions all have things like:

• name: rc
  type: ReplicationController
  properties:
  kind: ReplicationController

This should really be filled in from the type.

GCP DM supports things like:

imports:

• path: http://some.url/foo.py
  name: foo.py

The client should also understand how to do this.

Currently, DM knows how to pull types from URLs. It also understands type schemas that contain metadata describing the type, as well as JSON schema describing the format of the input parameters. We should build on these mechanisms to further formalize the notion of a type registry by:

• making it a Github repo
• requiring a types directory at the top level containing type definitions
• requiring a directory within the types directory for each type being defined (e.g., types/redis)
• requiring a version number directory within a type directory (e.g., types/redis/v1)
• requiring a type definition file in the version directory (e.g., types/redis/v1/redis.jinja)
• requiring a schema file in the version directory (e.g., types/redis/v1/redis.jinja.schema)

See gingko.

googlebot, k8s-bot, ... Need to find out which bots are required, and how to set them up.

As a Kubernetes user, I can rely on DM to instantiate Kubernetes resources in an order that satisfies declared dependencies between resources.

I want to be able to deploy this like I do other systems in Kubernetes via declarative config in kubectl create (e.g., without requiring local daemons/binaries running in a specialized local environment).

It should be possible to bundle one or more versions of a type, along with their schemas, in an archive with a metadata file that describes its contents - i.e., something like a JAR. DM should be able to resolve type references that point to such archives.

$ dm deploy redis:v1
2015/11/11 11:25:34 cannot deploy template named : status code: 400 status: 400 Bad Request : cannot expand template named (template expansion:expander service response:&{400 Bad Request 400 HTTP/1.1 1 1 map[Date:[Wed, 11 Nov 2015 19:25:34 GMT] Content-Length:[622] Content-Type:[text/plain; charset=utf-8]] 0xc20824e300 622 [] false map[] 0xc208033ad0 }
):
resources:

• name: redis:v1
  type: https://raw.githubusercontent.com/kubernetes/deployment-manager/master/types/redis/v1/redis.jinja

$ kubectl logs expandybird-rc-dj2di
2015/11/11 19:22:27 expandybird: expand: handling request:POST /expand
2015/11/11 19:22:28 Expansion process: pid: 14 SysTime: 20ms UserTime: 84ms
2015/11/11 19:22:28 expandybird: expand: returning response: status code:400, status:error (error expanding template : Traceback (most recent call last):
File "./expansion/expansion.py", line 372, in
main()
File "./expansion/expansion.py", line 369, in main
print Expand(template, imports, env=env, validate_schema=validate_schema)
File "./expansion/expansion.py", line 57, in Expand
raise ExpansionError('config', str(e))
main.ExpansionError: 'properties' is undefined Resource: config
) expanding template:
&{ resources:

• name: redis:v1
  type: https://raw.githubusercontent.com/kubernetes/deployment-manager/master/types/redis/v1/redis.jinja
  [0xc820010cc0 0xc820010ce0]}

We delete deployments in the system when they fail, so any k8s resources that may have been deployed are left in the system.

We should not delete the deployment but leave it failed, so that user can issue a delete or update to correct the issue.

When a deployment fails, we should store error messages along-side the status inside the deployment

I need to be able to specify a deployment name when deploying a type or config.

This should probably be dm deploy <type | template> so deployment name is in the same position as get, etc.

We need to do a run through and get some more consistent error messaging throughout the codebase.

Currently, the template registry is embedded in this repository. It should be its own repository for several reasons:

• Make it easier to expose and publicize. As a top level repository, it will be easier to find and access.
• Establish and maintain the focus of the registry by having no other kind of content in the repository.
• Make it possible to have different maintainers. People who maintain content contributions should not have to be the same people who maintain code contributions and vice versa.

Could start by copying the command line from the main README.

It loads expander locally with incorrect paths.

We may want to remove it and reimplement it as a service call to an expand API in the manager that proxies to expandybird.

Because users confuse it with metadata.name. Please use app: instead.

https://github.com/kubernetes/deployment-manager/blob/master/types/replicatedservice/v1/replicatedservice.py#L102

Given that we want members of the community to contribute type definitions, we should provide a guide that tells them how, instead of leaving it for them to try and figure out on their own.

Currently, resourcifier relies on kubectl to instantiate resources. However, it should be able to run arbitrary command line processes in a container, based on configuration metadata.

As a Kubernetes user, I see that the configuration artifacts I create using open source DM are persisted in the cluster.

GCP DM has the gcloud CLI which will, for an imported template, also look for and import its schema file and recursively pull in all imports defined within. e.g.,

imports: [foo.py]

may end up passing foo.py, foo.py.schema, and foo-helper.py to the API.

The DM client should also support this.

godeps or new tool go builder (http://dave.cheney.net/2015/06/09/gb-a-project-based-build-tool-for-the-go-programming-language) might be more appropriate as suggested by bburns@.