juju / charm-tools Goto Github PK

Tools for charm authors and maintainers

License: Other

Python 88.54% Shell 10.74% Makefile 0.36% DIGITAL Command Language 0.20% Roff 0.08% Jinja 0.09%

charm-tools's Introduction

Juju is an open source application orchestration engine that enables any application operation (deployment, integration, lifecycle management) on any infrastructure (Kubernetes or otherwise) at any scale (development or production) in the same easy way (typically, one line of code), through special operators called ‘charms’.


👉	Juju	Learn how to quickly deploy, integrate, and manage charms on any cloud with Juju. It's as simple as `juju deploy foo`, `juju integrate foo bar`, ..., on any cloud.

	Charmhub	Sample our existing charms on Charmhub. A charm can be a cluster (OpenStack, Kubernetes), a data platform (PostgreSQL, MongoDB, etc.), an observability stack (Canonical Observability Stack), an MLOps solution (Kubeflow), and so much more.

	Charm SDK	Write your own charm! Juju is written in Go, but our SDK supports easy charm development in Python.

Give it a try!

Let's use Juju to deploy, configure, and integrate some Kubernetes charms:

Set up

You will need a cloud and Juju. The quickest way is to use a Multipass VM launched with the charm-dev blueprint.

Install Multipass: Linux | macOS | Windows. On Linux:

sudo snap install multipass

Use Multipass to launch an Ubuntu VM with the charm-dev blueprint:

multipass launch --cpus 4 --memory 8G --disk 30G --name tutorial-vm charm-dev

Open a shell into the VM:

multipass shell tutorial-vm

Verify that you have Juju and two localhost clouds:

juju clouds

Bootstrap a Juju controller into the MicroK8s cloud:

juju bootstrap microk8s tutorial-controller

Add a workspace, or 'model':

juju add-model tutorial-model

Deploy, configure, and integrate a few things

Deploy Mattermost:

juju deploy mattermost-k8s

See more: Charmhub | mattermost-k8s

Deploy PostgreSQL:

juju deploy postgresql-k8s --channel 14/stable --trust

See more: Charmhub | postgresql-k8s

Enable security in your PostgreSQL deployment:

juju deploy tls-certificates-operator
juju config tls-certificates-operator generate-self-signed-certificates="true" ca-common-name="Test CA"
juju integrate postgresql-k8s tls-certificates-operator

Integrate Mattermost with PostgreSQL:

juju integrate mattermost-k8s postgresql-k8s:db

Watch your deployment come to life:

juju status --watch 1s

(Press Ctrl-C to quit. Drop the --watch 1s flag to get the status statically. Use the --relations flag to view more information about your integrations.)

Test your deployment

When everything is in active or idle status, note the IP address and port of Mattermost and pass them to curl:

curl <IP address>:<port>/api/v4/system/ping

You should see the output below:

{"AndroidLatestVersion":"","AndroidMinVersion":"","IosLatestVersion":"","IosMinVersion":"","status":"OK"}

Congratulations!

You now have a Kubernetes deployment consisting of a Mattermost backed by PosgreSQL with TLS-encrypted traffic!

Clean up

Delete your Multipass VM:

multipass delete --purge tutorial-vm

Uninstall Multipass: Linux | macOS | Windows. On Linux:

snap remove multipass

Next steps

Learn more

Read our user docs
Read our developer docs

Chat with us

Read our Code of conduct and:

Join our chat: Matrix
Join our forum: Discourse

File an issue

Report a Juju bug on Launchpad
Raise a general https://juju.is/docs documentation issue on Github | juju/docs

Make your mark

Read our documentation contributor guidelines and help improve a doc
Read our codebase contributor guidelines and help improve the codebase

charm-tools's People

Contributors

Stargazers

Watchers

Forkers

asherbond bac marcoceppi cholcombe973 ader1990 johnsca pdaniel-frk mbruzek adam-stokes axw stub42 tvansteenburgh aznashwan javacruft adamisrael bcsaller castrojo acidburn0zzz ajkavanagh ryan-beisner lutostag freeekanayaka nobuto-m tdsmith hloeung evandandrea hyperbolic2346 canonical canonicalltd gnuoy hatched ycliuhw freyes mhilton kyrofa mattcurrycom aeroglyphic tai271828 frankban cjwatson n-pochet timclicks guoqiao afreiberger sabaini chris-sanders macduff23 lourot mthaddon xavpaice bo98 isabella232 rgildein barryprice fnordahl addyess trellixvulnteam jugmac00 coreycb xtrusia markbeierl lathiat sombrafam

charm-tools's Issues

Our HTML should be semantic

In order to form a more perfect HTML doc and avoid crazy rendering issues we should link up with Travis CI and perform a lint of all the files.

I also suggest adding a Makefile which will properly lint so the user doesn't have to submit a pull request just to have the linting run.

test runner: setup timeout not being honored

I'm trying to run tests for an openstack charm and keep running into timeouts. I've specified --timeout 2h and --setup-timeout 1200 but it looks like I still get disconnected after approx. 5mins.

$ juju test -v -p AMULET_HTTP_PROXY tests/015-basic-trusty-icehouse --timeout 7200 --setup-timeout 1200 --set-e
juju-test INFO    : Starting test run on local using Juju 1.22.6
juju-test DEBUG   : Loading configuration options from testplan YAML
juju-test DEBUG   : Creating a new Conductor
juju-test.conductor DEBUG   : Starting a bootstrap for local, kill after 1200
juju-test.conductor DEBUG   : Running the following: juju bootstrap -e local
Bootstrapping environment "local"
Starting new instance for initial state server
Building tools to upload (1.22.6.1-trusty-amd64)
Installing Juju agent on bootstrap instance
...
2015-10-06 17:02:54  Deploying service rabbitmq-server using local:trusty/rabbitmq-server
Connection Timeout: disconnecting client after 300.0 seconds
Timeout occurred, printing juju status...environment: local
machines:
  "0":
    agent-state: started
    agent-version: 1.22.6.1
    dns-name: localhost

Version: charm-tools 1.7.1-0ubuntu3~~ubuntu14.04.1~~ppa1

broken url in generated hook

This puts a broken link in my generated hooks:

https://github.com/juju/charm-tools/blob/master/charmtools/build/tactics.py#L217

The URL from the above line should probably point to here, no?

https://jujucharms.com/docs/stable/authors-charm-building

Installation of dependencies undocumented

The base layer appears to magically install Python depencies using .pypi files in the lib/ directory. This mechanism appears to be undocumented. Is this for general use, or only for charm-tools to get itself bootstrapped?

charm-tools package should depend on python-pip

Steps to reproduce:
Environment: Ubuntu 14.04, minimal virtual machine image. In my case, I use uvt-kvm to provision throwaway development server instances on my laptop.

apt-add-repository ppa:juju/stable
apt-get update
apt-get install charm-tools

Tried to charm compose a layer I'd developed and encountered this error:

composer: Composing into /home/ubuntu/oo/charms
composer: Processing layer: layer:basic
composer: Processing layer: oostore
composer: Processing interface: pgsql
Traceback (most recent call last):
  File "/usr/bin/charm-compose", line 9, in <module>
    load_entry_point('charm-tools==1.7.1', 'console_scripts', 'charm-compose')()
  File "/usr/lib/python2.7/dist-packages/charmtools/compose/__init__.py", line 468, in main
    composer()
  File "/usr/lib/python2.7/dist-packages/charmtools/compose/__init__.py", line 393, in __call__
    self.generate()
  File "/usr/lib/python2.7/dist-packages/charmtools/compose/__init__.py", line 359, in generate
    self.exec_plan(self.plan, self.layers)
  File "/usr/lib/python2.7/dist-packages/charmtools/compose/__init__.py", line 337, in exec_plan
    tactic()
  File "/usr/lib/python2.7/dist-packages/charmtools/compose/tactics.py", line 410, in __call__
    spec)).throw_on_error()()
  File "/usr/lib/python2.7/dist-packages/charmtools/utils.py", line 181, in __call__
    p = subprocess.Popen(all_args, **kwargs)
  File "/usr/lib/python2.7/subprocess.py", line 710, in __init__
    errread, errwrite)
  File "/usr/lib/python2.7/subprocess.py", line 1327, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory
make: *** [compose] Error 1

With some debug logging, I determined that python-pip wasn't installed. apt-get install python-pip resolved the issue. This package needs to be added to the charm-tools package dependencies.

PPA does not work for wily

charm-tools cannot be installed on Ubuntu wily using ppa:juju/stable.

$ sudo apt install charm-tools
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 charm-tools : Depends: python-charmworldlib but it is not installable
               Depends: python-path.py but it is not installable
E: Unable to correct problems, you have held broken packages.

$ apt-cache policy charm-tools python-charmworldlib python-path.py
charm-tools:
  Installed: (none)
  Candidate: 1.8.0-0ubuntu1~ubuntu15.10.1~ppa1
  Version table:
     1.8.0-0ubuntu1~ubuntu15.10.1~ppa1 0
        500 http://ppa.launchpad.net/juju/stable/ubuntu/ wily/main amd64 Packages
     1.0.0-0ubuntu2 0
        500 http://jp.archive.ubuntu.com/ubuntu/ wily/universe amd64 Packages
python-charmworldlib:
  Installed: (none)
  Candidate: (none)
  Version table:
python-path.py:
  Installed: (none)
  Candidate: (none)
  Version table:

Add action support to build

To support reactive actions, we should create boilerplate action executables based on the contents of actions.yaml during the charm build process.

readme.md is not processed in charm-compose

When you have a file containing readme.md (notice the case) it will not properly render the final README.md created during charm-compose.

Moved from canonical/charms.reactive#8

charm-proof doesnt know about tags

It looks as though jujucharms.com supports the use of tags, however, charm proof frowns on it.

multiple layers that inherit layer-basic gets downloaded multiple times

An example output of my ghost layer https://github.com/battlemidget/juju-charm-ghost which inherits a few layers which all inherit the base layer as their starting point:

http://paste.ubuntu.com/13503577/

Lines 19, 21, 23 show the build process cloning the base layer multiple times. This doesn't break anything but does add an additional wait time when building out the layers

charm get should prompt when there are more than one series

The "charm get" command lead me astray when there were more than one version of a charm in the charm store. The command:

charm get postgresql

Gave me the precise version when I expected the trusty version. This caused a problem with a misdirected merge.

If there are more than one version for a charm can "charm get" prompt the user for which series, or fail gracefully with a message about the series being required for charms that have more than one series? I am willing to work on this bug myself if no one else wants to do it.

Add reactive hooks for defined storage

See canonical/layer-basic#4

charm build in a virtual environment fails

I am trying to build a charm with the most current charm-tools code. My understanding is I can run a virtualenv and pip install the charm-tools source directory. The "--user" flag for pip makes it impossible to build from a virtualenv.

Here are the steps I performed today.

virtualenv /tmp/venv
source /tmp/venv/bin/activate
git clone [email protected]:mbruzek/charm-tools
cd charm-tools
pip install -e .
cd ~/workspace/charms/trusty/layer-docker
(venv)mbruzek@warhorse:~/workspace/charms/trusty/layer-docker$ charm build
build: Composing into /home/mbruzek/workspace/charms
build: Processing layer: layer:basic
build: Processing layer: docker
Traceback (most recent call last):
  File "/tmp/venv/bin/charm-build", line 9, in <module>
    load_entry_point('charm-tools==1.7.1', 'console_scripts', 'charm-build')()
  File "/home/mbruzek/workspace/git/charm-tools/charmtools/build/__init__.py", line 501, in main
    build()
  File "/home/mbruzek/workspace/git/charm-tools/charmtools/build/__init__.py", line 405, in __call__
    self.generate()
  File "/home/mbruzek/workspace/git/charm-tools/charmtools/build/__init__.py", line 371, in generate
    self.exec_plan(self.plan, self.layers)
  File "/home/mbruzek/workspace/git/charm-tools/charmtools/build/__init__.py", line 349, in exec_plan
    tactic()
  File "/home/mbruzek/workspace/git/charm-tools/charmtools/build/tactics.py", line 435, in __call__
    spec), env=localenv).throw_on_error()()
  File "/home/mbruzek/workspace/git/charm-tools/charmtools/utils.py", line 193, in __call__
    result.throw_on_error()
  File "/home/mbruzek/workspace/git/charm-tools/charmtools/utils.py", line 150, in throw_on_error
    self.exit_code, self.command, output=self.output)
subprocess.CalledProcessError: Command '('pip', 'install', '--user', '--ignore-installed', u'charms.reactive>=0.1.0,
<2.0.0')' returned non-zero exit status 1

When I ran the install command manually and found the error:
Can not perform a '--user' install. User site-packages are not visible in this virtualenv.
Storing debug log for failure in /home/mbruzek/.pip/pip.log

I tried several things including installing charms.reactive in the virtualenv manually. The build command always fails because the "--user" option is specified.

add --clean flag to charm build

When heavily developing a charm, there are cases where I'm shuffling interfaces/layers between the dependent layer.yaml(s) - and as charm-build is a constructive tool, there's no easy way to signal in place "Please remove my prior constructed charm, and rebuild from scratch - i've removed something".

Introducing a --clean flag is indicative that its a destructive process, and as charm-build knows where the file is going, it could reasonably rm -rf the directory prior to doing the build to ensure a clean artifact is constructed.

charm build fails for wheelhouse

Traceback (most recent call last):
  File "/usr/bin/charm-build", line 9, in <module>
    load_entry_point('charm-tools==1.9.1', 'console_scripts', 'charm-build')()
  File "/usr/lib/python2.7/dist-packages/charmtools/build/__init__.py", line 525, in main
    build()
  File "/usr/lib/python2.7/dist-packages/charmtools/build/__init__.py", line 423, in __call__
    self.generate()
  File "/usr/lib/python2.7/dist-packages/charmtools/build/__init__.py", line 388, in generate
    self.exec_plan(self.plan, self.layers)
  File "/usr/lib/python2.7/dist-packages/charmtools/build/__init__.py", line 368, in exec_plan
    tactic()
  File "/usr/lib/python2.7/dist-packages/charmtools/build/tactics.py", line 517, in __call__
    self._add(pip, wheelhouse, 'pip')
  File "/usr/lib/python2.7/dist-packages/charmtools/build/tactics.py", line 501, in _add
    reqs).throw_on_error()()
  File "/usr/lib/python2.7/dist-packages/charmtools/utils.py", line 193, in __call__
    result.throw_on_error()
  File "/usr/lib/python2.7/dist-packages/charmtools/utils.py", line 150, in throw_on_error
    self.exit_code, self.command, output=self.output)
subprocess.CalledProcessError: Command '(Path(u'/tmp/tmpRhDlw9/bin/pip'), 'wheel', '--no-binary', ':all:', '-w', Path(u'/tmp/tmp_y2Ybn'), 'pip')' returned non-zero exit status 1

This seems to be because we're not casting Path to a string. This is becoming a common problem and I'm beginning to wonder if using Path is the right idea in this library instead of os.path

Add support for layer options in layer.yaml

It is useful for base layers to accept some manner of "config" from layers building on them and YAML files are a reasonable way to accomplish this. However, if each layer defines its own YAML file (e.g., the Apache PHP base layer reads an apache.yaml file), this will lead to a proliferation of YAML files and increase the potential for collision. It also doesn't cleanly support subsequent layers override the layer config of a previous layer, without having to replicate the entire YAML file.

The layer.yaml file should support two new sections: options and defines*.

The defines section should contain a jsonschema definition describing what options it supports and what their values can be. (This is the same format that actions use to define their params.) These options will automatically be namespaced by the layer name.

The options section will be a dictionary section with layer names as keys, and the set of options & values for that layer, as defined by that layer's defines section. The charm build command can use the schemas from the defines section to validate the options. The layer.yaml file of a higher level layer can override the options specified by lower level layers, although obviously care should be taken if doing so.

See https://gist.github.com/marcoceppi/ad318d081e3b30857819 for an example of what this would look like.

*: Feel free to bike-shed on the names.

Base layer doesn't have upgrade-charm hook

I see the base layer doesn't include the upgrade-charm hook. What is the reason behind this? I think it should be included in the base layer since the idea is that every Charm will have one.

Git repos on Launchpad are not handled properly

In charmtools/fetchers.py the test for a bzr repo matches anything on Launchpad, and the test for Git(Hub) repos only matches GitHub. This causes git repos on Launchpad to be fetched with bzr, which works, but seems to cause subtle differences, or may not always work.

.pypi libraries' script / bin tools are not installed in deployed charm

Original issue: canonical/charms.reactive#5

Any library installed via a .pypi file in a layer that installs a script or binary CLI tool will fail due to the location that pip tries to install the tool to. I think this can be fixed simply by having a bin directory in the base layer and having that added to the path for all hooks, but I'm not certain.

Rename all instances of (.*)composer.yaml and composer

The module should be referred to as (ie in logging, etc) as build and any reference to .composer.manifest or composer.yaml needs to be .layers.manifest and layers.yaml We should aim to support composer.yaml but throw a W deprecation notice when it's used. In 2.0 we'll drop it officially. All documentation should be updated to reflect layers.yaml and generate.

Furthermore generate and refresh need to be renamed to just build

composer/build.yaml duplicates includes

This was done with three layers:

layer:basic
nodejs
myapplication

Nodejs layer includes 'layer:basic', my app includes 'layer:basic', when running charm-compose/build it states that it processes layer:basic twice:

‹master*› »»»» charm-compose                                                                                                 0|20:54:43
composer: Composing into /home/adam/charms
composer: Processing layer: layer:basic
composer: Processing layer: node
composer: Processing layer: layer:basic
composer: Processing layer: wffmapi
composer: Processing interface: pgsql

Should this not duplicate by default or should my nodejs layer not include 'layer:basic' and only have the topmost layer include that?

`charm create` OSError on Yosemite (No such file or directory)

Run as a standard user in an empty directory.

Juju was installed with brew install juju --devel
charm-tools was installed with brew install charm-tools

Full error log here: https://gist.github.com/NotBobTheBuilder/e3f3e02796889692b3a4

Readme.md needs updating

Following the manual install steps doesn't work
Readme references 'bin' directory that doesn't exist in repository

Readme notes to check out charm-tools from bzr instead of github

charm proof warning message has spelling error

I was working on a charm that has warnings from charm proof and noticed that the message has a spelling error.

W: summary sould be less than 72

We should spell "should" correctly.

Charm build does not remove present interfaces/layers when removed

Charm build leaves behind declared layers/interfaces on the next attempted build.

Sample interfaces.yaml for first build

includes: ['layer: basic', 'interface: http']

Sample interfaces.yaml for second build

includes: ['layer: basic']

The http interface relation layer(s) are left behind when executing charm build

charm-layers to have a minimal flag

Running charm-layers pretty much dumps the tree of what a final charm looks like from charm-build

http://paste.ubuntu.com/13094417/

It would be nice to have a flag like --no-deps to only print the files I've actually created and not those installed via pip in the /lib dir.

$HOME is needed for juju-test with LXD provider

Removing the HOME env var, the $HOME/.config dir becomes /.config and lxd-provider cannot locate lxd.

Workaround: run juju-test with -p HOME

Expected behavior: workaround should be unnecessary.

Fetching charm with empty metadata.yaml fails

In charmtools/fetchers.py in the rename function, there are catches for a missing metadata.yaml as well as a metadata.yaml that is missing name, but those miss if there is an entirely empty metadata.yaml. See http://paste.ubuntu.com/12782870/ for the traceback of the error.

add a charm `verify-env` like command

Since a few environment variables are recommended to be defined during charm-build it may be nice to have something like charm verify-env to check for common settings to be available to help identify potential problems with a user's setup during build.

For example,

$ charm verify-env

Checking env variables

x LAYER_PATH not found
. INTERFACE_PATH found at $HOME/charms/layers
. JUJU_REPOSITORY found at $HOME/charms

Possible fixes for (found bash shell for $USER):

echo "export LAYER_PATH=$HOME/charms/layers" >> ~/.bashrc

Thoughts?

charm build in an empty directory yeilds a python traceback

The charm build command in an empty directory fails with a python traceback. I do expect the command to fail, but it should provide a better error message.

mbruzek@warhorse:/tmp/foo$ charm build
build: Composing into .
build: The top level layer expects a valid layer.yaml file, using defaults.
build: Processing layer: foo
Traceback (most recent call last):
  File "/usr/local/bin/charm-build", line 9, in <module>
    load_entry_point('charm-tools==1.8.0', 'console_scripts', 'charm-build')()
  File "/usr/local/lib/python2.7/dist-packages/charmtools/build/__init__.py", line 520, in main
    build()
  File "/usr/local/lib/python2.7/dist-packages/charmtools/build/__init__.py", line 422, in __call__
    self.generate()
  File "/usr/local/lib/python2.7/dist-packages/charmtools/build/__init__.py", line 387, in generate
    self.exec_plan(self.plan, self.layers)
  File "/usr/local/lib/python2.7/dist-packages/charmtools/build/__init__.py", line 374, in exec_plan
    self.write_signatures(signatures, layers)
  File "/usr/local/lib/python2.7/dist-packages/charmtools/build/__init__.py", line 382, in write_signatures
    ), indent=2))
  File "/usr/local/lib/python2.7/dist-packages/path.py", line 870, in write_text
    self.write_bytes(text, append=append)
  File "/usr/local/lib/python2.7/dist-packages/path.py", line 784, in write_bytes
    with self.open(mode) as f:
  File "/usr/local/lib/python2.7/dist-packages/path.py", line 749, in open
    return io.open(self, *args, **kwargs)
  File "/usr/lib/python2.7/contextlib.py", line 35, in __exit__
    self.gen.throw(type, value, traceback)
  File "/usr/local/lib/python2.7/dist-packages/path.py", line 106, in io_error_compat
    raise os_err
OSError: [Errno 2] No such file or directory: './trusty/foo/.build.manifest'

Generated storage-attached hooks fail

I'm building a reactive charm that has storage, and it fails on storage-attached. I'm using charm-tools 1.10. The error is:

ImportError: No module named 'charms'

storage-attached may run before the install hook. It looks like the charms.reactive package doesn't get installed until the install hook.

Interfaces and layers are served insecurely

'charm build' looks up layers and interfaces using plain HTTP from http://interfaces.juju.solutions, and is subject to various attacks potentially injecting malicious code into generated charms.

Generated relation and storage hooks are broken in 1.10

I've just built a reactive charm with 1.10, and had errors in storage-attached due to the generated hook file having a blank line before the shebang. Probably broken with the change over to using Python 3.

Relative paths don't work with charm build in the output flag.

This is effectively using a relative path to direct output to a distribution folder. Through working with @bcsaller @marcoceppi @johnsca and @mbruzek, we discovered that this fails to produce a bin directory in the generated charm.

@bcsaller has asked that I assign this to him.

charm build hide_metrics can not be disabled

When running charm build -l DEBUG I noticed that "hide_metrics" is true. I tried to turn it to false, using the --hide-metrics flag, but was unable to change the setting.

Shouldn't hide_metrics default to false and --hide-metrics set it to true? We should fix that.

charm generate requires INTERFACE_PATH directories to be named exactly as interface

charm generate requires directories under INTERFACE_PATH to be named with the exact interface name and if not I get some ugliness (I'm testing a local interface that's not available on http://interfaces.juju.solutions/).

http://paste.ubuntu.com/13623744/

generate runtests: commands[0] | charm generate --log-level DEBUG -o /home/ubuntu/openstack-charm-layer-dev/build charm                                                                          [1143/2811]
build: {
  "INTERFACE_PATH": "/home/ubuntu/openstack-charm-layer-dev/interfaces", 
  "JUJU_REPOSITORY": "/home/ubuntu/openstack-charm-layer-dev/build", 
  "LAYER_PATH": null, 
  "_charm": "charm", 
  "_name": "testcharm", 
  "config": "<charmtools.build.config.BuildConfig object at 0x2b5d81f3b450>", 
  "deps": "/home/ubuntu/openstack-charm-layer-dev/build/deps", 
  "force": false, 
  "hide_metrics": false, 
  "interface_service": "http://interfaces.juju.solutions", 
  "log_level": "DEBUG", 
  "output_dir": "/home/ubuntu/openstack-charm-layer-dev/build", 
  "repo": "/home/ubuntu/openstack-charm-layer-dev/build/trusty", 
  "series": "trusty", 
  "target_dir": "/home/ubuntu/openstack-charm-layer-dev/build/trusty/testcharm"
}
charmtools.fetchers: git clone https://github.com/juju-solutions/reactive-base-layer.git /home/ubuntu/openstack-charm-layer-dev/build/deps/layer/tmpgeuNxA: Cloning into '/home/ubuntu/openstack-charm-layer
-dev/build/deps/layer/tmpgeuNxA'...

Traceback (most recent call last):
  File "/home/ubuntu/openstack-charm-layer-dev/.tox/generate/bin/charm-generate", line 9, in <module>
    load_entry_point('charm-tools==1.9.3', 'console_scripts', 'charm-generate')()
  File "/home/ubuntu/openstack-charm-layer-dev/.tox/generate/local/lib/python2.7/site-packages/charmtools/build/__init__.py", line 493, in deprecated_main
    main()
  File "/home/ubuntu/openstack-charm-layer-dev/.tox/generate/local/lib/python2.7/site-packages/charmtools/build/__init__.py", line 525, in main
    build()
  File "/home/ubuntu/openstack-charm-layer-dev/.tox/generate/local/lib/python2.7/site-packages/charmtools/build/__init__.py", line 423, in __call__
    self.generate()
  File "/home/ubuntu/openstack-charm-layer-dev/.tox/generate/local/lib/python2.7/site-packages/charmtools/build/__init__.py", line 386, in generate
    layers = self.fetch()
  File "/home/ubuntu/openstack-charm-layer-dev/.tox/generate/local/lib/python2.7/site-packages/charmtools/build/__init__.py", line 211, in fetch
    return self.fetch_deps(layer)
  File "/home/ubuntu/openstack-charm-layer-dev/.tox/generate/local/lib/python2.7/site-packages/charmtools/build/__init__.py", line 215, in fetch_deps
    self.fetch_dep(layer, results)
  File "/home/ubuntu/openstack-charm-layer-dev/.tox/generate/local/lib/python2.7/site-packages/charmtools/build/__init__.py", line 239, in fetch_dep
    iface = Interface(base, self.deps).fetch()
  File "/home/ubuntu/openstack-charm-layer-dev/.tox/generate/local/lib/python2.7/site-packages/charmtools/build/__init__.py", line 83, in fetch
    self.directory = path(fetcher.fetch(self.target_repo))
  File "/home/ubuntu/openstack-charm-layer-dev/.tox/generate/local/lib/python2.7/site-packages/charmtools/build/fetchers.py", line 79, in fetch
    res = f.fetch(dir_)
  File "/home/ubuntu/openstack-charm-layer-dev/.tox/generate/local/lib/python2.7/site-packages/charmtools/fetchers.py", line 188, in fetch
    shutil.copytree(self.path, dst, symlinks=True)
  File "/usr/lib/python2.7/shutil.py", line 208, in copytree
    raise Error, errors
shutil.Error: [(path(u'/home/ubuntu/openstack-charm-layer-dev/build/deps/interface/openstack-charm-layer-dev/build/deps/interface/openstack-charm-layer-dev/build/deps/interface/openstack-charm-layer-dev/b
uild/deps/interface/openstack-charm-layer-dev/build/deps/interface/openstack-charm-layer-dev/build/deps/interface/openstack-charm-layer-dev/build/deps/interface/openstack-charm-layer-dev/build/deps/interf
ace/openstack-charm-layer-dev/build/deps/interface/openstack-charm-layer-dev/build/deps/interface/openstack-charm-layer-dev/build/deps/interface/openstack-charm-layer-dev/build/deps/interface/openstack-ch
arm-layer-dev/build/deps/interface/openstack-charm-layer-dev/build/deps/interface/openstack-charm-layer-dev/build/deps/interface/openstack-charm-layer-dev/build/deps/interface/openstack-charm-layer-dev/bu
ild/deps/interface/openstack-charm-layer-dev/build/deps/interface/openstack-charm-layer-dev/build/deps/interface/openstack-charm-layer-dev/build/deps/interface/openstack-charm-layer-dev/build/deps/interfa
ce/openstack-charm-layer-dev/build/deps/interface/openstack-charm-layer-dev/build/deps/interface/openstack-charm-layer-dev/build/deps/interface/openstack-charm-layer-dev/build/deps/interface/openstack-cha
rm-layer-dev/build/deps/interface/openstack-charm-layer-dev/build/deps/interface/openstack-charm-layer-dev/build/deps/interface/openstack-charm-layer-dev/build/deps/interface/openstack-charm-layer-dev/bui
ld/deps/interface/openstack-charm-layer-dev/build/deps/interface/openstack-charm-layer-dev/build/deps/interface/openstack-charm-layer-dev/build/deps/interface/openstack-charm-layer-dev/build/deps/interfac
e/openstack-charm-layer-dev/build/deps/interface/openstack-charm-layer-dev/build/deps/interface/openstack-charm-layer-dev/build/deps/interface/openstack-charm-layer-dev/build/deps/interface/openstack-char
m-layer-dev/build/deps/interface/openstack-charm-layer-dev/build/deps/interface/openstack-charm-layer-dev/build/deps/interface/openstack-charm-layer-dev/build/deps/interface/openstack-charm-layer-dev/buil
d/deps/interface/openstack-charm-layer-dev/build/deps/interface/openstack-charm-layer-dev/build/deps/interface/openstack-charm-layer-dev/build/deps/interface/openstack-charm-layer-dev/build/deps/

It seems like looking through the interface.yaml files in each directory for the name key would make more sense and/or better error handling for the scenario.

proof doesn't know about storage

INFO:root:Workspace dir: /var/lib/jenkins/workspace/charm_lint_check
INFO:root:Reading file: Makefile
INFO:root:Searching for: ['@flake8', '/flake8']
INFO:root:command:  make -f Makefile lint
E: Unknown root metadata field (storage)
I: config.yaml: option ephemeral-unmount has no default value
I: config.yaml: option monitor-secret has no default value
I: config.yaml: option ceph-public-network has no default value
I: config.yaml: option ceph-cluster-network has no default value
I: config.yaml: option key has no default value
I: config.yaml: option fsid has no default value
I: config.yaml: option osd-journal has no default value
I: config.yaml: option osd-reformat has no default value
I: config.yaml: option source has no default value
make: *** [lint] Error 200
ERROR:root:Make target returned non-zero.

It should probably be pretty similar to the relation checking.

tempita and yaml dependencies are platform dependent

One of the dependencies pulled in when building a charm with the base layer is tempita, and it builds a platform dependent .so file. This will cause templating to fail on different architectures.

The yaml dependency attempts to link with libyaml. If the necessary dependencies are installed (libyaml-dev), this also builds a platform dependent file.

Some care and feeding will be necessary, perhaps not installing dependencies that can be pulled in as debs.

charm-build help errors

$ charm-build help                                                                      
build: Composing into help
Traceback (most recent call last):
  File "/usr/bin/charm-build", line 9, in <module>
    load_entry_point('charm-tools==1.8.0', 'console_scripts', 'charm-build')()
  File "/usr/lib/python2.7/dist-packages/charmtools/build/__init__.py", line 520, in main
    build()
  File "/usr/lib/python2.7/dist-packages/charmtools/build/__init__.py", line 422, in __call__
    self.generate()
  File "/usr/lib/python2.7/dist-packages/charmtools/build/__init__.py", line 385, in generate
    layers = self.fetch()
  File "/usr/lib/python2.7/dist-packages/charmtools/build/__init__.py", line 203, in fetch
    layer = Layer(self.charm, self.deps).fetch()
  File "/usr/lib/python2.7/dist-packages/charmtools/build/__init__.py", line 89, in fetch
    self.url, self.ENVIRON))
OSError: Unable to locate help. Do you need to set LAYER_PATH

I've tried setting LAYER_PATH to my charm's src directory where layers.yaml is located, but that doesn't seem to work either.

Track metrics for requested layers on build

Track metrics around which layers and interfaces are being used to build charms.
This can help focus ~charmer attention.
Later we can support visualization of how charms are evolving in the ecosystem.

base layer hooks hardcode Python2

The install, upgrade-charm, start, stop etc. hooks in the reactive basic layer all use '#!/bin/env python' as the shebang line, which is Python2. Rather than force a version of Python that has been deprecated for several years and will soon stop being on the cloud images by default, 'charm build' should default to generating Python3 stubs and perhaps provide an option to use Python2 when compatibility is needed with legacy code.

layer.yaml could be used to declare if Python2 compatibility is needed.

If both Python2 and Python3 are supported, 'charm build' will need to detect and correct shebang lines somehow. The edge case here being a Python3 charm that includes copies of legacy Python2 scripts (eg. administrative scripts to be invoked from cron).

Feature req: support pulling layers from github/bitbucket etc

Not sure the scope of this but it would be nice to have something like this in my layer.yaml

includes: ['layer:battlemidget/nodejs']

Or maybe even further

includes: ['interface:bb:battlemidget/apache2', 'interface:gh:battlemidget/rethinkdb']

Or with the latest charm-tools layer.yaml and interface.yaml it could be shortened to

# layer.yaml
includes: ['gh:battlemidget/nodejs']

# interface.yaml
includes: ['bitbucket/battlemidget/rethinkdb']

Kind of like scoped packages that don't necessarily need to be pushed into the main http://interfaces.juju.solutions. Could possibly use the same interface for launchpad as well

includes: ['lp/adam-stokes/nodejs']

charm build includes arch-specific wheels

I did 'charm build' from charm-tools-1.9.3-0ubuntu1~~ubuntu14.04.1~~ppa1 on this source:

https://code.launchpad.net/~azulcharmers/charms/trusty/zulu8/trunk

I did this on a ppc64le box. The built charm includes apparently ppc-specific stuff:

$ tree wheelhouse/ | grep ppc                        
├── MarkupSafe-0.23-cp27-none-linux_ppc64le.whl
├── PyYAML-3.11-cp27-none-linux_ppc64le.whl

Now when I attempt to deploy this local charm to Azure (with x86 units), I get this:

unit-zulu8-0[17465]: 2015-12-03 15:52:16 INFO unit.zulu8/0.install logger.go:40 Successfully installed pip
unit-zulu8-0[17465]: 2015-12-03 15:52:16 INFO unit.zulu8/0.install logger.go:40 Cleaning up...
unit-zulu8-0[17465]: 2015-12-03 15:52:16 INFO unit.zulu8/0.install logger.go:40 Ignoring indexes: https://pypi.python.org/simple
unit-zulu8-0[17465]: 2015-12-03 15:52:17 INFO unit.zulu8/0.install logger.go:40 MarkupSafe-0.23-cp27-none-linux_ppc64le.whl is not a supported wheel on this platform.
unit-zulu8-0[17465]: 2015-12-03 15:52:17 INFO unit.zulu8/0.install logger.go:40 Traceback (most recent call last):
unit-zulu8-0[17465]: 2015-12-03 15:52:17 INFO unit.zulu8/0.install logger.go:40   File "/var/lib/juju/agents/unit-zulu8-0/charm/hooks/install", line 18, in <module>
unit-zulu8-0[17465]: 2015-12-03 15:52:17 INFO unit.zulu8/0.install logger.go:40     check_call(['pip', 'install', '-U', '--no-index'] + glob('wheelhouse/*'))
unit-zulu8-0[17465]: 2015-12-03 15:52:17 INFO unit.zulu8/0.install logger.go:40   File "/usr/lib/python2.7/subprocess.py", line 540, in check_call
unit-zulu8-0[17465]: 2015-12-03 15:52:17 INFO unit.zulu8/0.install logger.go:40     raise CalledProcessError(retcode, cmd)
unit-zulu8-0[17465]: 2015-12-03 15:52:17 INFO unit.zulu8/0.install logger.go:40 subprocess.CalledProcessError: Command '['pip', 'install', '-U', '--no-index', 'wheelhouse/pip-7.1.2-py2.py3-none-any.whl', 'wheelhouse/netaddr-0.7.18-py2-none-any.whl', 'wheelhouse/MarkupSafe-0.23-cp27-none-linux_ppc64le.whl', 'wheelhouse/PyYAML-3.11-cp27-none-linux_ppc64le.whl', 'wheelhouse/Jinja2-2.8-py2.py3-none-any.whl', 'wheelhouse/charmhelpers-0.6.0-py2-none-any.whl', 'wheelhouse/charms.reactive-0.3.6-py2-none-any.whl', 'wheelhouse/pyaml-15.8.2-py2-none-any.whl', 'wheelhouse/Tempita-0.5.2-py2-none-any.whl', 'wheelhouse/six-1.10.0-py2.py3-none-any.whl']' returned non-zero exit status 1

Does charm-build need to support something like "target arch" so it can include the correct arch-dependent deps? Or should these all be noarch -- specifically MarkupSafe and PyYAML?

removing lib/<module>.pypi after initial build fails

From @battlemidget on October 19, 2015 2:38

I have 2 files in lib/:

lib/
   requests.pypi
   pep8.pypi

Their contents only contain the module name required. Next I run charm-compose and it builds successfully and includes both pep8 and requests modules in the final lib/ directory.

Now if you remove lib/requests.pypi and re-run charm-compose you get an unexpected file failure:

‹master*› »»»» charm-compose -l DEBUG                                                                      0|22:35:18
composer: Composing into /home/adam/charms
composer: {
  "COMPOSER_PATH": null, 
  "INTERFACE_PATH": null, 
  "JUJU_REPOSITORY": "/home/adam/charms", 
  "_charm": ".", 
  "_name": "node", 
  "config": "<charmtools.compose.config.ComposerConfig object at 0x7fef5deb2a90>", 
  "deps": "/home/adam/charms/deps/trusty", 
  "force": false, 
  "interface_service": "http://interfaces.juju.solutions", 
  "log_level": "DEBUG", 
  "output_dir": "/home/adam/charms", 
  "repo": "/home/adam/charms/trusty", 
  "series": "trusty", 
  "target_dir": "/home/adam/charms/trusty/node"
}
composer: Added unexpected file, should be in a base layer: lib/requests/packages/chardet/__init__.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/urllib3/connection.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/chardet/sbcharsetprober.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/chardet/langbulgarianmodel.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/chardet/gb2312prober.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/chardet/utf8prober.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/chardet/mbcssm.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/urllib3/util/request.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/chardet/charsetprober.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/chardet/euctwprober.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/chardet/constants.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/chardet/compat.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/chardet/universaldetector.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/chardet/escsm.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/urllib3/poolmanager.py
composer: Added unexpected file, should be in a base layer: lib/requests-2.8.1.dist-info/metadata.json
composer: Added unexpected file, should be in a base layer: lib/requests/packages/urllib3/fields.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/urllib3/_collections.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/urllib3/packages/ssl_match_hostname/__init__.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/urllib3/exceptions.py
composer: Added unexpected file, should be in a base layer: lib/requests/certs.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/urllib3/__init__.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/chardet/jpcntx.py
composer: Added unexpected file, should be in a base layer: lib/requests/api.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/chardet/sjisprober.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/urllib3/util/response.py
composer: Added unexpected file, should be in a base layer: lib/requests/utils.py
composer: Added unexpected file, should be in a base layer: lib/requests/structures.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/urllib3/util/__init__.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/urllib3/util/connection.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/urllib3/packages/ordered_dict.py
composer: Added unexpected file, should be in a base layer: lib/requests-2.8.1.dist-info/METADATA
composer: Added unexpected file, should be in a base layer: lib/requests/packages/urllib3/connectionpool.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/urllib3/packages/ssl_match_hostname/_implementation.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/chardet/big5freq.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/urllib3/util/timeout.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/urllib3/contrib/ntlmpool.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/chardet/euckrprober.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/chardet/langcyrillicmodel.py
composer: Added unexpected file, should be in a base layer: lib/requests/compat.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/urllib3/util/url.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/urllib3/response.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/chardet/chardistribution.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/chardet/langgreekmodel.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/chardet/escprober.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/chardet/big5prober.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/chardet/charsetgroupprober.py
composer: Added unexpected file, should be in a base layer: lib/requests/status_codes.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/urllib3/contrib/appengine.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/chardet/hebrewprober.py
composer: Added unexpected file, should be in a base layer: lib/requests/auth.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/chardet/langhungarianmodel.py
composer: Added unexpected file, should be in a base layer: lib/requests/exceptions.py
composer: Added unexpected file, should be in a base layer: lib/requests/__init__.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/urllib3/request.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/chardet/mbcharsetprober.py
composer: Added unexpected file, should be in a base layer: lib/requests-2.8.1.dist-info/DESCRIPTION.rst
composer: Added unexpected file, should be in a base layer: lib/requests/cacert.pem
composer: Added unexpected file, should be in a base layer: lib/requests/packages/chardet/euctwfreq.py
composer: Added unexpected file, should be in a base layer: lib/requests-2.8.1.dist-info/top_level.txt
composer: Added unexpected file, should be in a base layer: lib/requests/packages/__init__.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/urllib3/util/ssl_.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/urllib3/contrib/pyopenssl.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/urllib3/packages/six.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/chardet/euckrfreq.py
composer: Added unexpected file, should be in a base layer: lib/requests-2.8.1.dist-info/RECORD
composer: Added unexpected file, should be in a base layer: lib/requests/packages/chardet/latin1prober.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/chardet/gb2312freq.py
composer: Added unexpected file, should be in a base layer: lib/requests/hooks.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/chardet/jisfreq.py
composer: Added unexpected file, should be in a base layer: lib/requests/adapters.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/chardet/eucjpprober.py
composer: Added unexpected file, should be in a base layer: lib/requests/cookies.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/urllib3/filepost.py
composer: Added unexpected file, should be in a base layer: lib/requests/models.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/urllib3/contrib/__init__.py
composer: Added unexpected file, should be in a base layer: lib/requests-2.8.1.dist-info/WHEEL
composer: Added unexpected file, should be in a base layer: lib/requests/packages/chardet/langthaimodel.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/chardet/sbcsgroupprober.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/chardet/codingstatemachine.py
composer: Added unexpected file, should be in a base layer: lib/requests/sessions.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/chardet/mbcsgroupprober.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/chardet/cp949prober.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/chardet/chardetect.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/chardet/langhebrewmodel.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/urllib3/util/retry.py
composer: Added unexpected file, should be in a base layer: lib/requests/packages/urllib3/packages/__init__.py
Traceback (most recent call last):
  File "/usr/bin/charm-compose", line 9, in <module>
    load_entry_point('charm-tools==1.7.1', 'console_scripts', 'charm-compose')()
  File "/usr/lib/python2.7/dist-packages/charmtools/compose/__init__.py", line 468, in main
    composer()
  File "/usr/lib/python2.7/dist-packages/charmtools/compose/__init__.py", line 392, in __call__
    self.validate()
  File "/usr/lib/python2.7/dist-packages/charmtools/compose/__init__.py", line 384, in validate
    "Unable to continue due to unexpected modifications (try --force)")
ValueError: Unable to continue due to unexpected modifications (try --force)

I also noticed that subsequent charm-compose commands do not clean out the existing final charm's python dependencies.

Copied from original issue: canonical/charms.reactive#9

Move hook template out to layer

https://github.com/juju/charm-tools/blob/master/charmtools/build/tactics.py#L14

This template should be provided by the layers and not embedded in charm-tools. Define a standard for layers to provide it, and provide the current default in layer-basic.

metadata YAML error dumps stack trace

When opening the metadata.yaml of a charm layer, it dumps a long stack trace with an indicator of the error when using the currently available charm-tools from brew. charm-tools-1.7.1-yosemite.bottle

 charm compose -o $JUJU_REPOSITORY
Traceback (most recent call last):
  File "/usr/local/Cellar/charm-tools/1.7.1/libexec/bin/charm-compose", line 9, in <module>
    load_entry_point('charm-tools==1.7.1', 'console_scripts', 'charm-compose')()
  File "/usr/local/Cellar/charm-tools/1.7.1/libexec/lib/python2.7/site-packages/charm_tools-1.7.1-py2.7.egg/charmtools/compose/__init__.py", line 468, in main
    composer()
  File "/usr/local/Cellar/charm-tools/1.7.1/libexec/lib/python2.7/site-packages/charm_tools-1.7.1-py2.7.egg/charmtools/compose/__init__.py", line 388, in __call__
    self.find_or_create_repo()
  File "/usr/local/Cellar/charm-tools/1.7.1/libexec/lib/python2.7/site-packages/charm_tools-1.7.1-py2.7.egg/charmtools/compose/__init__.py", line 180, in find_or_create_repo
    self.create_repo()
  File "/usr/local/Cellar/charm-tools/1.7.1/libexec/lib/python2.7/site-packages/charm_tools-1.7.1-py2.7.egg/charmtools/compose/__init__.py", line 166, in create_repo
    self.target_dir = (self.repo / self.name)
  File "/usr/local/Cellar/charm-tools/1.7.1/libexec/lib/python2.7/site-packages/charm_tools-1.7.1-py2.7.egg/charmtools/compose/__init__.py", line 138, in name
    data = yaml.load(md.open())
  File "/usr/local/Cellar/charm-tools/1.7.1/libexec/lib/python2.7/site-packages/PyYAML-3.11-py2.7-macosx-10.10-intel.egg/yaml/__init__.py", line 71, in load
  File "/usr/local/Cellar/charm-tools/1.7.1/libexec/lib/python2.7/site-packages/PyYAML-3.11-py2.7-macosx-10.10-intel.egg/yaml/constructor.py", line 37, in get_single_data
  File "/usr/local/Cellar/charm-tools/1.7.1/libexec/lib/python2.7/site-packages/PyYAML-3.11-py2.7-macosx-10.10-intel.egg/yaml/composer.py", line 36, in get_single_node
  File "/usr/local/Cellar/charm-tools/1.7.1/libexec/lib/python2.7/site-packages/PyYAML-3.11-py2.7-macosx-10.10-intel.egg/yaml/composer.py", line 55, in compose_document
  File "/usr/local/Cellar/charm-tools/1.7.1/libexec/lib/python2.7/site-packages/PyYAML-3.11-py2.7-macosx-10.10-intel.egg/yaml/composer.py", line 84, in compose_node
  File "/usr/local/Cellar/charm-tools/1.7.1/libexec/lib/python2.7/site-packages/PyYAML-3.11-py2.7-macosx-10.10-intel.egg/yaml/composer.py", line 127, in compose_mapping_node
  File "/usr/local/Cellar/charm-tools/1.7.1/libexec/lib/python2.7/site-packages/PyYAML-3.11-py2.7-macosx-10.10-intel.egg/yaml/parser.py", line 98, in check_event
  File "/usr/local/Cellar/charm-tools/1.7.1/libexec/lib/python2.7/site-packages/PyYAML-3.11-py2.7-macosx-10.10-intel.egg/yaml/parser.py", line 428, in parse_block_mapping_key
  File "/usr/local/Cellar/charm-tools/1.7.1/libexec/lib/python2.7/site-packages/PyYAML-3.11-py2.7-macosx-10.10-intel.egg/yaml/scanner.py", line 116, in check_token
  File "/usr/local/Cellar/charm-tools/1.7.1/libexec/lib/python2.7/site-packages/PyYAML-3.11-py2.7-macosx-10.10-intel.egg/yaml/scanner.py", line 220, in fetch_more_tokens
  File "/usr/local/Cellar/charm-tools/1.7.1/libexec/lib/python2.7/site-packages/PyYAML-3.11-py2.7-macosx-10.10-intel.egg/yaml/scanner.py", line 580, in fetch_value
yaml.scanner.ScannerError: mapping values are not allowed here
  in "./metadata.yaml", line 10, column 10

I think we have an opportunity to capture this and remove the stack, and instead error out stating the metadata.yaml has errors.

test runner: timeout not being honored

Testing an openstack charm with involved setup. I'm setting the --timeout to 7200s but still get:

$ juju test -v -p AMULET_HTTP_PROXY tests/015-basic-trusty-icehouse --timeout 7200 --set-e
...
Connection Timeout: disconnecting client after 300.0 seconds
Timeout occurred, printing juju status...environment: local

However, afaict the setup is actually succeeding (later).

It would be great if the specified timeout could be passed on to all called clients.

Version: charm-tools 1.7.1-0ubuntu3~~ubuntu14.04.1~~ppa1

discussion around utilizing config.yaml options

One thing I'm noticing while developing layers is that while the config.yaml options bubble up to the topmost layer you can run into issues where config options may get merged if their keys are the same. For example, a port which could be a setting in an nginx layer, and a setting in yourapplication layer if setup to utilize the reverseproxy.

The way i've currently dealt with it is to prefix my config options, so for example, nginx I have nginx-port, and my web application will then have mywebapp-port.

Depending on the amount of layers you have this could lead to more headache trying to manage those config options from several layers. This should probably rarely be an issue as the amount of layers used per charm would be minimal I thought it'd be something to think about for documentation.

Error: Too many values to unpack

When using composer, with a layer that did not include the string

includes:

before defining the layers/interfaces, the error response is a stack trace and the message: "Too many values to unpack" - a pre-check lint of the yaml before attempting to process may fix this.

Charm build with missing `INTERFACE_PATH` cuases python traceback

I was trying to build the openjdk layer and got an error. it is unclear to me why the error occurred whether it was incorrect layer code or incorrect tool code.

mbruzek@warhorse:~/workspace/layers$ git clone [email protected]:juju-solutions/layer-openjdk
Cloning into 'layer-openjdk'...
remote: Counting objects: 20, done.
remote: Total 20 (delta 0), reused 0 (delta 0), pack-reused 20
Receiving objects: 100% (20/20), done.
Resolving deltas: 100% (6/6), done.
Checking connectivity... done.
mbruzek@warhorse:~/workspace/layers$ cd layer-openjdk/
mbruzek@warhorse:~/workspace/layers/layer-openjdk$ ls
config.yaml  copyright  layer.yaml  metadata.yaml  reactive  README.md
mbruzek@warhorse:~/workspace/layers/layer-openjdk$ export JUJU_REPOSITORY=/home/mbruzek/workspace/charms
mbruzek@warhorse:~/workspace/layers/layer-openjdk$ charm build
build: Composing into /home/mbruzek/workspace/charms
build: Processing layer: layer:basic
build: Processing layer: openjdk
build: Processing interface: java
Traceback (most recent call last):
  File "/usr/bin/charm-build", line 9, in <module>
    load_entry_point('charm-tools==1.9.3', 'console_scripts', 'charm-build')()
  File "/usr/lib/python2.7/dist-packages/charmtools/build/__init__.py", line 525, in main
    build()
  File "/usr/lib/python2.7/dist-packages/charmtools/build/__init__.py", line 423, in __call__
    self.generate()
  File "/usr/lib/python2.7/dist-packages/charmtools/build/__init__.py", line 388, in generate
    self.exec_plan(self.plan, self.layers)
  File "/usr/lib/python2.7/dist-packages/charmtools/build/__init__.py", line 360, in exec_plan
    cont &= tactic.lint()
TypeError: unsupported operand type(s) for &=: 'bool' and 'NoneType'