Code Monkey home page Code Monkey logo

duffle's Introduction

Duffle: The CNAB Installer

๐Ÿšจ The Duffle project has been archived and is no longer maintained. See https://cnab.io/community-projects/ for current implementations of the CNAB specification.

Duffle is the reference implementation of the CNAB specification. It provides a comprehensive mapping of all features of the CNAB Core specification as of version 1.0.1.

Duffle's main utility, now that much of its internal code has migrated to the official cnab-go Golang library, is to demonstrate working proof-of-concepts of additions or modifications to applicable CNAB specifications. Of course, Duffle may still be used to install, manage and author bundles at a low level.

Future conformance updates per the Core and other CNAB specifications are intended to be added in the aforementioned cnab-go library, rather than in Duffle, which will not be updated going forward. Duffle will remain a reference implementation of the CNAB spec as of v1.0.1. If you are interested in using an up-to-date CNAB tool, check out https://cnab.io/community-projects/#tools.

The community has created implementations of the CNAB spec with opinionated takes on authoring bundles. Some even use Duffle's libraries to handle the CNAB implementation. If you want to make your own CNAB tooling, that is a great place to start!

Learn more about about CNAB and Duffle, check out our docs.

Getting Started

  1. Get the latest Duffle release for your operating system.

  2. Run the command to set duffle up on your machine:

    $ duffle init
    ==> The following new directories will be created:
    /home/janedoe/.duffle
    /home/janedoe/.duffle/bundles
    /home/janedoe/.duffle/logs
    /home/janedoe/.duffle/plugins
    /home/janedoe/.duffle/claims
    /home/janedoe/.duffle/credentials
    ==> The following new files will be created:
    /home/janedoe/.duffle/repositories.json
  3. Build and install your first bundle (you can find the examples directory in this repository):

    $ duffle build ./examples/helloworld/
    Step 1/6 : FROM alpine:latest
     ---> e21c333399e0
    Step 2/6 : RUN apk update
     ---> Running in 93480e25ef09
    fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/main/x86_64/APKINDEX.tar.gz
    fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/community/x86_64/APKINDEX.tar.gz
    v3.7.3-40-g354ae2b18a [http://dl-cdn.alpinelinux.org/alpine/v3.7/main]
    v3.7.3-38-gb9b86f0506 [http://dl-cdn.alpinelinux.org/alpine/v3.7/community]
    OK: 9055 distinct packages available
     ---> 4123d6b1dfc5
    Step 3/6 : RUN apk add -u bash
     ---> Running in 3db9dd96e10b
    (1/10) Upgrading busybox (1.27.2-r6 -> 1.27.2-r11)
    Executing busybox-1.27.2-r11.post-upgrade
    (2/10) Upgrading libressl2.6-libcrypto (2.6.3-r0 -> 2.6.5-r0)
    (3/10) Installing libressl2.6-libtls (2.6.5-r0)
    (4/10) Installing ssl_client (1.27.2-r11)
    (5/10) Installing pkgconf (1.3.10-r0)
    (6/10) Installing ncurses-terminfo-base (6.0_p20171125-r1)
    (7/10) Installing ncurses-terminfo (6.0_p20171125-r1)
    (8/10) Installing ncurses-libs (6.0_p20171125-r1)
    (9/10) Installing readline (7.0.003-r0)
    (10/10) Installing bash (4.4.19-r1)
    Executing bash-4.4.19-r1.post-install
    Executing busybox-1.27.2-r11.trigger
    OK: 13 MiB in 19 packages
     ---> 5a3670bf25d9
    Step 4/6 : COPY Dockerfile /cnab/Dockerfile
     ---> 58548d5a8553
    Step 5/6 : COPY app /cnab/app
     ---> 46ce2cca5f93
    Step 6/6 : CMD ["/cnab/app/run"]
     ---> Running in d2294cc8b7fd
     ---> 69abe3476d43
    Successfully built 69abe3476d43
    Successfully tagged cnab/helloworld-cnab:87d786be507769a4913c90d85134c85727c85f41
    ==> Successfully built bundle helloworld:0.1.1
  4. Check that it was built:

    $ duffle bundle list
    NAME            VERSION DIGEST
    helloworld      0.1.1   fae0c3a28bd850f6a9a2631b9abe4f8244c83ee4
  5. Now run it:

    $ duffle credentials generate helloworld-creds helloworld:0.1.1
    $ duffle install helloworld-demo -c helloworld-creds helloworld:0.1.1
    Executing install action...
    hello world
    Install action
    Action install complete for helloworld-demo
  6. Clean up:

    $ duffle uninstall helloworld-demo
    Executing uninstall action...
    hello world
    uninstall action
    Action uninstall complete for helloworld-demo

    Notes:

    • To build and install bundles, you need access to a Docker engine - it can be Docker for Mac, Docker for Windows, Docker on Linux, or a remote Docker engine. Duffle uses the Docker engine to build the invocation images, as well as for running actions inside invocation images.
    • Duffle has a driver architecture for different ways of executing actions inside invocation images, and more drivers will be available in the future.
    • Learn more about what a bundle is and its components here.
    • Get a feel for what CNAB bundles written for Duffle look like by referencing the examples directory.

Developing Duffle

See the Developer's Guide.

duffle's People

Contributors

adamreese avatar astrieanna avatar benpatt avatar benwhiting avatar carolynvs avatar carolynvs-msft avatar cbrit avatar chris-crone avatar fibonacci1729 avatar glyn avatar isurulucky avatar itowlson avatar jcsirot avatar jeremyrickard avatar jlegrone avatar johnmccabe avatar krisnova avatar moeryomenko avatar radu-matei avatar ryanmoran avatar santiagotorres avatar scothis avatar silvin-lubecki avatar simonferquel avatar simongdavies avatar st3v avatar tariq1890 avatar technosophos avatar vdice avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

duffle's Issues

Go: `duffle run TARGET`

Per @fibonacci1729 's suggestion, we should be able to add duffle exec TARGET, where TARGET is a user-specified name. This would execute an action in the invocation image whose name is the TARGET.

$ duffle install my_thing example/foo:1.2.3
$ duffle exec foobar my_thing example/foo:1.2.3  # Executes action=foobar in claim my_thing 

My gut feel is that we would merely execute that action and then pipe the results to STDOUT, but not change the claim.

Other options:

  • could generate a new revision (this might be a good idea) of the claim

I don't know what we do if the exec'ed option fails.

Go: Create `duffle list`

The duffle list command should print one line for every Claim in the claim storage. It should provide a tabular list of info about the installed CNAB apps that the claim storage knows about.

Go: Credentialset resolving

Depends on #62

In this part, we need to write a library that takes a credentialset definition and resolves the source data and places them into a destination. SO this involves doing logic to read the credentialset.CRED.source and reading files, reading env vars, reading static values, and executing commands, then taking the results and putting them into a destination datastructure.

Proposal: how to pass the bundle to duffle install command

Right now, pointing which bundle to install is done by referencing the container image of the bundle:

$ duffle install <installation-name> <bundle-container-image> <other-flags>

This is very specific for the Docker driver, meaning that for other drivers, this might not have a correspondent.
This is why it would probably make more sense to pass, instead of the container image, the bundle itself.

Starting with #73 we have support for bundle repos - so in theory, we could reference a bundle very much like we do Helm charts: <repo/bundle> - having the install command as follows:

$ duffle install <installation-name> <repo/bundle> <other-flags>

In addition, I think it would make things easier if we also supported installing bundles that are not in a repo, by passing a bundle file - local, or remote - the analogy for this would be kubectl create -f <file>:

$ duffle install <installation-name> -f <bundle-file> <other-flags>

Do these two methods seem a good starting point to reference the bundle to install?

Go: Write storage layer for storing claims

We need a nice pretty CRUD interface for storing claims. For now, the default implementation can be local (on disk) storage. But we want to make it work for object storage, a database, or some other backend.

CLI: implement duffle logs

Based on the current implementation, the builder writes the build logs under ~/.duffle/logs.

We should implement duffle logs, which reads the log files - the implementation should be fairly similar to Kubed/Draft.

Generate manifest from duffle.toml

The initial input of a duffle build is a duffle.toml file; its output is a manifest.json file, which can then be used as input for duffle install, together with a file containing values for parameters.

This issue is to track the initial generation of the manifest from duffle.toml.

Go: `duffle uninstall`

The duffle uninstall command should run the uninstall action on a Claim.

$ duffle uninstall foo

The above should find the claim named foo and run the CNAB for that bundle, with the action set to uninstall

Most of the logic is present in the duffle install command.

It is an open question whether we need to allow parameter values to be passed to Duffle for an uninstall.

Depends on #47

duffle build: implement container builder

We need a way to build the containers required in a CNAB bundle. There's a WIP over at #44 which implements building the containers individually by pulling most of the code from previous work (kubed).

There's still a few things we need to do though:

  • change it to build only the Dockerfiles present in duffle.toml rather than searching for every Dockerfile from the root directory
  • come up with a "manifest" format that is constructed during either duffle build or duffle push

We need the manifest so that we have the "bill of materials" for running a CNAB bundle via duffle pull/duffle run. @radu-matei has done some research/experimentation on how Docker creates and manages these manifests, and there's one proposal in #23 (comment) that mimics the Docker multi-arch manifest artifact. Right now the question stands: can we upload a custom manifest via duffle pull, or will the registry refute an invalid schema? We're running under the assumption that it doesn't. As long as we implement the schemaVersion and mediaType present in our custom manifest, we hope to be able to upload a custom manifest to a registry...

We have Hope. Rebellions are built on Hope. โ„ข๏ธ

Go: `duffle status`

The duffle status command loads an image from an existing Claim and runs the status action on that container.

$ duffle install my_app example/new_bundle:1.2.3
$ duffle status my_app    # runs the `status` action in example/new_bundle:1.2.3

Note that the intention is that the CNAB bundle itself is what generates the status data. So all we are doing is piping the output of that call to STDOUT.

Most of the base logic for this exists in duffle install #47

Go: `duffle credentials` command group

We need a way to generate credentialsets. I am guessing we want a suite of commands under duffle credentials. However, I don't have a strong set of opinions about what these tools should look like.

Probably, as a starting point, we should do duffle credentials list, which would just list the different credentialset names:

$ duffle credentials list
production
staging
development
personal

docs: proposal for CNAB Architecture

We should determine the basic underlying architecture/layout of a CNAB manifest and document it. The architecture should be able to handle all the existing example use cases:

#11
#12
#13
#14
#15
#16
#21

relatedly, it should also present a proposal for how a CNAB manifest is addressed in storage via #5 and #6.

Go: Installing duffle

Duffle currently needs a home directory configured with a few things like a credentials directory.

We should probably create a duffle init or something similar, and have that scaffold out the $HOME/.duffle directory.

TODO: Should we name this duffle init? Or is there a better name? Or do we do things from an external tool?

Research: Write a `nomad` driver

Duffle exposes a driver layer where custom drivers can be used to execute the bundle's invocation image. This can be done via shell script, Go program, python, node.js... whatever....

What would be nice is to be able to demo this with Nomad.

Depends on #47

Go: `duffle upgrade`

The duffle upgrade command takes a claim name and a new bundle, and it replaces the claim's old bundle with the new bundle, executing the upgrade action on the new bundle.

This one may be rather complex, so don't feel like we need to solve all of the problems.

$ duffle upgrade my_app example/my_bundle:1.2.3
# my_app is created
$ duffle upgrade my_app example/my_bundle:2.3.4

Most of the base logic can be found in duffle install in #47

Go: Write replacement logic for image references

The bundle.json (formerly manifest.json) contains replacement rules that look like this:

"images": [โ€‹
    { โ€‹
      "name": "frontend",โ€‹
      "uri": "gabrtv.azurecr.io/gabrtv/vote-frontend:a5ff67...",โ€‹
      "refs": [โ€‹
        {โ€‹
          "path": "./charts/azure-voting-app/values.yaml",โ€‹
          "field": "AzureVoteFront.deployment.image"โ€‹
        }โ€‹
      ]โ€‹
    }
]

We need a library implementation that reads the file at path, determines whether it is JSON, YAML, XML or (stretch goal) TOML, and then replaces the value at field with the given rewritten image name.

Go: `duffle downgrade`

The duffle downgrade command rolls back to a previous version of a release by executing the downgrade action on the older bundle

This one may be overly complex to do well, and may be better handled by having users just run upgrade on older images. So don't do this one w/o giving the team a chance go offer feedback.

Using AZ to execute a VM image instead of a container image

The idea would be to do something like this:

  • Have the driver generate a cloud-init file and then run az vm create
  • The Cloud-Init should...
    • Use runcmd to load env vars, then run /cnab/app/run
    • Use power_state to shutdown the VM
  • az vm create should (at minimum) set the resource group, and set the VM image to whatever the bundle.json specifies

Research: How does Packer's workflow work?

We'd like to support Packer as a build target. What is the equivalent workflow to Docker's docker build && docker push? What do we need to do to configure for Packer?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.