edeliver / edeliver Goto Github PK

View Code? Open in Web Editor NEW

2.0K 38.0 185.0 1.39 MB

Deployment for Elixir and Erlang

Shell 42.74% Perl 18.46% Elixir 36.68% Erlang 2.08% Dockerfile 0.05%

edeliver's People

Stargazers

Watchers

Forkers

msch hamann liveforeverx norbu09 freshvolk appunite dlozano eugenehp tehprofessor substantial ubernil linearregression icicletech hippware geekbull col talklittle manukall gaynetdinov aaronjensen kirillzubovsky tralamazza danhper dustinfarris cclam0827 simple555a romul frost nashangban ascrookes fpalluel snewcomer schnittchen mfeckie rmoorman tonyarkles bombinatetech amokan st23am fredbots crowdhailer scout-e-recruitment hackishword oleghaidul rvanlieshout labzero shosti niklas harrisonl bartotten justinaiken jesseshieh 221v xtian arnodirlam novaugust hidetakakojo sorliem msrdjan kfit-dev gullitmiranda dexterogieosahon dhainlin seeekr octosteve jeizsm benrom yammine gducharme otaq amco 023soft zhuyihome belousovav verypossible oivoodoo gpad montebrown ellispritchard lattenwald dmitryplatonov lessless maggy96 jonnymo5 brienw britto building39 phonbopit h1u2i3 denispeplin bitwalker qhwa tonilapuyade johannese namjae tchernicum joshrieken eraclitux m8thub boomerchi

edeliver's Issues

`mix edeliver build upgrade` should exit non-zero if it fails:

expect a non-zero exit code, got zero

BUILDING UPGRADE OF BOOKING APP ON BUILD HOST

-----> Authorizing hosts
-----> Validating * version 1.0.0-16-g9aa99a2 is in s3 release store
-----> Ensuring hosts are ready to accept git pushes
-----> Pushing new commits with git to: [email protected]

FAILED  1:
git push --tags -f [email protected] edge &> /dev/null

mix edeliver build release fails to find built release

$ mix edeliver build release --verbose
...
==> Building release with MIX_ENV=prod.
==> Generating relx configuration...
==> Generating sys.config...
==> Generating boot script...
==> Performing protocol consolidation...
==> Conform: Loading schema...
==> Conform: No schema found, conform will not be packaged in this release!
==> Generating release...
==> Generating nodetool...
==> Packaging release...
==> The release for myapp-0.0.1 is ready!
-----> Copying release to local release store
-----> Copying myapp-0.0.1.tar.gz to release store
scp: /tmp/erlang/myapp/builds/rel/myapp/myapp-0.0.1.tar.gz: No such file or directory

RELEASE BUILD OF MYAPP WAS SUCCESSFUL!

But there is a file /tmp/erlang/myapp/builds/rel/myapp/releases/0.0.1/myapp.tar.gz

$ ls /tmp/erlang/myapp/builds/rel/myapp/releases/0.0.1/
итого 20660
drwxrwxr-x 2 deployer deployer     4096 Июл 10 11:34 ./
drwxrwxr-x 3 deployer deployer     4096 Июл 10 11:34 ../
-rw-rw-r-- 1 deployer deployer    37894 Июл 10 11:34 myapp.boot
-rw-rw-r-- 1 deployer deployer      548 Июл 10 11:34 myapp.rel
-rw-rw-r-- 1 deployer deployer    51350 Июл 10 11:34 myapp.script
-rw-rw-r-- 1 deployer deployer 20997987 Июл 10 11:34 myapp.tar.gz
-rw-rw-r-- 1 deployer deployer    37894 Июл 10 11:34 start.boot
-rw-rw-r-- 1 deployer deployer      912 Июл 10 11:34 sys.config
-rw-rw-r-- 1 deployer deployer      421 Июл 10 11:34 vm.args

And mix edeliver deploy release fails too:

$ mix edeliver deploy release to staging
DEPLOYING RELEASE OF MYAPP APP TO STAGING HOSTS

-----> Authorizing hosts
ls: .deliver/releases/myapp_*.*.tar.gz: No such file or directory
No *(s) found in local release store.
You can build one with the ./edeliver build * task.

Any insights how to build proper release?

Compile step from README fails creating initial release

I was following the README instructions to test out a deployment using edeliver all on my local machine. I'm using elixir 1.2.0, edeliver 1.2.4, and exrm 1.0.4, running on OSX. This is the output:

mix edeliver build release

BUILDING RELEASE OF WEBHOOKS APP ON BUILD HOST

-----> Authorizing hosts
-----> Ensuring hosts are ready to accept git pushes
-----> Pushing new commits with git to: [email protected]
-----> Resetting remote hosts to 3a545059f53b2fc07436acffd97c8758772bc93a
-----> Cleaning generated files from last build
-----> Fetching / Updating dependencies
-----> Compiling sources

== Compilation error on file lib/webhooks/event.ex ==
** (ArgumentError) Poison.Encoder is not available, cannot derive Poison.Encoder for Webhooks.Event
    (elixir) lib/protocol.ex:69: Protocol.assert_protocol!/2
    (elixir) lib/protocol.ex:582: Protocol.derive/5
    (stdlib) lists.erl:1337: :lists.foreach/2
    (elixir) lib/protocol.ex:570: Protocol.__derive__/3
    lib/webhooks/event.ex:40: (module)


== Compilation error on file lib/event_poster/handler.ex ==
** (CompileError) lib/event_poster/handler.ex:7: Webhooks.Event.__struct__/0 is undefined, cannot expand struct Webhooks.Event
    (elixir) src/elixir_map.erl:58: :elixir_map.translate_struct/4
    (stdlib) lists.erl:1353: :lists.mapfoldl/3


== Compilation error on file lib/event_poster.ex ==
** (CompileError) lib/event_poster.ex:20: NSQ.Config.__struct__/0 is undefined, cannot expand struct NSQ.Config
    (elixir) src/elixir_map.erl:58: :elixir_map.translate_struct/4
    lib/event_poster.ex:19: (module)

FAILED 1:

    [ -f ~/.profile ] && source ~/.profile
    set -e
    cd .deliver
    if [ "mix" = "rebar" ]; then
      echo "using rebar to compile files"
      [[ "" != "true" ]] && ./rebar  clean skip_deps=true || :
      ./rebar  compile
    elif [ "mix" = "mix" ] && [ "mix" = "mix" ]; then
      if [[ "" = "true" ]]; then
        APP="webhooks" MIX_ENV="prod" AUTO_VERSION="" mix do compile
      else
        APP="webhooks" MIX_ENV="prod" AUTO_VERSION="" mix do clean, compile
      fi
    elif [ "mix" = "mix" ]; then
      echo "using mix to compile files"
      if [[ "" = "true" ]]; then
        APP="webhooks" MIX_ENV="prod" AUTO_VERSION="" mix do deps.compile, compile
      else
        APP="webhooks" MIX_ENV="prod" AUTO_VERSION="" mix do clean, deps.compile, compile
      fi
    fi

I did some testing locally and found that the command APP="webhooks" MIX_ENV="prod" AUTO_VERSION="" mix do clean, compile was the only one throwing that error.

And this is my config file:

#!/usr/bin/env bash
APP="webhooks" # name of your release
BUILD_HOST="127.0.0.1" # host where to build the release
BUILD_USER="maxschnur" # local user at build host
BUILD_AT=".deliver" # build directory on build host
RELEASE_DIR="/opt/webhooks" # deploy directory on production hosts

If I run SKIP_MIX_CLEAN=true mix edeliver build release, compilation succeeds but it then fails at the "Generating release" step with this error:

zsh:2: no matches found: /opt/webhooks/releases/*/
Failed to detect generated release version at:
[email protected]:/opt/webhooks/releases/

No built release found.

Something seems off here... if this is the first release, why does it need to find an existing built release? From what I can tell, this error happens because there are no folders in /opt/webhooks. Manually creating an empty folder with any name in that directory fixes the error. Alternatively, explicitly setting the RELEASE_VERSION environment variable also fixes it.

I should mention that this error was after getting an error about RELEASE_DIR not being set. I added that to my config to get around the issue, but I didn't see that in the example config or in the example command-line, where it would be valid as an environment variable.

So, overall I was able to work around this, but I'm unsure of what the correct behavior is actually supposed to be. Should I be manually defining RELEASE_VERSION on first build? Should I need to set SKIP_MIX_CLEAN? Is there some other configuration I've messed up? Is what I did expected and it's just documentation that needs an update?

module ReleaseManager.Plugin is not loaded and could not be found

I am unable to compile edeliver from master:

Compiled lib/edeliver.ex

== Compilation error on file lib/exrm/plugins/link_vm_args.ex ==
** (CompileError) lib/exrm/plugins/link_vm_args.ex:2: module ReleaseManager.Plugin is not loaded and could not be found
    (elixir) expanding macro: Kernel.use/1
    lib/exrm/plugins/link_vm_args.ex:2: ReleaseManager.Plugin.LinkVMARgs (module)
    (elixir) lib/kernel/parallel_compiler.ex:97: anonymous fn/4 in Kernel.ParallelCompiler.spawn_compilers/8


== Compilation error on file lib/exrm/plugins/link_sys_config.ex ==
** (CompileError) lib/exrm/plugins/link_sys_config.ex:2: module ReleaseManager.Plugin is not loaded and could not be found
    (elixir) expanding macro: Kernel.use/1
    lib/exrm/plugins/link_sys_config.ex:2: ReleaseManager.Plugin.LinkSysConfig (module)
    (elixir) lib/kernel/parallel_compiler.ex:97: anonymous fn/4 in Kernel.ParallelCompiler.spawn_compilers/8

I've tried to call mix deps.compile edeliver and mix deps.update edeliver - no success.

Is edeliver still maintained?

Hey all,

So far edeliver seems great. Thanks for putting it out there!

There are a couple issues and some pull requests for some of them, but it seems there has bene no activity since July. Is it boldpoker's plan to continue maintaining edeliver? Are you looking for new maintainers? Some status would be appreciated while we start to depend on it.

Thanks!

/cc @bharendt

Could not compile :edeliver

I can't compile edeliver.

Could not compile :edeliver, no "mix.exs", "rebar.config" or "Makefile" (pass :compile as an option to customize compilation, set it to "false" to do nothing)

my mix.exs

  def application do
    [mod: {Pxblog, []},
     applications: [:phoenix, :phoenix_html, :cowboy, :logger, :gettext,
                    :phoenix_ecto, :postgrex, :comeonin, :earmark, :conform,
                    :conform_exrm, :edeliver]]
  end
  defp deps do
    [{:phoenix, "~> 1.1.4"},
     {:postgrex, ">= 0.0.0"},
     {:phoenix_ecto, "~> 2.0"},
     {:phoenix_html, "~> 2.4"},
     {:phoenix_live_reload, "~> 1.0", only: :dev},
     {:gettext, "~> 0.9"},
     {:cowboy, "~> 1.0"},
     {:comeonin, "~> 2.1"},
     {:ex_machina, "~> 0.6", only: :test},
     {:earmark, "~> 0.2.1"},
     {:exrm, "~> 1.0.0-rc7"},
     {:conform, "~> 1.0.0-rc8"},
     {:conform_exrm, "~> 0.2.0"},
     {:edeliver, "~> 1.1"}
   ]
  end

Option --mix-env doesn't seem to work

When I run

$ mix edeliver build release --mix-env=qa --verbose

I got the following error:

** (Mix.Config.LoadError) could not load config config/--mix-env=qa.exs

Apparently the whole --mix-env=env option is treated as MIX_ENV value.

Add rebar template that creates adds the update command and the unpack_release.escript

`--increment-version` option breaks `relup` file

Hello.

I've just noticed that if I set --increment-version option in the upgrade command, generated relup file would contain incorrect references to the old release.

This is beginning of the generated relup file of the upgrade which I generated with --increment-version=minor:

{"0.1.0+deploy-with-edeliver-fc42ded",
 [{"0.1.0+deploy-with-edeliver-1c05295",[],

The problem here is that previous release also got version bump here. Currently deployed release called 0.0.1+deploy-with-edeliver-1c05295, not 0.1.0+deploy-with-edeliver-1c05295, only the new release should get a version bump. Since there is no release with 0.1.0+deploy-with-edeliver-1c05295 name, mix edeliver deploy upgrade got broken.

Publish on hex.pm

This should be available via hex, see https://hex.pm/docs/publish

deploy release command shows upgrades

mix edeliver deploy release to staging shows a list of releases AND upgrades, while mix edeliver deploy upgrade to staging shows only upgrades.

Mark new release as 'permanent' in erlang-install-update strategy

Update releases/RELEASE file and mark old permanent release as 'old' and new 'unpacked' release as 'permanent'.
Could maybe better be done in release/bin/unpack_release.escript or in 'update' task of release/bin/app_binary.

build upgrades not incrementing versions

I am trying build and hot deploy an upgrade from a branch. I use the following cmd to run the upgrade build task.

mix edeliver build upgrade --from=0.0.1 --branch=exception_logging --verbose

To get this cmd to complete successfully I have to bump the version in my mix file or I would get the following error:

FAILED 1:

    [ -f ~/.profile ] && source ~/.profile
    set -e
    cd "/data/web/staging-my_app.example.com/my_app"
    [[ -f "my_appt-0.0.2.tar.gz" ]] && rm "my_app-0.0.2.tar.gz"
    [[ -n "releases/0.0.2" ]] && [[ -d "releases/0.0.2" ]] && rm -rf "releases/0.0.2"

After bumping the version in the mix file It would finish without errors, but would then download an upgrade as my_app_0.0.1.upgrade.tar.gz

The output at the end of the task:

==> All dependencies have appups ready for release!
==> Generated .appup for my_app 0.0.1 -> 0.0.2
===> relup successfully created!
==> The release for my_app-0.0.2 is ready!
==> You can boot a console running your release with `$ rel my_app/bin my_app console`
-----> Removing built release 0.0.2 from remote release directory
-----> Copying release 0.0.1 to local release store
-----> Copying my_app.tar.gz to release store

If I tried to deploy the upgrade zipe and it won't change the code running on the server. I was checking this by running a remote_console and triing to run new code.

Here is a copy of my config

!/usr/bin/env bash

APP="my_app" # name of your release
LINK_SYS_CONFIG="/data/shared/sys.config"
RELEASE_DIR="/data/web/staging-myapp.example.com/build/rel/my_app"
RELEASE_VERSION=0.0.2

BUILD_HOST="162.248.180.55" # host where to build the release
BUILD_USER="deploy" # local user at build host
BUILD_AT="/data/web/staging-myapp.example.com/build" # build directory on build host

STAGING_HOSTS="162.248.180.55" # staging / test hosts separated by space
STAGING_USER="deploy" # local user at staging hosts
TEST_AT="/data/web/staging-myapp.example.com/current" # deploy directory on staging hosts. default is DELIVER_TO

PRODUCTION_HOSTS="162.248.180.39" # deploy / production hosts separated by space
PRODUCTION_USER="deploy" # local user at deploy hosts
DELIVER_TO="/data/web/myapp.example.com/current" # deploy directory on production hosts

pre_erlang_clean_compile() {
status "Running phoenix.digest" # log output prepended with "----->"
__sync_remote " # runs the commands on the build host
set -e # fail if any command fails (recommended)
cd $BUILD_AT # enter the build directory on the build host (required)
# run your custom task
APP="$APP" MIX_ENV="$TARGET_MIX_ENV" $MIX_CMD phoenix.digest
"
}

Found several ecto repository modules

I am deploying a phoenix application using edeliver 1.2.3, ecto 1.1.4, swoosh 0.3.0.

mix edeliver show migrations on production

Returns the following error

  response: ==> Found several ecto repository modules ([MyApp.Mailer, MyApp.Repo]).
    Please specify the repository to use in the edeliver config as ECTO_REPOSITORY env.
<<"error">>

The MyApp.Mailer module looks like this. I am using swoosh to send email https://github.com/swoosh/phoenix_swoosh

defmodule MyApp.Mailer do
    use Swoosh.Mailer, otp_app: :designdrop
end

It looks like the Swoosh.Mailer is being picked up as a potential ecto repo.

I tried setting the ECTO_REPOSITORY environment variable on my local machine before running the command: ECTO_REPOSITORY="MyApp.Repo" mix edeliver show migrations on production, but this didn't work.

Tried exporting the ECTO_REPOSITORY in the ~/.profile file on the production machine, but that didn't work either.

Tried adding "ECTO_REPOSITORY="MyApp.Repo" to the .deliver/config file. No luck.

The message "Please specify the repository to use in the edeliver config as ECTO_REPOSITORY env." is a bit confusing.

How should I export this environment variable?

Do not extract initial release if folder exists already

maybe that should be allowed when executed with --force

Hard to debug deploy failures

I'm getting a variety of hard to debug deploy failures. I've had some success but lots of failure, and it has been hard to tell why some things are working and others not.

I'm attempting to build and deploy to the same server, and use my local machine as the release store.
I tried removing the application, and the builds folder from my production sever, as well as removing all releases from my local .deliver folder

This is my .edeliver/config:

#!/usr/bin/env bash

APP="xxx" # name of your release

BUILD_HOST="xxx" # host where to build the release
BUILD_USER="xxx" # local user at build host
BUILD_AT="/home/xxx/builds" # build directory on build host

PRODUCTION_HOSTS="xxx" # deploy / production hosts separated by space
PRODUCTION_USER="xxx" # local user at deploy hosts
DELIVER_TO="/home/xxx/xxx/" # deploy directory on production hosts

GIT_CLEAN_PATHS=${GIT_CLEAN_PATHS:="rel"}

I then run mix edeliver build release --branch=master --verbose and I get the following error:

using mix to generate release
Building release with MIX_ENV=prod.
This is an upgrade, verifying appups exist for updated dependencies..
==> All dependencies have appups ready for release!
==> Generated .appup for xxx 0.0.1 -> 0.0.7

FAILED 1:

    [ -f ~/.profile ] && source ~/.profile
    set -e
    cd /home/xxx/builds
    if [ "mix" = "rebar" ]; then
      echo "using rebar to generate release"
      ./rebar   -f generate
    elif [ "mix" = "relx" ]; then
      echo "using relx to generate release"
      ./relx release
    elif [ "mix" = "mix" ]; then
      echo "using mix to generate release"
      MIX_ENV="prod" LINK_SYS_CONFIG="" LINK_VM_ARGS="" APP="xxx" mix release  --no-confirm-missing
    fi

Don't know how to debug based on that that message. I've been getting similar failure messages when trying to do releases and upgrades. I'm sure there must be something wrong with my config file or server setup but hard to know what.

Any help appreciated. I assume this is not an edeliver error and if so I can post this in StackOverflow if need be.

refusing to update checked out branch: refs/heads/master error

It seems that now edeliver does not handle well cases when SSH connection was dropped.

I've tried to run my build twice:

vagrant@trusty64:/vagrant$ mix edeliver build release

BUILDING RELEASE OF PLUMBER APP ON BUILD HOST

-----> Authorizing hosts
-----> Ensuring hosts are ready to accept git pushes
-----> Pushing new commits with git to: [email protected]
-----> Resetting remote hosts to 97f757b0fee0f7ac7602152472cf11bc301bc346
-----> Stashing build directory on remote build host
-----> Fetching / Updating dependencies
-----> Compiling sources
-----> Generating release

FAILED  1:
ssh -o ConnectTimeout=3 [email protected]
    set -e
    [ -f ~/.profile ] && source ~/.profile
    cd /app/myapp/build &> /dev/null
    if [ "mix" = "rebar" ]; then
      echo "using rebar to generate release" &> /dev/null
      ./rebar   -f generate &> /dev/null
    elif [ "mix" = "relx" ]; then
      echo "using relx to generate release" &> /dev/null
      ./relx release &> /dev/null
    elif [ "mix" = "mix" ]; then
      echo "using mix to generate release" &> /dev/null
      MIX_ENV=prod mix release &> /dev/null
    fi

   &> /dev/null

vagrant@trusty64:/vagrant$ mix edeliver build release

BUILDING RELEASE OF PLUMBER APP ON BUILD HOST

-----> Authorizing hosts
-----> Ensuring hosts are ready to accept git pushes
-----> Pushing new commits with git to: [email protected]
-----> Resetting remote hosts to 97f757b0fee0f7ac7602152472cf11bc301bc346
-----> Stashing build directory on remote build host

FAILED  1:
ssh -o ConnectTimeout=3 [email protected]
    set -e
    [ -f ~/.profile ] && source ~/.profile
    cd /app/myapp/build &> /dev/null
    [[ $(git stash list | wc -l) -gt 0 ]] && _oldest_stash=$(git stash list | tail -1 | grep -o 'stash@{[0-9]\+}' | grep -o '[0-9]\+') || _oldest_stash=0
    [[ ${_oldest_stash} -gt 50 ]] && git stash drop stash@{$_oldest_stash} &> /dev/null
    git stash --all &> /dev/null
   &> /dev/null

And then, for the next time I got the following error:

-----> Pushing new commits with git to: [email protected]
remote: error: refusing to update checked out branch: refs/heads/master
remote: error: By default, updating the current branch in a non-bare repository
remote: error: is denied, because it will make the index and work tree inconsistent
remote: error: with what you pushed, and will require 'git reset --hard' to match
remote: error: the work tree to HEAD.
remote: error:
remote: error: You can set 'receive.denyCurrentBranch' configuration variable to
remote: error: 'ignore' or 'warn' in the remote repository to allow pushing into
remote: error: its current branch; however, this is not recommended unless you
remote: error: arranged to update its work tree to match what you pushed in some
remote: error: other way.
remote: error:
remote: error: To squelch this message and still keep the default behaviour, set
remote: error: 'receive.denyCurrentBranch' configuration variable to 'refuse'.
To [email protected]:/app/myapp/build
 ! [remote rejected] master -> master (branch is currently checked out)
error: failed to push some refs to '[email protected]:/app/myapp/build'

Removing and cloning the repo again on the target machine resolves the issue. It would be nice if edeliver cleaned up repo by itself in case of trouble.

I must admit that the generally speaking, requirement to have valid, cloned git repo on the target machine is a trouble-maker.

First of all, while provisioning build servers you must clone the repo, which forces you to use different authentication scheme to clone it from git than one that is used by the developers on a daily basis.

Than you must ensure that it is valid to avoid problems mentioned before.

Wouldn't it be just easier to always clone fresh repo before building starts and use SSH key forwarding to ensure that developer's credentials are used?

Strategy for secrets

I like what I see, thankyou... Just experimenting to deploy a phoenix application. Still very green to it all.

Within this phoenix app, there is a config/prod.secret.exs. I'm not commiting this file to the git store as it's full of 'secrets'. Instead and traditionally, before a build/release (cap/ruby) we'd link this and similar into the checkout. e.g. upload folders and so on.

Presently the mix release step fails as this file isn't present.

In Capistrano, we'd have the symlink dir and files step. I'm wondering if you had or are intending on a strategy for this scenario?

For the moment I've added a secrets file with ENV, so:
config :app_name, AppName.Endpoint,
secret_key_base: System.get_env("APP_NAME_SECRET_KEY_BASE")

Specify custom SSH port?

The server i'm trying to deploy to, it's ssh port is not 22 and I can't seem to find a way to specify the ssh port in the edeliver configs. So, is there a way to specify the ssh port currently?

No such file or directory

I'm getting the following errors when I try to do a build.

$ mix edeliver build release
deps/edeliver/bin/edeliver: line 26: /Users/col/Projects/test_app/deps/edeliver
/Users/col/Projects/test_app/deps/edeliver/libexec/defaults: No such file or directory
deps/edeliver/bin/edeliver: line 27: /Users/col/Projects/test_app/deps/edeliver
/Users/col/Projects/test_app/deps/edeliver/libexec/output: No such file or directory
deps/edeliver/bin/edeliver: line 28: /Users/col/Projects/test_app/deps/edeliver
/Users/col/Projects/test_app/deps/edeliver/libexec/deprecations: No such file or directory
deps/edeliver/bin/edeliver: line 29: /Users/col/Projects/test_app/deps/edeliver
/Users/col/Projects/test_app/deps/edeliver/libexec/core: No such file or directory
deps/edeliver/bin/edeliver: line 30: /Users/col/Projects/test_app/deps/edeliver
/Users/col/Projects/test_app/deps/edeliver/libexec/common: No such file or directory
deps/edeliver/bin/edeliver: line 31: /Users/col/Projects/test_app/deps/edeliver
/Users/col/Projects/test_app/deps/edeliver/libexec/generated: No such file or directory
deps/edeliver/bin/edeliver: line 32: /Users/col/Projects/test_app/deps/edeliver
/Users/col/Projects/test_app/deps/edeliver/libexec/erlang: No such file or directory
deps/edeliver/bin/edeliver: line 39: /Users/col/Projects/test_app/deps/edeliver
/Users/col/Projects/test_app/deps/edeliver/libexec/erlang-init: No such file or directory
deps/edeliver/bin/edeliver: line 41: __capture_runtime_configs: command not found
deps/edeliver/bin/edeliver: line 42: __find_all_strategies: command not found
deps/edeliver/bin/edeliver: line 45: __load_app_config: command not found
deps/edeliver/bin/edeliver: line 46: __default_app_config: command not found
deps/edeliver/bin/edeliver: line 47: __apply_runtime_configs: command not found
deps/edeliver/bin/edeliver: line 48: __load_strategy: command not found
deps/edeliver/bin/edeliver: line 49: __remote_hosts: command not found
deps/edeliver/bin/edeliver: line 50: __check_config: command not found
deps/edeliver/bin/edeliver: line 56: begin: command not found
deps/edeliver/bin/edeliver: line 57: run: command not found
deps/edeliver/bin/edeliver: line 58: finish: command not found

I've had a look into it and it seems to come down to this line in /bin/edeliver

BASE_PATH="$(cd "$DIR/.." && pwd)"

Somehow this is actually setting BASE_PATH to:

/Users/col/Projects/test_app/deps/edeliver /Users/col/Projects/test_app/deps/edeliver

I think this can be fixed by using this command instead.

BASE_PATH="$(cd "$DIR/.." && `pwd`)"

HTH

Ecto migrations on version change?

One of the hard problem that I am unable to solve is running ecto migration when updating the code. does edeliver have any nice solution for it? thanks

Custom environment names

Is it possible to have different environment names besides staging and production?

git-branch in auto version always says master

If you do:

mix edeliver build release --branch=whatever --auto-version=git-branch+git-revision

Your version number will always be XXX+master-abcdef where abcdef is the latest revision on branch whatever

I'd expect it to be XXX+whatever-abcdef

unfork repository?

Hi there, I think that edeliver has sufficiently changed from deliver such that it should no longer be considered a fork. Would you consider making edeliver not a fork? My primary concern is that forked repositories' code can't be searched, so it is annoying to search for things (I have to do it locally instead of on github)

Release build fails with "Failed to detect generated release version at..."

I had mix clean and git clean disabled, reenabling them and blowing away my directory has fixed this for now.

This is the relevant portion of the log:

[15:58:33][Step 1/1] ==> Generated .appup for booking 1.0.0-608-g47b6e9c -> 1.0.0-582-gf5a86b7
[15:58:38][Step 1/1] ===> relup successfully created!
[15:58:54][Step 1/1] ==> The release for booking-1.0.0-582-gf5a86b7 is ready!
[15:58:54][Step 1/1] ==> You can boot a console running your release with `$ rel/booking/bin/booking console`
[15:58:54][Step 1/1] Failed to detect generated release version at
[15:58:54][Step 1/1] [email protected]:/tmp/erlang/app/edge/builds/rel/app/releases/
[15:58:54][Step 1/1]  (B
[15:58:54][Step 1/1] Please set RELEASE_VERSION=x (B
[15:58:54][Step 1/1] Detected several releases: (B
[15:58:54][Step 1/1]  1.0.0-582-gf5a86b7 (B
[15:58:54][Step 1/1]  1.0.0-608-g47b6e9c (B

There are a couple interesting things here. One is that we're building a release, not an upgrade, yet it still generates an appup based on the latest version it finds.

I'm guessing that we reset our edge branch which is why it looks like we are building an earlier version.

If I supply RELEASE_VERSION set to the same version as --version it works when building releases, however if we build an upgrade in the same situation, even with RELEASE_VERSION set, it fails with the same type of error.

For now, we have reenabled mix clean and git clean (probably git clean is enough, but we're being safe)

Support incremental build?

At the moment, edeliver always cleans. I'm not experienced with elixir enough to know that this is necessary, but I do know that it greatly increases build time. I wonder if we should make it optional?

Errors swallowed by pre_erlang_clean_compile

When attempting to use the example pre_erlang_clean_compile hook on the Run additional build tasks page any errors that happen are essentially swallowed and the build appears to succeed. For example in the case that priv/static is ignored by git (default for phoenix applications) you'll receive the following:

The input path "priv/static" does not exist

The only way to know this however is to run this command manually inside of the BUILD_AT directory

Error when building

I'm getting this error when trying to get my first build going. However, if I use git config receive.denyCurrentBranch ignore on the remote server then everything works fine.

What exactly is this doing and is this the correct thing to do? (i tried it on a test repo).

Also I accidentally entered the above command on my local dev machine, should I undo this somehow?

baz|master ⇒ mix edeliver build release --verbose

BUILDING RELEASE OF BAZ APP ON BUILD HOST

-----> Authorizing hosts
-----> Ensuring hosts are ready to accept git pushes
-----> Pushing new commits with git to: [email protected]
remote: error: refusing to update checked out branch: refs/heads/master
remote: error: By default, updating the current branch in a non-bare repository
remote: error: is denied, because it will make the index and work tree inconsistent
remote: error: with what you pushed, and will require 'git reset --hard' to match
remote: error: the work tree to HEAD.
remote: error:
remote: error: You can set 'receive.denyCurrentBranch' configuration variable to
remote: error: 'ignore' or 'warn' in the remote repository to allow pushing into
remote: error: its current branch; however, this is not recommended unless you
remote: error: arranged to update its work tree to match what you pushed in some
remote: error: other way.
remote: error:
remote: error: To squelch this message and still keep the default behaviour, set
remote: error: 'receive.denyCurrentBranch' configuration variable to 'refuse'.
To [email protected]:/tmp/erlang/baz/builds
 ! [remote rejected] master -> master (branch is currently checked out)
error: failed to push some refs to '[email protected]:/tmp/erlang/baz/builds'

FAILED 1:
git push --tags -f [email protected] master

Host key verification failed while deploying from remote release store

I've encountered a "Host key verification failed" while deplyoing release from remote release store:

vagrant@trusty64:/vagrant$ mix edeliver deploy release to staging

DEPLOYING RELEASE OF AAAA APP TO STAGING HOSTS

-----> Authorizing hosts
-----> Uploading archive of release 0 from remote release store
Host key verification failed.

I had to go to my release store server and add manually validate authenticity of the staging hosts.

In my opinion it should be either:

automated (tell ssh/scp to trust by default - but it does not seem to be secure)
forwarded to the user, so instead not really useful "Host key verification failed." user can see the question

The authenticity of host 'yourserver.org (46.101.123.123)' can't be established.
ECDSA key fingerprint is 1a:be:d4:6e:fe:03:4f:3e:98:e8:41:f0:28:16:3f:f7.
Are you sure you want to continue connecting (yes/no)?

Clarification on "Run additional build tasks"

On the "Run additional build tasks" wiki page it states:

cd $DELIVER_TO # enter the build directory on the build host (required)

Since this happens during the build process I would think this should be:

cd $BUILD_AT # enter the build directory on the build host (required)

Can't find repo module

I have been trying to run database migrations on a staging and production server. Our app has to connect too and read from several databases but only writes to one.

I have added to following line the config file, with the repo that I need to run migrations for.
ECTO_REPOSITORY="MyApp.Repo"

but I keep getting the following error when I run mix edeliver migrate staging

==> No ecto repository module found.
Please specify the repository in the edeliver config as ECTO_REPOSITORY env.
<<"error">>

if [ "rebar" = "rebar" ]; then

When i deploy my elixir app it is failing on this step:

-----> Generating release
using rebar to generate release
bash: line 6: ./rebar: No such file or directory

FAILED  1:
ssh -o ConnectTimeout=3 [email protected] 
    set -e
    [ -f ~/.profile ] && source ~/.profile
    cd /tmp/erlang/location_updater/builds 
    if [ "rebar" = "rebar" ]; then
      echo "using rebar to generate release" 
      ./rebar   -f generate 
    elif [ "rebar" = "relx" ]; then
      echo "using relx to generate release" 
      ./relx release 
    elif [ "rebar" = "mix" ]; then
      echo "using mix to generate release" 
      MIX_ENV=prod mix release 
    fi

I'm usin mix so i have no rebar. Also when i run MIX_ENV=prod mix release from my project folder i have this info :

MIX_ENV=prod mix release 
Unchecked dependencies for environment prod:
* edeliver (git://github.com/boldpoker/edeliver.git)
  could not find an app file at _build/prod/lib/edeliver/ebin/edeliver.app. This may happen if the dependency was not yet compiled, or you specified the wrong application name in your deps, or the dependency indeed has no app file (then you can pass app: false as option)
** (Mix) Can't continue due to errors on dependencies

any idea why ?

Option to clear stashes or avoid stashing

With every build the git repository increases because stashes won't be cleared

Deploy from different repository

Is there a way to build from different git repository or even different VCS? Currently, I assume the build is performed from origin remote, isn't that right?

Response from edeliver commands truncated

When running an edeliver command (such as mix edeliver migrate staging up) the reponse seems to be truncated.

How can I get the full response?

Need to pull sources manually when deploying

Hi guys, I'm facing a problem. When deploying, each time before I run mix edeliver build release I need to go and git pull the sources on the build node. If I don't do this, I get this:

 → mix edeliver build release -V

BUILDING RELEASE OF ROCKSTAR APP ON BUILD HOST

-----> Authorizing hosts
-----> Ensuring hosts are ready to accept git pushes
-----> Pushing new commits with git to: [email protected]
remote: error: refusing to update checked out branch: refs/heads/master
remote: error: By default, updating the current branch in a non-bare repository
remote: error: is denied, because it will make the index and work tree inconsistent
remote: error: with what you pushed, and will require 'git reset --hard' to match
remote: error: the work tree to HEAD.
remote: error:
remote: error: You can set 'receive.denyCurrentBranch' configuration variable to
remote: error: 'ignore' or 'warn' in the remote repository to allow pushing into
remote: error: its current branch; however, this is not recommended unless you
remote: error: arranged to update its work tree to match what you pushed in some
remote: error: other way.
remote: error:
remote: error: To squelch this message and still keep the default behaviour, set
remote: error: 'receive.denyCurrentBranch' configuration variable to 'refuse'.
To [email protected]:/home/alg/builds/rockstar
 ! [remote rejected] master -> master (branch is currently checked out)
error: failed to push some refs to '[email protected]:/home/alg/builds/rockstar'

FAILED 1:
git push --tags -f [email protected] master

Is this by intention that I need to go pull the updates first, or is it something that I missed during the setup? I just scp'ed my sources initially to the build server.

UPDATE: I found #20. Is it the only way to handle this -- to config receive.denyCurrentBranch ignore?

I wonder if you have any better ideas? I could spend some time on it.

Fail if several build hosts are configured

BUILD_HOST variable should have only one host

Fix deploy of other branches than master

common:git_push() pushes only the master branch, but if different branch is specified in the REVISION environment, this branch should be pushed.

Bypass prompt for release type?

If I have a release and an upgrade of a specific version and I use deploy release edeliver will ask me if I want to upgrade or release. If I wanted to upgrade, I would have specified upgrade.

Is there a way to force it to use release? I can't use it in CI if I'm asked questions like this.

Configuration for umbrella applications

When the project has been organized as an umbrella, where should edeliver be setup, inside the umbrella or should each app in the app folder have its on .deliver folder and deployment configuration?

ls: cannot access /data/web/staging.my-app.com/build/rel/my-app/releases/*/: No such file or directory
Failed to detect generated release version at
[email protected]:/data/web/staging.my-app.com/build/rel/my-app/releases/

Please set RELEASE_VERSION=x
Detected several releases:

 FAILED
 2:
 ssh
basename: illegal option -- o
usage: basename string [suffix]
   basename [-a] [-s suffix] string [...]

 ConnectTimeout=3
 [email protected]
 ls
basename: illegal option -- A
usage: basename string [suffix]
       basename [-a] [-s suffix] string [...]

List installable releases also on remote release stores

in libexec/erlang/__detect_remote_release_version()

deploy release fails with: No release_or_upgrade_with_release(s) found in local release store.

I have a new phoenix app and I'm trying to setup edeliver for it

Now It successfully builds a release:

BUILDING RELEASE OF AUTHORIZATION_AND_ROUTING APP ON BUILD HOST

-----> Authorizing hosts
-----> Ensuring hosts are ready to accept git pushes
-----> Pushing new commits with git to: [email protected]
-----> Resetting remote hosts to d562d4a2ee57294f9072beed709af27c9eaacf62
-----> Cleaning generated files from last build
-----> Fetching / Updating dependencies
-----> Compiling sources
-----> Detecting exrm version
-----> Generating release
-----> Copying release 0.0.2 to local release store
-----> Copying authorization_and_routing.tar.gz to release store

RELEASE BUILD OF AUTHORIZATION_AND_ROUTING WAS SUCCESSFUL!

But on deploy stage it fails:

$ mix edeliver deploy release to production        

DEPLOYING RELEASE OF AUTHORIZATION_AND_ROUTING APP TO PRODUCTION HOSTS

-----> Authorizing hosts
ls: .deliver/releases/authorization_and_routing_*.release_or_upgrade_with_release.tar.gz: No such file or directory
No release_or_upgrade_with_release(s) found in local release store.
You can build one with the ./edeliver build release_or_upgrade_with_release task.

there are release files inside of .deliver/releases, but edeliver detects wrong release type

$ ls -l .deliver/releases
total 98352
-rw-rw-r--  1 amalaev  staff  25159494  2 фев 12:33 authorization_and_routing_0.0.1.release.tar.gz
-rw-rw-r--  1 amalaev  staff  25193615  2 фев 14:07 authorization_and_routing_0.0.2.release.tar.gz

Possible to have app in subdir of repo?

Our repo has the app & mix.exs in a sub directory rather than the top level directory. Is there a setting for this? It doesn't appear to work out of the box.

allow for private dependencies

it would be great if this library provided a way to handle private dependencies. that is, dependencies that can't be fetched from the build server but are present when the build command is kicked off.

Failure during "Stashing build directory"

Hi, I just added edeliver to my project and I'm getting this error:

BUILDING RELEASE OF PRIVATE APP ON BUILD HOST

-----> Authorizing hosts
-----> Ensuring hosts are ready to accept git pushes
-----> Pushing new commits with git to: root@private
Everything up-to-date
-----> Resetting remote hosts to 81f27e2556d3e62ae0cbf4bd2ab016311b7cc44c
HEAD is now at 81f27e2 update comeonin
-----> Stashing build directory on remote build host

FAILED  1:
ssh -o ConnectTimeout=3 root@private
    set -e
    [ -f ~/.profile ] && source ~/.profile
    cd /opt/private/builds 
    [[ $(git stash list | wc -l) -gt 0 ]] && _oldest_stash=$(git stash list | tail -1 | grep -o 'stash@{[0-9]\+}' | grep -o '[0-9]\+') || _oldest_stash=0
    [[ ${_oldest_stash} -gt 50 ]] && git stash drop stash@{$_oldest_stash} 
    git stash --all

Any idea what could be going wrong here? If I paste that command into the terminal it completes successfully.

Also, it appears that neither --verbose or -V have any effect.

Broken upgrades using edeliver and Ecto.Repo

Hi,

I've been struggling with what I think is a bug in edeliver for a while and I finally got to a small-ish reproduction case

TL;DR;

when MyApp.Repo is not supervised directly by the phoenix Application or when I have another OTP application that uses MyApp.Repo if edeliver is used in the project the release upgrade process fails.

I've created a reproduction application (you'll need to setup postgres to make it work) with two branches and instructions on how to reproduce the bug

https://github.com/ghedamat/foo (see README for detailed description)

the only difference between master and edeliver-test is the fact that the latter adds the edeliver package and application.

I'm kinda stuck as I'm quite new to all this (otp/releases etc) so any help would be much appreciated

thanks a lot!

Error on when trying to start build on production

When I run mix edeliver start production --verbose

it tells me

EDELIVER PROJECT WITH START COMMAND

-----> starting production servers

production node:

  user    : apps
  host    : example.com
  path    : /home/apps/test123-project-website

Pseudo-terminal will not be allocated because stdin is not a terminal.
Using /home/apps/test123-project-website/project/releases/0.0.1/project.sh

START DONE!

Any idea how I solve this Pseudo-terminal will not be allocated because stdin is not a terminal. issue?

Ecto 2.0 and missing repo/0

Ecto 2.0 removes the __repo__/0 function from Ecto.Repo, so edeliver fails to detect Repos when running e.g.:

mix edeliver migrate production

Detect rebar binary if it is in the $PATH on build hosts

At the moment only the local ./rebar binary is supported that must be present in the project root directory.

edeliver/exrm integration uses outdated paths scheme

At the moment edeliver fails to copy release to the release store if exrm is used.

-----> Copying release to remote release store
Copying release file failed
  source: /app/myapp/build/rel/myapp/myapp-0.0.1.tar.gz on [email protected]
  destination: /app/myapp/releases/myapp_0.0.1.release.tar.gz on [email protected]

It seems that it is due to change that happened in exrm at some point. In the recent versions the correct path for such release is

/app/myapp/build/rel/myapp/releases/0.0.1/myapp.tar.gz