cider-ci / cider-ci_deploy Goto Github PK

Shell 55.25% Ruby 43.10% Makefile 1.65%

cider-ci_deploy's Introduction

Cider-CI

Cider-CI is an application and service stack for highly parallelized and resilient integration testing. See also http://www.cider-ci.info/ and http://docs.cider-ci.info/introduction/.

This is the main repository of Cider-CI. The Cider-CI components are referenced from this repository as git submodules.

Status

![Integration-Tests](http://ci.zhdk.ch/cider-ci/ui/public/Cider-CI/master/Integration-Tests/summary.svg?respond_with_200&orientation=vertical =250x)

Contributors

The following people have contributed to Cider-CI:

License

Each component/submodule comes with its own License. The core componentes use the GNU Affero General Public License Version 3.

Contributing to Cider-CI

See contributing.

cider-ci_deploy's People

Contributors

Stargazers

Watchers

Forkers

zhdk eins78 10xsebastian schmijos

cider-ci_deploy's Issues

rabbitmq_vhost and rabbitmq_user failed

Ansible was not able to create vhost, user and permission in my case

ansible-playbook 1.8.4

Ansible was failing with:

TCP connection succeeded but Erlang distribution failed

No package matching 'ntpdate' is available

Im on ansible 1.9.2 now.

Following error occurred already last time I tried to run on 1.8:

/ TASK: debuntu-setup | apt pkg=ntpdate state=latest \
\ update_cache=yes cache_valid_time=3600             /
 ---------------------------------------------------- 
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||


<51.254.32.165> ESTABLISH CONNECTION FOR USER: root
<51.254.32.165> REMOTE_MODULE apt pkg=ntpdate state=latest update_cache=yes cache_valid_time=3600
<51.254.32.165> EXEC sshpass -d9 ssh -C -tt -vvv -o ForwardAgent=yes -o StrictHostKeyChecking=no -o GSSAPIAuthentication=no -o PubkeyAuthentication=no -o User=root -o ConnectTimeout=10 51.254.32.165 /bin/sh -c 'mkdir -p /tmp/ansible/ansible-tmp-1439711469.76-269405053573804 && echo /tmp/ansible/ansible-tmp-1439711469.76-269405053573804'
<51.254.32.165> PUT /var/folders/gl/x8g8g77s38q834hy7bjtsjzm0000gn/T/tmppWI5cv TO /tmp/ansible/ansible-tmp-1439711469.76-269405053573804/apt
<51.254.32.165> EXEC sshpass -d9 ssh -C -tt -vvv -o ForwardAgent=yes -o StrictHostKeyChecking=no -o GSSAPIAuthentication=no -o PubkeyAuthentication=no -o User=root -o ConnectTimeout=10 51.254.32.165 /bin/sh -c 'LANG=en_US.UTF-8 LC_CTYPE=en_US.UTF-8 /usr/bin/python /tmp/ansible/ansible-tmp-1439711469.76-269405053573804/apt; rm -rf /tmp/ansible/ansible-tmp-1439711469.76-269405053573804/ >/dev/null 2>&1'
failed: [demo-machine] => {"failed": true}
msg: No package matching 'ntpdate' is available

FATAL: all hosts have already failed -- aborting

openjdk-7-jre-headless : Depends: tzdata-java but it is not going to be installed

Also this happened last time:

< TASK: install_openjdk-7 | install jdk >
 --------------------------------------- 
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||


<51.254.32.165> ESTABLISH CONNECTION FOR USER: root
<51.254.32.165> REMOTE_MODULE apt pkg=openjdk-7-jre-headless,openjdk-7-jdk state=latest update_cache=yes cache_valid_time=3600
<51.254.32.165> EXEC sshpass -d12 ssh -C -tt -vvv -o ForwardAgent=yes -o StrictHostKeyChecking=no -o GSSAPIAuthentication=no -o PubkeyAuthentication=no -o User=root -o ConnectTimeout=10 51.254.32.165 /bin/sh -c 'mkdir -p /tmp/ansible/ansible-tmp-1439712875.0-267827570954654 && echo /tmp/ansible/ansible-tmp-1439712875.0-267827570954654'
<51.254.32.165> PUT /var/folders/gl/x8g8g77s38q834hy7bjtsjzm0000gn/T/tmpzPVA1v TO /tmp/ansible/ansible-tmp-1439712875.0-267827570954654/apt
<51.254.32.165> EXEC sshpass -d12 ssh -C -tt -vvv -o ForwardAgent=yes -o StrictHostKeyChecking=no -o GSSAPIAuthentication=no -o PubkeyAuthentication=no -o User=root -o ConnectTimeout=10 51.254.32.165 /bin/sh -c 'LANG=en_US.UTF-8 LC_CTYPE=en_US.UTF-8 /usr/bin/python /tmp/ansible/ansible-tmp-1439712875.0-267827570954654/apt; rm -rf /tmp/ansible/ansible-tmp-1439712875.0-267827570954654/ >/dev/null 2>&1'
failed: [demo-machine] => (item=openjdk-7-jre-headless,openjdk-7-jdk) => {"failed": true, "item": "openjdk-7-jre-headless,openjdk-7-jdk"}
stderr: E: Unable to correct problems, you have held broken packages.

stdout: Reading package lists...
Building dependency tree...
Reading state information...
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 openjdk-7-jre-headless : Depends: tzdata-java but it is not going to be installed

msg: '/usr/bin/apt-get -y -o "Dpkg::Options::=--force-confdef" -o "Dpkg::Options::=--force-confold"   install 'openjdk-7-jre-headless' 'openjdk-7-jdk'' failed: E: Unable to correct problems, you have held broken packages.

Last time I fixed that by resolving the dependency issues by hand.
More details in a sec.

server-monitoring fails

In the end of the installation the complete service monitoring is failing:

As those errors are ignored, I would like to ask if thats really normal behaviour?

< TASK: server-monitoring | Service monitoring is present >
 --------------------------------------------------------- 
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||


failed: [demo-machine] => (item=cider-ci_api) => {"failed": true, "item": "cider-ci_api", "name": "cider-ci_api", "state": "present"}
msg: cider-ci_api process not configured with monit
...ignoring
failed: [demo-machine] => (item=cider-ci_builder) => {"failed": true, "item": "cider-ci_builder", "name": "cider-ci_builder", "state": "present"}
msg: cider-ci_builder process not configured with monit
...ignoring
failed: [demo-machine] => (item=cider-ci_dispatcher) => {"failed": true, "item": "cider-ci_dispatcher", "name": "cider-ci_dispatcher", "state": "present"}
msg: cider-ci_dispatcher process not configured with monit
...ignoring
failed: [demo-machine] => (item=cider-ci_notifier) => {"failed": true, "item": "cider-ci_notifier", "name": "cider-ci_notifier", "state": "present"}
msg: cider-ci_notifier process not configured with monit
...ignoring
failed: [demo-machine] => (item=cider-ci_repository) => {"failed": true, "item": "cider-ci_repository", "name": "cider-ci_repository", "state": "present"}
msg: cider-ci_repository process not configured with monit
...ignoring
failed: [demo-machine] => (item=cider-ci_storage) => {"failed": true, "item": "cider-ci_storage", "name": "cider-ci_storage", "state": "present"}
msg: cider-ci_storage process not configured with monit
...ignoring
failed: [demo-machine] => (item=cider-ci_user-interface) => {"failed": true, "item": "cider-ci_user-interface", "name": "cider-ci_user-interface", "state": "present"}
msg: cider-ci_user-interface process not configured with monit
...ignoring
 ___________________________________________ 
< TASK: server-monitoring | Monitor service >
 ------------------------------------------- 
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||


failed: [demo-machine] => (item=cider-ci_api) => {"failed": true, "item": "cider-ci_api", "name": "cider-ci_api", "state": "monitored"}
msg: cider-ci_api process not presently configured with monit
...ignoring
failed: [demo-machine] => (item=cider-ci_builder) => {"failed": true, "item": "cider-ci_builder", "name": "cider-ci_builder", "state": "monitored"}
msg: cider-ci_builder process not presently configured with monit
...ignoring
failed: [demo-machine] => (item=cider-ci_dispatcher) => {"failed": true, "item": "cider-ci_dispatcher", "name": "cider-ci_dispatcher", "state": "monitored"}
msg: cider-ci_dispatcher process not presently configured with monit
...ignoring
failed: [demo-machine] => (item=cider-ci_notifier) => {"failed": true, "item": "cider-ci_notifier", "name": "cider-ci_notifier", "state": "monitored"}
msg: cider-ci_notifier process not presently configured with monit
...ignoring
failed: [demo-machine] => (item=cider-ci_repository) => {"failed": true, "item": "cider-ci_repository", "name": "cider-ci_repository", "state": "monitored"}
msg: cider-ci_repository process not presently configured with monit
...ignoring
failed: [demo-machine] => (item=cider-ci_storage) => {"failed": true, "item": "cider-ci_storage", "name": "cider-ci_storage", "state": "monitored"}
msg: cider-ci_storage process not presently configured with monit
...ignoring
failed: [demo-machine] => (item=cider-ci_user-interface) => {"failed": true, "item": "cider-ci_user-interface", "name": "cider-ci_user-interface", "state": "monitored"}
msg: cider-ci_user-interface process not presently configured with monit
...ignoring
 ________________________________________ 
< TASK: server-monitoring | Reload monit >
 ---------------------------------------- 
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

TLS: executors should install&trust self-signed cert from master

Should be easy to do and there is no reason to use --insecure when talking between two own machines.

Display all the public ssh keys in the UI

Would be convenient to display all the public ssh keys in the UI (like the executor show pages).

Would speed up adding them to github.

python/pip env broken

I looked into the mkdocs trait and why it was failing.
Turns out:

the python ecosystem seems even more broken than ruby's
state of python on the executors is especially bad.

I "fixed" one of the executors manually:

apt-get purge python-pip
easy_install pip
pip install --upgrade pip
# now mkdocs could be installed:
pip install --upgrade mkdocs

I especially had fun with this detail, it took several tries to get pip to install the newest version:

root@Cislave3:/tmp# pip install --upgrade mkdocs
…
Installing collected packages: mkdocs
  Found existing installation: mkdocs 0.14.0
    Uninstalling mkdocs-0.14.0:
      Successfully uninstalled mkdocs-0.14.0
Successfully installed mkdocs-0.11.1

root@Cislave3:/tmp# pip install --upgrade mkdocs
…
Installing collected packages: mkdocs
  Found existing installation: mkdocs 0.11.1
    Uninstalling mkdocs-0.11.1:
      Successfully uninstalled mkdocs-0.11.1
Successfully installed mkdocs-0.15.1

root@Cislave3:/tmp# pip install --upgrade mkdocs
…
Installing collected packages: mkdocs
  Found existing installation: mkdocs 0.15.1
    Uninstalling mkdocs-0.15.1:
      Successfully uninstalled mkdocs-0.15.1
Successfully installed mkdocs-0.15.3

root@Cislave3:/tmp# pip install --upgrade mkdocs
…
root@Cislave3:/tmp#

nodejs can not be authenticated

Running installation with 5309fd39f44b55f25f1c4bb0a702aa22dca6aad4 on fresh machines leads to:

failed: [xxx] => {"failed": true}
stderr: E: There are problems and -y was used without --force-yes

stdout: Reading package lists...
Building dependency tree...
Reading state information...
The following extra packages will be installed:
  rlwrap
The following NEW packages will be installed:
  nodejs rlwrap
0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Need to get 8,674 kB of archives.
After this operation, 40.2 MB of additional disk space will be used.
WARNING: The following packages cannot be authenticated!
  nodejs