kbaseattic / awe_service Goto Github PK

The awe_service repository provides KBase release engineering tools for the deployment of the AWE client and server. The AWE repository is included as a submodule in the awe_service repository and is maintained at http://github.com/MG-RAST/AWE.

License: MIT License

Makefile 39.56% Shell 14.67% Python 45.77%

awe_service's Introduction

awe_service

Enables deployment of AWE server and clients within the kbase release environment.

The following are instructions on how to deploy the AWE service in KBase from launching a fresh KBase instance to starting the service...

Create a security group for your AWE server (this isn't required with the test ports in the Makefile but makes it convenient if you want to change those ports later)
Using this security group and your key, launch a KBase instance
Get the code:

sudo -s
cd /kb
git clone [email protected]:dev_container.git
cd dev_container/modules
git clone [email protected]:awe_service.git
cd awe_service

Edit the Makefile if you would like to configure the IP and port numbers for your AWE server.
Create a mongo data directory on the mounted drive and symbolic link to point to it (note that mongodb is only required if you are deploying the AWE server, not the client):

cd /mnt
mkdir db
cd /
mkdir data
cd data/
ln -s /mnt/db .

Start mongod (preferably in a screen session). Mongod needs a few minutes to start. Once you can run mongo from the command line and connect to mongo db, then proceed with deploying AWE:

cd /kb/dev_container
./bootstrap /kb/runtime
source user-env.sh
mongod

Deploy AWE:

cd /kb/dev_container
./bootstrap /kb/runtime
source user-env.sh
make
make deploy

Start/Stop AWE Server:

/kb/deployment/services/awe_service/start_service
/kb/deployment/services/awe_service/stop_service

After deployment has completed, if you've associated an IP with your instance you should be able to confirm that AWE is running by going to either url below (IP address and ports are defined in Makefile prior to deployment):

site ->  http://[AWE Server IP]:7079/
api  ->  http://[AWE Server IP]:7080/

Start/Stop AWE Client. Before starting awe-client, make sure the fields in awe_client.cfg.tt have been configured with proper values by modifying Makefile.

/kb/deployment/services/awe_service/start_aweclient
/kb/deployment/services/awe_service/stop_aweclient

awe_service's People

Contributors

Watchers

Forkers

wtangiit leereilly wgerlach teharrison brettin pranjan77 olsonanl scanon kkellerlbl

awe_service's Issues

AWE submodule is not updated on a make

I just updated my awe_service repo:

crusherofheads@icrushdeheads:/kb/dev_container/modules/awe_service$ git log
commit 25a2a17039ca4a7fe4856bb91c161e733b1b9e35
Author: Jared Bischof <[email protected]>
Date:   Fri Jul 18 15:46:47 2014 -0500

    Update to latest AWE code.

crusherofheads@icrushdeheads:/kb/dev_container/modules/awe_service$ git pull
remote: Counting objects: 35, done.
remote: Compressing objects: 100% (24/24), done.
remote: Total 35 (delta 12), reused 32 (delta 11)
Unpacking objects: 100% (35/35), done.
From http://github.com/kbase/awe_service
   25a2a17..244e569  master     -> origin/master
Fetching submodule AWE
remote: Counting objects: 1109, done.
remote: Compressing objects: 100% (388/388), done.
remote: Total 1109 (delta 755), reused 973 (delta 666)
Receiving objects: 100% (1109/1109), 232.96 KiB, done.
Resolving deltas: 100% (755/755), done.
From https://github.com/MG-RAST/AWE
   d014dd9..74232cf  master     -> origin/master
From https://github.com/MG-RAST/AWE
 * [new tag]         v0.9.10    -> v0.9.10
 * [new tag]         v0.9.11    -> v0.9.11
 * [new tag]         v0.9.4     -> v0.9.4
 * [new tag]         v0.9.5     -> v0.9.5
 * [new tag]         v0.9.6     -> v0.9.6
 * [new tag]         v0.9.8     -> v0.9.8
Updating 25a2a17..244e569
Fast-forward
 AWE                       |    2 +-
 awe_client.cfg.tt         |    3 +++
 service/stop_aweclient.tt |    2 +-
 service/stop_service.tt   |    2 +-
 4 files changed, 6 insertions(+), 3 deletions(-)

crusherofheads@icrushdeheads:/kb/dev_container/modules/awe_service$ make
rm -rf /tmp/go_build.tmp
mkdir -p /tmp/go_build.tmp/src/github.com/MG-RAST
cp -r AWE /tmp/go_build.tmp/src/github.com/MG-RAST/
export GOPATH=/tmp/go_build.tmp; go get -v github.com/MG-RAST/AWE/...
code.google.com/p/go-uuid (download)
github.com/MG-RAST/golib (download)
Fetching https://labix.org/v2/mgo?go-get=1
*snip*
`/tmp/go_build.tmp/bin/awe-client' -> `../../bin/awe-client'

crusherofheads@icrushdeheads:/kb/dev_container/modules/awe_service$ cd AWE
crusherofheads@icrushdeheads:/kb/dev_container/modules/awe_service/AWE$ git log
commit d014dd940475cc741050eb351f367771f38e5bf2
Merge: fec1bef 6a3743a
Author: jaredbischof <[email protected]>
Date:   Fri Jul 18 15:18:42 2014 -0500

    Merge pull request #271 from jaredbischof/master

    Update to version number.

but the the AWE submodule version hash from github is 5cfd7c2.

If I clone from scratch, the submodule is the correct version:

crusherofheads@icrushdeheads:/kb/dev_container/modules$ rm -rf awe_service/
crusherofheads@icrushdeheads:/kb/dev_container/modules$ git clone https://github.com/kbase/awe_service
Cloning into 'awe_service'...
remote: Counting objects: 389, done.
remote: Total 389 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (389/389), 55.82 KiB, done.
Resolving deltas: 100% (194/194), done.
crusherofheads@icrushdeheads:/kb/dev_container/modules$ cd awe_service/
crusherofheads@icrushdeheads:/kb/dev_container/modules/awe_service$ make
git submodule init
Submodule 'AWE' (https://github.com/MG-RAST/AWE.git) registered for path 'AWE'
git submodule update
Cloning into 'AWE'...
remote: Counting objects: 5904, done.
remote: Compressing objects: 100% (19/19), done.
remote: Total 5904 (delta 7), reused 0 (delta 0)
Receiving objects: 100% (5904/5904), 2.39 MiB | 1.37 MiB/s, done.
Resolving deltas: 100% (3586/3586), done.
Submodule path 'AWE': checked out '5cfd7c29cd80b62a3382307175333f0ffb6849c7'
rm -rf /tmp/go_build.tmp
mkdir -p /tmp/go_build.tmp/src/github.com/MG-RAST
cp -r AWE /tmp/go_build.tmp/src/github.com/MG-RAST/
export GOPATH=/tmp/go_build.tmp; go get -v github.com/MG-RAST/AWE/...
code.google.com/p/go-uuid (download)
github.com/MG-RAST/golib (download)
*snip*

`/tmp/go_build.tmp/bin/awe-client' -> `../../bin/awe-client'
crusherofheads@icrushdeheads:/kb/dev_container/modules/awe_service$ cd AWE/
crusherofheads@icrushdeheads:/kb/dev_container/modules/awe_service/AWE$ git log
commit 5cfd7c29cd80b62a3382307175333f0ffb6849c7
Merge: e66cd55 8a00a9d
Author: Travis Harrison <[email protected]>
Date:   Mon Sep 8 11:36:58 2014 -0500

    Merge pull request #294 from teharrison/master

    fix pointer derefence

Missing items in config files?

After deploying the AWE service, I had to add the [Auth] section and the database, user, and password keys to the [Mongodb] section

Finally, totalworkers and app_path are missing from the client config - I'm not sure yet if that's important.

Daemonize logs

Not sure if this is fixable, but as far as I can tell daemonize discards all the output from the AWE server (as opposed to upstart if configured correctly). This makes it difficult to debug if the server doesn't start correctly. Is there any way to preserve the output somewhere? Reading through the daemonize docs I don't see any way to do so.

EDIT - could just redirect the output to a file, I suppose.

Default non-production build causes status monitor error

The non-production TPAGE args leave the host for the site as blank, and so the generated config.js looks like this:

var RetinaConfig = {
    "awe_ip": ":7080",
    "workflow_ip": ":7080/awf",
    "authResources": { "default": "KBase",
               "KBase": { "icon": "KBase_favicon.ico",
                  "prefix": "kbgo4711" } },
    "mgrast_api": "http://api.metagenomics.anl.gov",
    "authentication": true
}

But the Status Monitor website seems to automatically include a host if it's missing - but keeps the port, so in the JS console I get:

"NetworkError: 404 Not Found - http://localhost:7079/:7080/queue"
queue
"NetworkError: 404 Not Found - http://localhost:7079/:7080/job?query&limit=20&offset=0"
job?qu...ffset=0
"NetworkError: 404 Not Found - http://localhost:7079/:7080/job?active"
job?active
"NetworkError: 404 Not Found - http://localhost:7079/:7080/job?suspend"
job?suspend
"NetworkError: 404 Not Found - http://localhost:7079/:7080/job?query&state=completed&limit=20&offset=0"
job?qu...ffset=0
"NetworkError: 404 Not Found - http://localhost:7079/:7080/work?query&state=queued"
work?q...=queued
"NetworkError: 404 Not Found - http://localhost:7079/:7080/work?query&state=checkout"
work?q...heckout
"NetworkError: 404 Not Found - http://localhost:7079/:7080/client"
client

Changing the config to

var RetinaConfig = {
    "awe_ip": "http://localhost:7080",
    "workflow_ip": "http://localhost:7080/awf",
    "authResources": { "default": "KBase",
               "KBase": { "icon": "KBase_favicon.ico",
                  "prefix": "kbgo4711" } },
    "mgrast_api": "http://api.metagenomics.anl.gov",
    "authentication": true
}

fixes the problem. The TPAGE args should probably be set such that the site host is http://localhost.

Configurable AWE_DIR in Makefile

This is very minor, but it'd be a bit easier to deploy AWE if the awe directory was settable at the top of the makefile, or at least the AWE_DIR assignments were optional (e.g. ?= rather than =). My laptop doesn't have a /mnt directory so the deploy failed. Everything else can be edited in the config file after a deployment, but the directory is required to be accessible for the deploy to work.

kbaseattic / awe_service Goto Github PK

awe_service's Introduction

awe_service

awe_service's People

Contributors

Watchers

Forkers

awe_service's Issues

AWE submodule is not updated on a make

Missing items in config files?

Daemonize logs

Default non-production build causes status monitor error

Configurable AWE_DIR in Makefile

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent