Code Monkey home page Code Monkey logo

lineup's Introduction

Zeeguu-Ecosystem

To clone this repository run the following:

git clone --recursive https://github.com/zeeguu-ecosystem/Lineup

Zeeguu container images

Prebuilt images for testing are already available on dockerhub.

Dependencies:

Docker needs to be installed. Install it with:

sudo apt-get install docker.io -y

Configuration

Each of the subprojects has it's own configuration file (core, api, web, teacherdash, etc.). A set of reasonable defaults is already present in the corresponding ./config/ folder for each of these projects. If you want to run zeeguu with these defaults, simply copy each of the folders to a corresponding file w/o the .default extension. if later you want to modify the config, you can modify these .cfg files which are now not under version control (and are actually actively ignored by the .gitignore file, such that you don't upload secrets by mistake).

Building automatically

Set the environment variables (that can be found at the top of config_vars.sh) with the API keys for the third party services.

Run the script setup_all.sh and wait for it to complete.

Building manually

The rest of the commands must be run from the Lineup folder:

cd Lineup

To build the zeeguu-mysql container image:

docker build -t zeeguu-mysql -f docker-files/zeeguu-mysql/Dockerfile .

To build the zeeguu-api-core container image:

docker build -t zeeguu-api-core -f docker-files/zeeguu-api-core/Dockerfile .

Note: To deploy the Flask-MonitoringDashboard with the API make sure that the corresponding lines are uncommented in docker-files/zeeguu-api-core/Dockerfile . Then make sure to pass the API_VERSION as a --build-arg. Something like:

docker build --build-arg API_VERSION=`cat .git/modules/Zeeguu-API/HEAD` -t zeeguu-api-core -f docker-files/zeeguu-api-core/Dockerfile .

To build the zeeguu-web container image (Make sure to replace 1.1.1.1:9001 with the url:port where your API can be reached from other clients.): Before building the zeeguu-web container image you must copy the apache-zeeguu.conf.default to docker-files/zeeguu-api-core/apache-zeeguu.conf and modify it accordingly.

docker build -t zeeguu-web --build-arg ZEEGUU_API__EXTERNAL="http://1.1.1.1:9001"  -f docker-files/zeeguu-web/Dockerfile .

To run the containers:

docker run --net=host -d --name=<container_name> <image_name>

Example:

docker run --net=host -d --name=zeeguu-mysql zeeguu-mysql
# Wait for a minute to allow the zeeguu mysql to complete initialization
# before starting Zeeguu API. You can check the status by running:
# docker logs zeeguu-mysql --follow
# If you see "mysqld: ready for connections" then you are ready to continue.
docker run --net=host -d --name=zeeguu-api-core zeeguu-api-core
docker run --net=host -d --name=zeeguu-web zeeguu-web

Extra environment variables for containers

For zeeguu-api-core you need to define the following API keys:

  • GOOGLE_TRANSLATE_API_KEY
  • MICROSOFT_TRANSLATE_API_KEY
  • WORDNIK_API_KEY

To pass the variable, add the -e flag to docker run command. Example:

docker run --net=host -d -e MICROSOFT_TRANSLATE_API_KEY='key' -e GOOGLE_TRANSLATE_API_KEY='key' -e WORDNIK_API_KEY='key'  --name=zeeguu-api-core zeeguu-api-core

Storing MySQL data under a given directory on the system

To store the MySQL data under a given path, follow the steps:

  1. Create the data directory on the host system (e.g. /opt/mysql_datadir)

  2. Add the -v option to the MySQL container:

docker run --net=host -v /opt/mysql_datadir:/var/lib/mysql -d --name=zeeguu-mysql zeeguu-mysql

Adding articles for the reader

To add a new RSS feed for the reader, you have to run the following and follow the prompts:

docker exec -i zeeguu-api-core python /opt/Zeeguu-API/tools/add_rssfeed.py

Example:

$ docker exec -i zeeguu-api-core python /opt/Zeeguu-API/tools/add_rssfeed.py
ZEEGUU: Loaded Zeeguu-Core config from /opt/Zeeguu-Core/default_core.cfg
ZEEGUU: Linked model with: mysql://zeeguu_test:[email protected]/zeeguu_test
Feed url:  http://rss.cnn.com/rss/edition_world.rss
Found image url at: http://i2.cdn.turner.com/cnn/2015/images/09/24/cnn.digital.png
Feed seems healthy: 28 items found.
Feed name (Enter for: CNN.com - RSS Channel - World):  CNN World RSS Feed
= CNN World RSS Feed
Icon name to be found in resources folder (e.g. 20min.png):  cnn.png
= cnn.png
Description (Enter for: CNN.com delivers up-to-the-minute news and information on the latest top stories, weather, entertainment, politics and more.): CNN World RSS Feed for news
= CNN World RSS Feed for news
Language code (e.g. en): en
= en
Done:
CNN World RSS Feed
CNN World RSS Feed for news
1
http://rss.cnn.com/rss/edition_world.rss
https://zeeguu.unibe.ch/api/resources/cnn.png

To fetch the articles from the RSS feed, run the following:

docker exec -i zeeguu-api-core python /opt/Zeeguu-Core/tools/feed_retrieval.py

RSS feed examples:

lineup's People

Contributors

mircealungu avatar alinbalutoiu avatar mfurak avatar jonsnielsen avatar mbrodt avatar

Watchers

avatar James Cloos avatar Lars Holdijk avatar

Forkers

alinbalutoiu mbrodt jonsnielsen mfurak

lineup's Issues

docker build not having permissions

Following the Readme I execute:

docker build -t zeeguu-mysql -f docker-files/zeeguu-mysql/Dockerfile .

However, an error is reported:

Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post http://%2Fvar%2Frun%2Fdocker.sock/v1.38/build?buildargs=%7B%7D&cachefrom=%5B%5D&cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=docker-files%2Fzeeguu-mysql%2FDockerfile&labels=%7B%7D&memory=0&memswap=0&networkmode=default&rm=1&session=qkcy9e0hondxpnwxpl0i9adz9&shmsize=0&t=zeeguu-mysql&target=&ulimits=null&version=1: dial unix /var/run/docker.sock: connect: permission denied

external config file

I remember that we talked about this @alinbalutoiu but i still think that the API KEYS would make more sense if they were in a config file.

docker run --net=host -d -e MICROSOFT_TRANSLATE_API_KEY='key' -e GOOGLE_TRANSLATE_API_KEY='key' -e WORDNIK_API_KEY='key' --name=zeeguu-api-core zeeguu-api-core

In general, I still wonder if it would make sense to have a Lineup/config in which to map all the config files, such that we allow the configuration of the entire app in one place.

errors while running the mysql image?

this is what the logs show:

2018-12-13T17:43:22.463230Z 0 [Note] mysqld: ready for connections.
Version: '5.7.24'  socket: '/var/run/mysqld/mysqld.sock'  port: 0  MySQL Community Server (GPL)
Warning: Unable to load '/usr/share/zoneinfo/iso3166.tab' as time zone. Skipping it.
Warning: Unable to load '/usr/share/zoneinfo/leap-seconds.list' as time zone. Skipping it.
2018-12-13T17:43:24.729724Z 3 [ERROR] InnoDB: posix_fallocate(): Failed to preallocate data for file ./ibdata1, desired size 67108864 bytes. Operating system error number 28. Check that the disk is not full or a disk quota exceeded. Make sure the file system supports this function. Some operating system error numbers are described at http://dev.mysql.com/doc/refman/5.7/en/operating-system-error-codes.html
2018-12-13T17:43:24.731252Z 3 [Warning] InnoDB: 1048576 bytes should have been written. Only 831488 bytes written. Retrying for the remaining bytes.
2018-12-13T17:43:24.731587Z 3 [Warning] InnoDB: Retry attempts for writing partial data failed.
2018-12-13T17:43:24.731631Z 3 [ERROR] InnoDB: Write to file ./ibdata1failed at offset 14680064, 1048576 bytes should have been written, only 831488 were written. Operating system error number 28. Check that your OS and file system support files of this size. Check also that the disk is not full or a disk quota exceeded.
2018-12-13T17:43:24.731741Z 3 [ERROR] InnoDB: Error number 28 means 'No space left on device'
2018-12-13T17:43:24.731846Z 3 [Note] InnoDB: Some operating system error numbers are described at http://dev.mysql.com/doc/refman/5.7/en/operating-system-error-codes.html
2018-12-13T17:43:24.731884Z 3 [Warning] InnoDB: Error while writing 67108864 zeroes to ./ibdata1 starting at offset 12582912
2018-12-13T17:43:24.736980Z 3 [ERROR] mysqld: The table 'time_zone_transition' is full
ERROR 3019 (HY000) at line 120849: Undo Log error: No more space left over in system tablespace for allocating UNDO log pages. Please add new data file to the tablespace or check if filesystem is full or enable auto-extension for the tablespace

can't access the web server at the end of setup_all.sh

@alinbalutoiu - this is a question for you :)

at the end of setup_all.sh I can do:

curl 127.0.0.1:8080/available_languages

and it works. this means that the api_core container is correctly mapped to the 8080 port on the host.

however, I can not do:

curl 127.0.0.1

even if apache is supposed to be listening
on port 80 within this container and the container
is started with --net=host.

moreover, if I start the flask web server for testing
with python zeeguu_web.wsgi run which listens
to port 9000 the web site works.

when i look inside the apache logs I see:

  • nothing in access.log
  • no errors in error.log

the following lines in other_vhosts_access.log

api.zeeguu.local:80 127.0.0.1 - - [12/Nov/2019:10:29:48 +0000] "GET / HTTP/1.1" 404 405 "-" "curl/7.52.1"
api.zeeguu.local:80 127.0.0.1 - - [12/Nov/2019:10:29:58 +0000] "GET / HTTP/1.1" 404 405 "-" "curl/7.52.1"
api.zeeguu.local:80 127.0.0.1 - - [12/Nov/2019:10:30:02 +0000] "GET / HTTP/1.1" 404 405 "-" "curl/7.52.1"
api.zeeguu.local:80 ::1 - - [12/Nov/2019:10:31:36 +0000] "GET / HTTP/1.1" 404 405 "-" "curl/7.52.1"
api.zeeguu.local:80 ::1 - - [12/Nov/2019:10:45:23 +0000] "GET /create_account HTTP/1.1" 404 405 "-" "curl/7.52.1"

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.