Code Monkey home page Code Monkey logo

swim-ti-yp-prototype's Introduction

SWIM-TI Yellow Profile Prototype

Overview

This document describes the procedure to get up and running a SWIM-TI demo platform which enables external client systems to get real time air traffic information. The real time air traffic is retrieved from OpenSky Network based on arrivals and departures of pre-configured airports. Based on the SWIM-TI specifications, this demo utilises the AMQP v1.0 protocol in order to enable a producer to publish real time air traffic data per topic (arrivals or departures of specific airports) and a consumer to subscribe to one or more such topics and receive the relevant information.

More specifically, the main services involved are:

  • Subscription Manager: is the core of the whole system wrapping up the broker where the data flows through by managing its topics to be published and its queues to consume from. It is supposed to be broker agnostic and for this demo RabbitMQ is used as a broker. Moreover it stores topics' and subscriptions' metadata in a PostgreSQL database and it exposes a REST API based on the OpenAPI specifications.
  • SWIM ADSB is the producer implementation which publishes real time air traffic data of arrivals and departures to specific airports.
  • SWIM Explorer is the consumer implementation which subscribes to arrival or departure topics and displays the air traffic in real time on a map.

Both SWIM ADSB and SWIM Explorer make use of the pubsub-facades library which allows them to speak to the Subscription Manager as well as the broker.

The platform is designed with a microservices approach and is run with docker.

Below you can see the architecture design diagram along with the relationships of the COTS as well as the internally developed services and libraries:

alt text

Installation

Step by step

The steps bellow will allow you to build and run the SWIM-TI demo from scratch. Make sure you follow them one by one in the given order.

Prerequisites

Before starting, make sure the following software is installed and working on your machine:

Linux/Mac users
  • git which will be used to download the necessary repositories.
  • docker which will be used to host and run the whole demo platform
  • docker-compose which will be used to orchestrate the deployment locally
  • python which will be used to run python custom scripts
Windows users
  • Docker Toolbox on Windows (which installs all the required tools and runs the Docker Engine via VirtualBox)
  • python which will be used to run python custom scripts
Post-installation
  • Before using git commands you need to disable conversion of checked out files to Windows format:

    git config --global core.autocrlf input
    
  • Since Docker runs via VirtualBox, the various app/server sites involved in this demo will not be able to be accessed from your host machine unless you apply some port forwarding rules. You may find instructions about how to do it here. The rules that we need are:

    Name Host port Guest Port
    broker 15671 15671
    https 443 443
    explorer 5000 5000

Download repositories

NOTE: The next steps are done on a Unix-like command line interface (CLI) for all users. Linux and Mac users can use any terminal application, however the Windows users will have to use the Docker CLI client that comes with Docker Toolbox on Windows. This can be accessed by starting Docker Quickstart Terminal.

First we need to clone this repository:

git clone https://github.com/eurocontrol-swim/SWIM-TI-YP-prototype.git
cd SWIM-TI-YP-prototype

Configuration

Then we have to provide the necessary configuration of the services. This involves:

  • setting up the required users
  • application specific configuration
SWIM users

Several users are required across the SWIM platform such as db users, broker users etc. You can use the following command in order to generate them automatically:

. ./swim.sh user_config

the leading . before ./swim.sh user_config is required in order the provided usernames and passwords to be exported as environment variables on the host machine

The output will look like:

SWIM user configuration...
==========================

Database user
[OK]
Subscription Manager admin user
[OK]

SWIM ADSB user for Subscription Manager
[OK]

SWIM Explorer user for Subscription Manager
[OK]

Broker admin user
[OK]

Broker Management user
[OK]

SWIM Explorer user for the broker
[OK]

However, you can always choose your own usernames/passwords by using the prompt option of the command:

. ./swim.sh user_config --prompt

Passwords can take any character and need to be 10 or more characters long. Each provided password will be checked for robustness and if it is deemed that it is not robust enough you will be re-prompted to choose a different one.

The interaction will look like:

SWIM user configuration...
==========================

Database user
 username: swim 
 password: 
[OK]

Subscription Manager admin user
 username: sm-admin 
 password: 
[OK]

SWIM ADSB user for Subscription Manager
 username: swim-adsb 
 password: 
The password is not strong enough. Please try again:
 password: 
[OK]

SWIM Explorer user for Subscription Manager
 username: swim-explorer 
 password: 
[OK]

Broker admin user
 username: swim-broker-admin 
 password: 
[OK]

Broker Management user
 username: swim-broker-mgmt 
 password: 
[OK]

SWIM Explorer user for the broker
 username: swim-explorer-broker 
 password: 
[OK]

You can verify that the provided usernames and passwords were exported as environment variables by issuing the following command:

env
Application config files

Under the services folder you can find one folder per app containing a config.yml file. These are already configured but you may want to update the airports involved in the SWIM ADSB application. The config entry looks as following and you can add a new airport by choosing a name and assigning to it the corresponding ICAO code (more info here).

The INTERVAL_IN_SEC variable defines the refresh rate of the air traffic data in seconds. The chosen value depends on the number of airports to retrieve information from as well as your internet speed:

ADSB:
  CITIES:
    Brussels: 'EBBR'
    Amsterdam: 'EHAM'
    Paris: 'LFPG'
    Berlin: 'EDDB'
    Athens: 'LGAV'
    Heathrow: 'EGLL'
    <my city>: 'icao code' 
  INTERVAL_IN_SEC: 5

Deployment

For the deployment process you can use the provided shell script swim.sh. Before using it though make it executable with the following command:

chmod +x swim.sh

Moreover you can see the available options of this script with:

./swim.sh help
Usage: swim.sh [COMMAND] [OPTIONS]

Commands:
    user_config             Generates username/password for all the SWIM related users
    user_config --prompt    Prompts for username/password for all the SWIM related users
    build                   Clones/updates the necessary git repositories and builds the involved docker images
    provision               Provisions the Subscription Manager with initial data (users)
    start                   Starts up all the SWIM services
    stop                    Stops all the services
    stop --clean            Stops all the services and cleans up the containers
    stop --purge            Stops all the services and cleans up the containers and volumes
    status                  Displays the status of the running containers

In order to get the demo platform up and running we first need to download the necessary repositories and build the involved docker images with the following command:

./swim.sh build

The first time you run this command it will take some time because of the download/build of docker images. this has to be run only the first time we start SWIM

In case you run this command after having deployed successfully SWIM all the old data will be removed.

After the necessary images are built we are ready to get the services up and running. Before that we'll need to provision the Subscription Manager and the broker with some initial data about the involved users and this can be done with:

./swim.sh provision

this has to be run only the first time we start SWIM

and then we can start the services:

./swim.sh start

In order to make sure that all services (docker containers) are running you can run:

./swim.sh status

and you should see something similar to:

CONTAINER ID        IMAGE                  COMMAND                  CREATED              STATUS              PORTS                                                                     NAMES
9c71654ca34b        swim-explorer          "/usr/bin/tini -- ba…"   About a minute ago   Up About a minute   0.0.0.0:5000->5000/tcp                                                    swim_explorer
3d4746c48354        nginx                  "nginx -g 'daemon of…"   About a minute ago   Up About a minute   0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp                                  nginx
645e40b8827a        swim-adsb              "/usr/bin/tini -- ba…"   About a minute ago   Up About a minute                                                                             swim_adsb
a8ffd7d67b7d        subscription-manager   "/usr/bin/tini -- gu…"   About a minute ago   Up About a minute   0.0.0.0:8080->8080/tcp                                                    subscription_manager
0ba8dd7aa1cb        rabbitmq_amqp10        "docker-entrypoint.s…"   About a minute ago   Up About a minute   4369/tcp, 5671-5672/tcp, 15671/tcp, 25672/tcp, 0.0.0.0:15672->15672/tcp   rabbitmq
645b4de022e4        postgres               "docker-entrypoint.s…"   About a minute ago   Up About a minute   0.0.0.0:5432->5432/tcp                                                    postgres

Usage

SWIM Explorer

As soon as the platform is up and running, you can point your browser to http://0.0.0.0:5000 in order to access SWIM Explorer and play around. You can subscribe, pause, resume or unsubscribe from topics like arrivals.brussels and see the real time position of the involved airplanes on the map. The interface will look similar to the below image:

NOTE: SWIM Explorer is supposed to be a client application, .i.e out of the scope of SWIM-TI platform. However, for the purpose of this demo it comes together with the rest of the services, but it runs as a standalone web application not passing through the web server.

alt text

SWIM Subscription Manager

Additionally, you may access the SWIM Subscription Manager OpenAPI specs site using the following link: https://localhost/subscription-manager/api/1.0/ui/#/. Notice that you may get a warning saying the the connections if not safe. This is expected as this demo uses mock certificates and not real ones. In there you can interact with it by observing the various transactions of topics and subscriptions that happen while using SWIM Explorer. You can login using the user and password you provided during the user configuration step earlier. The interface will look like the following image:

alt text

RabbitMQ Management

Lastly, you can also access the RabbitMQ management page using the following link: https://localhost:15671/#/. Notice that you may get a warning saying the the connections if not safe. This is expected as this demo uses mock certificates and not real ones. In there you can observe the queues that are created/deleted while using SWIM Explorer as well as statistics about the flow of the incoming messages. You can login using the user and password you provided during the user configuration step earlier. The interface will look similar to the following image:

alt text

Stopping the Platform

In order to tear the platform down you can do:

./swim.sh stop

If you also want to remove completely the involved docker containers you can do:

./swim.sh stop --clean

and if you want to remove the involved docker containers and volumes as well you can do:

./swim.sh stop --purge

In case there is a change on the involved repositories you can update them by calling:

./swim.sh stop --clean  # if SWIM is up running
./swim.sh build

swim-ti-yp-prototype's People

Contributors

antavelos avatar fjcrabiffosse avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

swim-ti-yp-prototype's Issues

Some Dockerfiles' parent images are deprecated

Hello there, I am trying to make work this demo but i am facing some troubles

  • I manage to correct some build errors due to some changes in debian packets URL, by updating the image parent of concerned Dockerfiles.
  • I still have some SSL issues, the SWIM explorer does not work, I manage to manually create a subscribtion with CURL but not with the explorer.

Is it possible to contact you ? thanks for your help !

swim-base fails to build

The build process of swim-base fails which causes the geofencing build process to fail too, preventing the deployment.

swim-base image fails to build

When running ./swim.sh build for the first time the following error is generated:

Building swim-adsb
Step 1/11 : FROM swim-base
ERROR: Service 'swim-adsb' failed to build: pull access denied for swim-base, repository does not exist or may require 'docker login'

There seems to be some conflict with the conformance branch in the Dockerfile of base.

swim.sh issues

Hey @antavelos ,

Did some tests trying to run the deploy script from scratch following the README.md. Here are some issues I found:

  1. When running . ./swim.sh user_config for the first time the user will get Unable to find image 'swim-user-config:latest' locally

    • Problem: when passing user_config variable to the script we have not run yet clone_repos and build.
    • Solution: We can solve this either via documentation, detailing the steps that need to be run before user_config or make user_config execute them if needed.
  2. usage function should document user_config and option.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.