Code Monkey home page Code Monkey logo

platform's Introduction

Jembi Platform

Tech Used

  • Docker
  • Golang (cli dev)
  • Terraform (remote cluster setup)
  • Ansible (remote cluster setup)

Quick Start for devs (local single node)

  1. If running into an error invalid mount config for type "bind": bind source path does not exist: /tmp/logs on running the CLI binary, run the following command: sudo mkdir -p /tmp/logs/.

  2. ./build-image.sh - builds the platform image

  3. Initialise Docker Swarm mode: docker swarm init

  4. Run go cli binary to launch the project:

    • Linux. From terminal run: ./instant-linux with your selected arguments. A list of available arguments can be found in the help menu by running ./instant-linux help
    • Mac. From terminal run: ./instant-macos with your selected arguments. A list of available arguments can be found in the help menu by running ./instant-macos help

      Warning: Mac has an issue with the binary as it views the file as a security risk. See this article to bypass warning

    • Windows. Double click: platform.exe (Windows users will need to use a release version below 2.0.0)

Quick Start for devs (remote cluster)

To set up a remote cluster environment, see readme in the cloud repo.

  1. Ensure that you have the latest instant repository checked out in the same folder that this repo is in.

  2. ./build-image.sh - builds the platform image

  3. Add .env.prod file with your remote env vars option set.

    Each Package contains a metadata.json file which lists the configurable Env vars and their default values

  4. Run go cli binary to launch the project (Make sure to add the DOCKER_HOST variable indicating your lead Swarm manager, i.e. DOCKER_HOST=ssh://{lead_ip} ./instant-linux):

  5. Check the current cli version in ./get-cli.sh and run to download the binaries. This script can be run with the OS as the first parameter to download only the binary for your prefered OS.

    • Linux. From terminal run: ./instant-linux with your selected arguments. A list of available arguments can be found in the help menu by running ./instant-linux help
    • Mac. From terminal run: ./instant-macos with your selected arguments. A list of available arguments can be found in the help menu by running ./instant-macos help

      Warning: Mac has an issue with the binary as it views the file as a security risk. See this article to bypass warning

    • Windows. Double click: platform.exe (Windows users will need to use a release version below 2.0.0)

Go Cli Dev

The Go Cli scripts are kept in the OpenHIE Package Start Kit Repo. To make changes to the Cli clone the repo and make your changes in the cli directory.

To create new binaries, create a new tag and release and then change the cli version in ./get-cli.sh in the platform repo and run the script to download the latest.

Platform Package Dev

The Go Cli runs all services from the jembi/platform docker image. When developing packages you will need to build your dev image locally with the following command:

./build-image.sh

As you add new packages to the platform remember to list them in config.yaml file. This config file controls what packages the GO CLI can launch.

For logging all output to a file, ensure that you have created the file and it has the required permissions to be written to. The default log file with it's path is set in .env.local in BASHLOG_FILE_PATH. The logPath property in the config.yml is used to create a bind mount for the logs to be stored on the host.

Resource Allocations

The resource allocations for each service can be found in each service's respective docker-compose.yml file under deploy.resources. The field reservations specifies reserved resources for that service, per container. The field limits specifies that maximum amount of resources that can be used by that service, per container.

Each service's resource allocations can be piped into their .yml file through environment variables. Look at the .yml files for environment variable names per service.

Notes on Resource Allocations

  • CPU allocations are specified as a portion of the total number of cores on the host system, i.e., a CPU limit of 2 in a 6-core system is an effective limit of 33.33% of the CPU, and a CPU limit of 6 in a 6-core system is an effective limit of 100% of the CPU.
  • Memory (RAM) allocations are specified as a number followed by their multiplier, i.e., 500M, 1G, 10G, etc.
  • Be wary of allocating CPU limits to ELK Stack services. These seem to fail with CPU limits and their already implemented health checks.
  • Take note to not allocate less memory to ELK Stack services than their JVM heap sizes.
  • Exit code 137 indicates an out-of-memory failure. When running into this, it means that the service has been allocated too little memory.

Build multi-platform docker images

It's essential to make sure that any docker image should be available for multiple platforms : AMD, ARM, ... (not only linux, but MacOS as well). To do so you can follow the steps below :

  1. Create your own custom builder by running docker buildx create --name mycustombuilder --driver docker-container --bootstrap
  2. Ask docker to use this new builder for future builds by running docker buildx use mycustombuilder
  3. Inspect buildx to see if docker has indeed switched builders to the new one you asked it to use by running docker buildx inspect
  4. Then you can perform the build and push, for example : docker buildx build --platform linux/amd64,linux/arm64 --push -t jembi/hapi:v7.0.3-wget .

Tests

Tests are located in /test

Cucumber

Tests that execute platform-linux with parameters and observe docker to assert expected outcomes

View /test/cucumber/README.md for more information

platform's People

Contributors

ajinkyagadewar avatar arran-standish avatar bmartinos avatar bradsawadye avatar brett-onions avatar castellog avatar chroniclesofnadia avatar drizzentic avatar ilyesbenamara avatar itsmurumba avatar lumiere-lodi avatar mark-labuschagne avatar marrouchi avatar matthewerispe avatar mattyj007 avatar michaeljembi avatar michaelloosen avatar nour-borgi avatar rcrichton avatar tumbledwyer avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

platform's Issues

ENV for JEMPI not updating

Using the recipe (Central Data Repository with Data Warehousing), when customizing the ENV for jempi is not taking effect. It is still calling the localhost. I am changing it to the IP address of the sever. I've pulled the latest jembi/platform:latest created 2024-01-19 21:37:51.

REACT_APP_JEMPI_BASE_API_HOST=http://server-ip
REACT_APP_JEMPI_BASE_API_PORT=50000

REACT_APP_JEMPI_BASE_URL=http://server-ip/JeMPI
KC_JEMPI_ROOT_URL=http://server-ip:3033

jempi

Unbound variable

If the environment variable file does not specify CLUSTERED_MODE=..., the error unbound variable is thrown. This variable should have a default value in the package-metadata.json file.

Got the error when deploying Kibana.

Logstash copy config not working

The addition of the -a to the docker cp command seems to be breaking copy-configs.
Removing it may have some permission implications though so needs to be checked out.

The offending commit: cb82388

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.