First of, great work on this project! I've gotten a bit stuck with g

Hi, <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url=

Thanks for the help <a class="user-mention notranslate" data-hovercard-type="user" dat

Document how to use rabbitmq inside the docker setup about aleph HOT 5 CLOSED

alephdata commented on July 21, 2024

Document how to use rabbitmq inside the docker setup

from aleph.

Comments (5)

pudo commented on July 21, 2024

Hi, @JakeChampion glad to hear you find this useful :) Looking at your docker-compose config, I'm wondering how exactly you've configured the rabbitmq connection string (CELERY_BROKER_URL, or environment variable ALEPH_BROKER_URI) in the settings.py config you are using. The 'docker' way of doing this would be to use container-to-container networking, using a string like amqp://guest:guest@rabbitmq:5672// and to use expose instead of ports to expose the port 5672 only to the other docker containers.

from aleph.

JakeChampion commented on July 21, 2024

Thanks for the help @pudo. I got the service working locally using what you mentioned. Here is my new, now working, docker-compose.yml file:

postgres:
  image: postgres:9.4
  volumes:
    - "/opt/aleph/data/postgres:/var/lib/postgresql/data"
    - "/opt/aleph/logs/postgres:/var/log"
  environment:
    POSTGRES_USER:     aleph
    POSTGRES_PASSWORD: aleph
    POSTGRES_DATABASE: aleph
  ports:
   - "127.0.0.1:5439:5432"

elasticsearch:
  image: elasticsearch:2.2.0
  volumes:
    - "/opt/aleph/data/elasticsearch:/usr/share/elasticsearch/data"
    - "/opt/aleph/logs/elasticsearch:/var/log"
  ports:
    - "127.0.0.1:9201:9209"
  # environment:
  # ES_HEAP_SIZE: 4g

worker:
    build: .
    command: celery -A aleph.queue worker -c 10 -l INFO --logfile=/var/log/celery.log
    links:
      - postgres
      - elasticsearch
      - rabbitmq
    volumes:
      - "/:/host"
      - "/opt/aleph/data:/opt/aleph/data"
      - "/opt/aleph/logs/worker:/var/log"
    environment:
      C_FORCE_ROOT: 'true'
      ALEPH_ELASTICSEARCH_URI: http://elasticsearch:9200/
      ALEPH_DATABASE_URI: postgresql://aleph:aleph@postgres/aleph
      POLYGLOT_DATA_PATH: /opt/aleph/data
      TESSDATA_PREFIX: /usr/share/tesseract-ocr
      ALEPH_BROKER_URI: amqp://guest:guest@rabbitmq:5672
    env_file:
      - aleph.env

beat:
    build: .
    command: celery -A aleph.queue beat -s /var/run/celerybeat-schedule
    links:
      - postgres
      - elasticsearch
      - rabbitmq
    volumes:
      - "/opt/aleph/logs/beat:/var/log"
      - "/opt/aleph/run/beat:/var/run"
    environment:
      C_FORCE_ROOT: 'true'
      ALEPH_ELASTICSEARCH_URI: http://elasticsearch:9200/
      ALEPH_DATABASE_URI: postgresql://aleph:aleph@postgres/aleph
      ALEPH_BROKER_URI: amqp://guest:guest@rabbitmq:5672
    env_file:
      - aleph.env

web:
    build: .
    command: gunicorn -w 5 -b 0.0.0.0:8000 --log-level info --log-file /var/log/gunicorn.log aleph.manage:app
    ports:
      - "13376:8000"
    links:
      - postgres
      - elasticsearch
      - rabbitmq
    volumes:
      - "/opt/aleph/logs/web:/var/log"
    environment:
      ALEPH_ELASTICSEARCH_URI: http://elasticsearch:9200/
      ALEPH_DATABASE_URI: postgresql://aleph:aleph@postgres/aleph
      ALEPH_BROKER_URI: amqp://guest:guest@rabbitmq:5672
    env_file:
      - aleph.env
rabbitmq:
  image: rabbitmq
  ports:
    - 5672

I'm now trying to bypass the need for Google OAUTH, I hope it's as easy as swapping out SQS for RabbitMQ :)

from aleph.

pudo commented on July 21, 2024

Do you have access to any other form of OAuth? It's possible to hook into the login code to support other providers -- here's the code we're using at OCCRP:

@signals.handle_oauth_session.connect
def handle_google_oauth(sender, provider=None, session=None):
    from aleph.model import Role
    if 'investigativedashboard.org' not in provider.base_url:
        return
    me = provider.get('api/2/accounts/profile/')
    user_id = 'idashboard:user:%s' % me.data.get('id')
    role = Role.load_or_create(user_id, Role.USER,
                               me.data.get('display_name'),
                               email=me.data.get('email'),
                               is_admin=me.data.get('is_admin'))
    session['roles'].append(role.id)
    session['user'] = role.id
    for group in me.data.get('groups', []):
        group_id = 'idashboard:%s' % group.get('id')
        group_role = Role.load_or_create(group_id, Role.GROUP,
                                         group.get('name'))
        session['roles'].append(group_role.id)

This needs to be made to load as a plugin module, using a distutils entrypoint to make sure the plugin gets loaded:

from setuptools import setup, find_packages

setup(
    name='aleph_occrp',
    version='0.2',
    [...]
    entry_points={
        'aleph.init': [
            'occrpext = aleph_occrp:register'
        ]
    }
)

Otherwise this will fall back onto #52.

from aleph.

pudo commented on July 21, 2024

How's it going? :)

from aleph.

JakeChampion commented on July 21, 2024

I don't have access to another form of OAuth, sorry.

from aleph.

Document how to use rabbitmq inside the docker setup about aleph HOT 5 CLOSED

Comments (5)

Related Issues (20)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent