Code Monkey home page Code Monkey logo

fabric_investigation's Introduction

fabric_investigation

This repository is for code related to the investigation of FABRIC as a type of resource useful to CMB-S4. More information for FABRIC is at FABRIC.

A full investigation would have prompt map making and transient processing occur in a fabric-like infrastructure.

Phase one

The Phase One project is to mock up transient processing for the CMB-S4 DC1 within Fabric. The minimal intergration planned for evaluation is to attempt::

  • Dynamically provision FABRIC Resources
  • Ingest the maps needed for transient processing, thought to be 30 "per observations maps"
  • Run transient detection codes.
  • Emit transients to an endpoint representing "the transient community"
  • Tear Down FABRIC resources.

and write up results.

Notes on SCIMMA HOP usage

We are using the hop-client package available from pip.

In the scimma hopskotch instance, The phase one channel is cmb-s4-fabric-tests.phase-one-testing

Before use,

  • obtain an identity and SCRAM credential from my.hop.scimma.org
  • use the "hop auth add" to store the credential (Leave hostname blank)
  • Ask Don to authorize you to acess the cmb-s4-fabric-tests.phase-one-testing topic.
  • arrange for outbound (kafka) port 9092 to be open
  • test connectivity with 'echo 'test' | nc -w 10 kafka.scimma.org 9092; echo $?'

scimma_utils

The scimma_utils packages included here further wraps the pbulish and subscribe subfunctions in hop.

More about hop

Tutorial is here. Shell Command documentation is here.

If you get errors about certificates

your OS may be supplying a version of openssl with a bug in it. A work around is:

  • $hop auth locate
  • cat the file gven by hop auth locate
  • edit the file pointed to by ssl_ca_location
  • find the DST Root CA X3, in that file delete/cut that certificate out of it,

'

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.