Code Monkey home page Code Monkey logo

spaceboard's Introduction

spaceboard

A distributed blackboard spawning Linux® daemons inside Singularity containers managed by Luerl processes that grow over time into other units; nodes provide granular unit control with in clusters.

Community

Spacebeam is a multidisciplinary open-source research & development community that conducts work on distributed systems, artificial intelligence and high-performance computing.

Explore and remember; as the system grow in number, spawn more nodes to control it.

Help wanted

Would you like to help with the project? Pick any of the issues and contribute!

Contributing

See Contributing.

spaceboard's People

Contributors

anthony6891 avatar jchassoul avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

spaceboard's Issues

MPI Support

Mirror singularity container support of standard HPC MPI protocols and open-source implementation, our support for MPI is by delegation through the container layer this is given the nature of HPC centers and schedulers like slurm where the integration between the scheduler and the MPI job is bundled.

For more information on this, please read the MPI section of the awesome singularity documentation.

The blackboard pattern

What is the blackboard pattern?

The blackboard pattern is a behavioral design pattern that provides a "computational framework " (?) for the design and implementation of systems that integrate large and diverse specialized processes, and implement complex, non-deterministic control strategies.

The blackboard pattern allows multiple processes to work closer together on separate threads, polling and reacting when necessary, the pattern was originally designed as a way to handle complex, ill-defined problems, where the solution is the sum of its parts.

Structure

This architecture defines three main components:

  1. The blackboard, a shared repository of problems, partial solutions, suggestions, and contributed information. A dynamic "library" of contributions to the current problem that have been recently "published" by other knowledge sources.
  2. The knowledge sources, independent processes with specific expertise needed by the application.
  3. The control component selects, configures and executes processes which controls the flow of the problem-solving activity in the system.

Blackboard system

An artificial intelligence approach based on the blackboard architectural model, where a common knowledge base, the "blackboard", is iteratively updated by a diverse group of specialist knowledge sources, starting with a problem specification and ending with a solution. Each process updates the blackboard with a partial solution when its internal constraints match the blackboard state. In this way, the processes work together to solve the problem.

Opportunistic reasoning

A method of selecting a suitable logical inference strategy within artificial intelligence applications.

Specific reasoning methods may be used to draw conclusions from a set of given facts in a knowledge base, e.g. forward chaining versus backward chaining. However, in opportunistic reasoning, pieces of knowledge may be applied either forward or backward, at the "most opportune time".

An opportunistic reasoning system may combine elements of both forward and backward reasoning. It is useful when the number of possible inferences is very large and the reasoning system must be responsive to new data that may become known.

timestamp on sub_bind received from erlang

From python and lua we're sending a unix timestamp o every message event, all zmq messages are handle with the BEAM, there's this random issue sending int's so, please put the timestamps inside a python or lua string() just for now, yep it's ugly but it works and we're sending them with JSON so...

mon is for monitor

Are we planning to support our current stack of monit, circus and supervisord? How this work together with processes and pids of singularity instances.

WX Failed loading "wxe_driver"@"/usr/lib/erlang/lib/wx-1.8/priv"

on run.sh trying to enabling observer by -run observer results on:

WX Failed loading "wxe_driver"@"/usr/lib/erlang/lib/wx-1.8/priv"

the error it's found on the console and the observer does not show.

pragmatically, we're just disabling the option that was enabled by default in current dev mode.

but... observer is cool! we need to figure it out what is the problem!

(=

apps resource missing

what is this? a generic application resource that mirror singularity container apps and follow the scientific standard container specification.

also add the links of the spec and singularity documentation about applications in the context of singularity containers.

Space-based architecture

Space-based what?

Space-based architecture is a software architecture pattern for achieving linear scalability of stateful, high-performance applications using the tuple space paradigm.

With a space-based architecture, applications are built out of set of self-sufficient units, known as processing-units. These units are independent of each other, so that the application can scale by adding more units.

The model is closely related to other patterns that have been proved successful in addressing the application scalability challenge, such as the shared nothing architecture used by Erlang.

Tuple space paradigm

A tuple space is an implementation of the associative memory paradigm for parallel/distributed computing. It provides a repository of tuples that can be accessed concurrently.

Consider that there are a group of processors that produce pieces of data and a group of processors that use the data. Producers post their data as tuples in the space, and the consumers then retrieve data from the space that match a certain pattern. This is also known as the blackboard metaphor.

Space-style systems

In the "space" or "tuple space" style of distributed computing, participating processors share data containing objects through an intermediary system that acts like a huge "associative memory". An associative memory lets you address objects by their content rather than their physical location.

The term "tuple" in "tuple space" also "tuplespace" implies that the data containing objects may contain more than one value that can be used in locating the object.

Processes that need some computing resources write into the space an object with values defining what is needed defining the variables that must be returned. Worker processes register with the space as being able to perform work on objects with specified values. The space matches jobs with workers and sends the object to a worker, which returns it after performing the needed computation. Complex tasks can be accomplished by passing an object from one specialized worker program to the next.

Associative memory

Compares input search data (tag) against a table of stored data, and returns the addresses of matching data (or in the case of associative memory, the matching data). Typically exposes a digest generated by a cryptographic hash function from the document it refers to.

Tuples

In mathematics, a tuple is a finite ordered list (sequence) of elements. An n-tuple is a sequence (or ordered list) of n elements, where n is a non-negative integer. There is only one 0-tuple, an empty sequence, or empty tuple, as it is referred to.

Check luerl_zmq implementation

The implementation define the interface with the goal of consistently math lzqm the luajit ZeroMQ API, when it is possible and only if it make sense in the context of the luerl implementation.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.