Code Monkey home page Code Monkey logo

msgflo-python's Introduction

MsgFlo - Flow-Based Programming with Message Queues Build Status

Implementation of the Flow-Based Programming using message queues as the communications layer between different processes. Initial message queue transports targeted are AMQP and MQTT.

MsgFlo lets you build robust polyglot FBP systems spanning multiple computers/devices. A node can be implemented in any language, to reuse existing code, libraries and developer know-how.

In FBP each component is a black-box that processes and produces data, without knowledge about where the input data comes from, or where the output data goes. This ensures that a service is easy to change, and facilitates automated testing.

MsgFlo is designed to enable partial and gradual integration into existing systems; by using standard broker/transports, not placing restrictions on message payloads, allowing to use existing queue names, and integrating non-MsgFlo nodes seamlessly.

Status

In Production

  • Used in production at TheGrid website builder, with AMQP/RabbitMQ. 20 roles, 1'000'000 jobs/weekly+
  • Used in production in imgflo image processing server. 4 roles, 200'000 jobs/weekly+
  • Used for IoT networks at hackerspaces c-base and Bitraf, using MQTT/Mosquitto.

Client support

Tooling

  • msgflo executable implements the FBP runtime protocol.
  • Initial support for automated testing using fbp-spec
  • Experimental support for visually building networks using Flowhub
  • guv provides autoscaling of workers when using Heroku/AMQP.

Licence

MIT, see ./LICENSE

Documentation

Please refer to https://msgflo.org

Support

Flowhub logo

MsgFlo is a part of Flowhub, a platform for building robust IoT systems and web services.
We offer an Integrated Development Environment and consulting services.

Debugging

The msgflo executable, as well as the transport/participant library uses the debug NPM module. You can enable (all) logging using:

export DEBUG=msgflo*

msgflo-python's People

Contributors

bergie avatar chadrik avatar greenkeeper[bot] avatar jonnor avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

Forkers

chadrik uwekamper

msgflo-python's Issues

Register python participant as topic

In Nodejs or noflo I can register the prticipant under the name. But in python How to register the participant

msgflo-nodejs --name repeater ./node_modules/msgflo-nodejs/examples/Repeat.coffee

Here I registered the participant under the "repeater" name. So I can send message to repeater.IN in mqtt. But how to do that for msgflo-python?

I'm following the example here -> https://github.com/msgflo/msgflo-python

Kindly help me on this. Thanks in advance :)

Allow non-JSON payloads

Currently msgflo-python fails if payloads can't be serialized as JSON:

Traceback (most recent call last):
  File "/home/bergie/.local/lib/python2.7/site-packages/gevent/greenlet.py", line 536, in run
    result = self._run(*self.args, **self.kwargs)
  File "/home/bergie/Projects/c-base/farbgeber/farbgeber.py", line 107, in loop
    self.send_palette(palette)
  File "/home/bergie/Projects/c-base/farbgeber/farbgeber.py", line 128, in send_palette
    self.send('palette', palette)
  File "/home/bergie/.local/lib/python2.7/site-packages/msgflo/msgflo.py", line 54, in send
    self._engine._send(outport, outdata)
  File "/home/bergie/.local/lib/python2.7/site-packages/msgflo/msgflo.py", line 234, in _send
    serialized = json.dumps(data)
  File "/usr/lib/python2.7/json/__init__.py", line 243, in dumps
    return _default_encoder.encode(obj)
  File "/usr/lib/python2.7/json/encoder.py", line 207, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib/python2.7/json/encoder.py", line 270, in iterencode
    return _iterencode(o, 0)
  File "/usr/lib/python2.7/json/encoder.py", line 184, in default
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: <Color #cc5d00> is not JSON serializable

An in-range update of msgflo is breaking the build 🚨

Version 0.10.24 of msgflo just got published.

Branch Build failing 🚨
Dependency msgflo
Current Version 0.10.23
Type devDependency

This version is covered by your current version range and after updating it in your project the build failed.

As msgflo is β€œonly” a devDependency of this project it might not break production or downstream projects, but β€œonly” your build or test tools – preventing new deploys or publishes.

I recommend you give this issue a high priority. I’m sure you can resolve this πŸ’ͺ

Status Details
  • ❌ continuous-integration/travis-ci/push The Travis CI build could not complete due to an error Details

Commits

The new version differs by 16 commits.

  • 415ac5a Bump
  • c5375ae Merge pull request #144 from msgflo/nodes_to_json
  • adc855a Merge pull request #143 from msgflo/fix-remove
  • 9c620d8 Edges are between roles, not between participant IDs
  • eb7221a Read correct var
  • 10059e2 Get nodes for JSON graph from nodes data structure, not participants
  • 4a758a4 Update node data structure only after sanity checks
  • 8abf1bb Ensure discovered participants exist as nodes
  • 92e3b15 clearGraph: Also clear nodes and participants
  • f926cee tests: Check clearing graph
  • 98f236d stopParticipant(): Fix improper cleanup/call
  • b1ac80a tests: Check that removing node is removed from graph source
  • 454a403 runtime: Also stop coordinator
  • 513d942 stopParticipant(): Remove node/discovered participant
  • 451dbe0 tests: Use dedicated component name in setsource test

There are 16 commits in total.

See the full diff

Not sure how things should work exactly?

There is a collection of frequently asked questions and of course you may always ask my humans.


Your Greenkeeper Bot 🌴

IIP support

Just like noflo-runtime-msgflo, would be great to support initial information packets for msgflo-python

Support Python 3

gevent 1.1alpha2 apparently supports it. Might need work in Haigha though

Tests must run both Py 2 and Py 3

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.