integrandio / lucidmq Goto Github PK
View Code? Open in Web Editor NEWLucidMQ - Simple Ops Event Streaming. Alternative to Kafka and RabbitMQ
Home Page: https://integrand.io
License: MIT License
LucidMQ - Simple Ops Event Streaming. Alternative to Kafka and RabbitMQ
Home Page: https://integrand.io
License: MIT License
On startup, allow for the user to pass in a flag to indicate log level. This will be useful for reducing logs in CI and enabling them during local development.
In order to increase velocity and correctness, we need to be able to run integration tests during our CI stage. We will leverage docker and docker compose to accomplish this.
lucidmq-py
Right now producer in LucidMQ-py only sends one message. Let's create a clean API for sending multiple message including a batching solution for many that are sent in short succession.
There is a lot of code duplication in our LucidMQ and LucidMQ-cli code. We can reduce code by creating a shared library and importing it across both packages.
A consumer calls poll which waits for messages to come through the commit-log. Since it is using a RWLock there can be lock contention on the same topic/commit-log. If a consumer gives a high timeout, other operations could be locked out due to this timeout period. We need a less contentious heavy solution to no block all producers/consumers to the same topic.
In order to make our Python client easy to use, we want it pushed to PyPI.
Tests cover happy path. However, when a client sends bytes that cannot be deserialized to a protocol message, LucidMQ panics/breaks. We want to gracefully handle this and add to our test suite.
There is no standalone library for LucidMQ in rust. All that code is in the cli. Let's refactor it out and creat a client library.
Add a golang client library to interact with the LucidMQ instance.
Utilize raft to make LucidMQ fault tolerant and distributed.
Library:
https://github.com/tikv/raft-rs
Here is an example of someone actually using the library:
https://www.hyperledger.org/blog/2019/01/11/floating-the-sawtooth-raft-implementing-a-consensus-algorithm-in-rust
The all top is message using the topic request message. This requires a topic name. Seems unnecessary to even send those bytes across the wire. We want a better solution for this as well as tests for this in out integ test suite.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.