Code Monkey home page Code Monkey logo

echo's Introduction

Echo

A Sequentially Consistent Replicated Key-Value data store.

API

  1. Get(key): This method retrieves the value corresponding to the given key from the currently connected replica. The system is expected to provide stale read values until the latest commit is received by the connected replica.
  2. Put(key,value): This method puts the value with the given key in the data store. In the current instance, a replica can send at most one write request in its request queue.

Architecture

Leader

The leader orders every write request in the data store. When a leader receives a write request from the replica, it assigns a unique number to it and performs its commit. Since all the requests are added to the queue, there exists a valid sequence to order the requests. The leader uses two threads to commit a transaction (one to listen to write requests from the replicas and one to start the commit protocol).

Threads

Replica

The replica receives requests from the clients. If the request is of type 'get', then the replica will serve the request from its local datastore. If the request is of type 'put', it sends the request to the leader to commit the write to all the other replicas in the same order. The replica is a three-threaded program that maintains a server to handle connections from clients and a socket to connect to the leader.

system architecture

Dependencies

  1. Java SE 17
  2. Gradle
  3. Apache ZooKeeper
  4. Apache Commons Lang
  5. Log4J

Gradle automatically installs the above dependencies during the project build.

Download the Apache ZooKeeper to setup ZooKeeper server.

Installation steps

  1. Download the latest repository from git.

  2. Open the project in any (IntelliJ preferred) IDE and build the project.

  3. Start the ZooKeeper Server on port 8000.

  4. Run SetupScript.java to set up znodes in ZooKeeper Server.

  5. Run Central.java to start the load balancer.

  6. Run Service.java on a separate machine.

  7. start multiple instances of Service.java (except steps 3 and 4) in different machines to set the desired number of replicas.

  8. Run Client.java on a different machine (multiple instances for multiple clients)

Replicas and clients can be run in a single machine in a standalone environment.

Note

We kept our API as simple as possible to give the users a standard interface like a regular data store. The decision to make the write requests to process only through the leader is intentional to provide Sequential Consistency. Our datastore is sequentially consistent if the network guarantees no partitions and jitters in the network.

Most of the design decisions are finalized after considering many alternative design models and implementation difficulties. We want to build this project with realistic assumptions. In addition, we modified the 1-phase commit protocol to integrate with our architecture. This decision is to make the I/O channels transfer as little data as possible to commit a transaction.

Contribution

We welcome any feedback and suggestions for this project. Feel free to submit an issue/feedback/suggestions.

echo's People

Contributors

var-nan avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.