Code Monkey home page Code Monkey logo

purple's Introduction

Purple

GitHub Actions status

An all-in-one data service with support for:

  • Key/value operations
  • Counters and sets
  • Flags (basically key/value pairs where the value is a Boolean with a default value of false)
  • Caching with TTL

Purple is meant to abstract away complex database interfaces (Redis, DynamoDB, Mongo, memory, disk, etc.) in favor of a unified set of dead-simple operations (see the full list of operations below).

You can run Purple as a gRPC server or an HTTP server (both expose the same interfaces). There's currently a gRPC client for Go only but in principle gRPC clients could be added for other languages. There are also three backends available: memory, disk, and Redis.

Since any server type can work with any backend, the following server/backend combinations are currently supported:

Server Backend
gRPC Memory
gRPC Disk
gRPC Redis
HTTP Memory
HTTP Disk
HTTP Redis

The project

Goals

Microservices or FaaS functions that rely on stateful data operations can use Purple instead of needing to interact with multiple databases. This greatly simplifies the service/function development process by sharply reducing the hassle of dealing with databases (i.e. no need to install/learn/use 5 different database clients).

Does your service need something that isn't provided by Purple? File an issue or submit a PR and I'll add it!

Current status

Purple is in its very early stages. The data interfaces it provides are almost comically simple. Please do not use Purple as a production data service just yet (though I'd like to get there). Instead, use it for prototyping and experimenting.

Also be aware that Purple runs as a single instance and has no clustering built in (and thus isn't highly available). If you use the Redis backend, however, you can run multiple instances of Purple that connect to a single Redis cluster.

Future directions

In the future, I imagine Purple acting as an abstraction layer over lots of different data systems, exposing a powerful interface that covers the overwhelming majority of data use cases without exposing the system internals of any of those systems. This would entail:

  • Making the current data interfaces more sophisticated and capable of covering a wider range of use cases
  • Adding new interfaces, such as a timeseries interface, a simple graph interface, etc.
  • Providing a relational interface that supports a subset of SQL (SQLite would likely suffice for this)
  • Providing optional pluggable backends behind Purple (e.g. using Redis for caching, Elasticsearch for search, etc.)
  • Providing a message queue/pub-sub interface, eliminating the need for a Kafka/Pulsar/RabbitMQ/etc. client

Want to contribute?

See the contributors guide for details.

Operations

The table below lists the available client operations for the Go client:

Operation Service Semantics
CacheGet(key string) Cache Fetches the value of a key from the cache or returns a not found error if the key doesn't exist or has expired.
CacheSet(key, value string, ttl int32) Cache Sets the value associated with a key and assigns a TTL (the default is 5 seconds). Overwrites the value and TTL if the key already exists.
CounterIncrement(key string, amount int64) Counter Increments a counter by the designated amount. Returns the new value of the counter or an error.
CounterGet(key string) Counter Fetches the current value of a counter. Returns zero if the counter isn't found.
FlagGet(key string) Flag Fetches the current Boolean value of a flag. If the flag hasn't yet been set, the default value is false.
FlagSet(key string, value bool) Flag Sets the Boolean value of a flag.
SetGet(set string) Set Fetch the items currently in the specified set. Returns an empty string set ([]string) if the set isn't found.
SetAdd(set, item string) Set Adds an item to the specified set and returns the resulting set.
SetRemove(set, item string) Set Removes an item from the specified set and returns the resulting set. Returns an empty set isn't found or is already empty.
KVGet(key string) KV Gets the value associated with a key or returns a not found error. The value is currently just a byte array payload but could be made more complex later (e.g. a payload plus a content type, metadata, etc.).
KVPut(key string, value *Value) KV Sets the value associated with a key, overwriting any existing value.
KVDelete(key string) KV Deletes the value associated with a key or returns a not found error.

Backends

There are currently three backends available for Purple:

Backend Explanation
Disk Data is stored persistently on disk using the Badger library. Each service (cache, KV, etc.) is stored in its own separate on-disk DB, which guarantees key isolation.
Memory Data is stored in native Go data structures (maps, slices, etc.). This backend is blazing fast but all data is lost when the service restarts.
Redis The Purple server stores all data in a persistent Redis installation. Each service uses a different Redis database, which provides key isolation.

Try it out

To try out Purple locally, you can run the Purple gRPC server in one shell session and some example client operations in another session:

git clone https://github.com/purpledb/purple && cd purple

# Start the gRPC server...
go run examples/grpc-server/main.go

# And then in a different session...
go run examples/grpc-client/main.go

Installation

gRPC server

To install the Purple gRPC server:

# Executable
go install github.com/purpledb/purple/cmd/purple-grpc

# Docker image
docker pull purpledb/purple-grpc:latest

Then you can run it:

# Executable
purple-grpc

# Docker image
docker run --rm -it -p 8081:8081 purpledb/purple-grpc:latest

You should see log output like this:

2019/07/27 14:37:09 Starting up the server on port 8081

HTTP server

To install the purple HTTP server:

# Executable
go install github.com/purpledb/purple/cmd/purple-http

# Docker image
docker pull purpledb/purple-http:latest

Then you can run it:

# Executable
purple-http

# Docker image
docker run --rm -it -p 8080:8080 purpledb/purple-http:latest

gRPC Go client

To use the Go client in your service or FaaS function:

go get github.com/purpledb/purple

To instantiate a client:

import "github.com/purpledb/purple"

// Supply the address of the purple gRPC server
client, err := purple.NewGrpcClient("localhost:8081")
if err != nil { 
    // Handle error
}

// Now you can run the various data operations, for example:
if err := client.CacheSet("player1-session", "a1b2c3d4e5f6", 120); err != nil {
    // Handle error
}

HTTP Go client

import "github.com/purpledb/purple"

client, err := purple.NewHttpClient("http://localhost:8080")
if err != nil {
    // Handle error
}

count, err := client.CounterIncrement("player1-points", 100)

Deployment

Kubernetes

There are two configuration files in the deploy directory that enable you to run the purple gRPC and HTTP servers, respectively, on Kubernetes. Both use the default namespace.

gRPC

kubectl apply -f deploy/purple-grpc-k8s.yaml

HTTP

kubectl apply -f deploy/purple-http-k8s.yaml

Accessing the service

Once you've deployed purple on Kubernetes, you can access it in your local environment using port forwarding:

# gRPC
kubectl port-forward svc/purple 8081:8081

# HTTP
kubectl port-forward svc/purple 8080:8080

purple's People

Contributors

lucperkins avatar

Watchers

James Cloos avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.