An all-in-one data service with support for:
- Key/value operations
- Counters and sets
- Flags (basically key/value pairs where the value is a Boolean with a default value of
false
) - Caching with TTL
Purple is meant to abstract away complex database interfaces (Redis, DynamoDB, Mongo, memory, disk, etc.) in favor of a unified set of dead-simple operations (see the full list of operations below).
You can run Purple as a gRPC server or an HTTP server (both expose the same interfaces). There's currently a gRPC client for Go only but in principle gRPC clients could be added for other languages. There are also three backends available: memory, disk, and Redis.
Since any server type can work with any backend, the following server/backend combinations are currently supported:
Server | Backend |
---|---|
gRPC | Memory |
gRPC | Disk |
gRPC | Redis |
HTTP | Memory |
HTTP | Disk |
HTTP | Redis |
Microservices or FaaS functions that rely on stateful data operations can use Purple instead of needing to interact with multiple databases. This greatly simplifies the service/function development process by sharply reducing the hassle of dealing with databases (i.e. no need to install/learn/use 5 different database clients).
Does your service need something that isn't provided by Purple? File an issue or submit a PR and I'll add it!
Purple is in its very early stages. The data interfaces it provides are almost comically simple. Please do not use Purple as a production data service just yet (though I'd like to get there). Instead, use it for prototyping and experimenting.
Also be aware that Purple runs as a single instance and has no clustering built in (and thus isn't highly available). If you use the Redis backend, however, you can run multiple instances of Purple that connect to a single Redis cluster.
In the future, I imagine Purple acting as an abstraction layer over lots of different data systems, exposing a powerful interface that covers the overwhelming majority of data use cases without exposing the system internals of any of those systems. This would entail:
- Making the current data interfaces more sophisticated and capable of covering a wider range of use cases
- Adding new interfaces, such as a timeseries interface, a simple graph interface, etc.
- Providing a relational interface that supports a subset of SQL (SQLite would likely suffice for this)
- Providing optional pluggable backends behind Purple (e.g. using Redis for caching, Elasticsearch for search, etc.)
- Providing a message queue/pub-sub interface, eliminating the need for a Kafka/Pulsar/RabbitMQ/etc. client
See the contributors guide for details.
The table below lists the available client operations for the Go client:
Operation | Service | Semantics |
---|---|---|
CacheGet(key string) |
Cache | Fetches the value of a key from the cache or returns a not found error if the key doesn't exist or has expired. |
CacheSet(key, value string, ttl int32) |
Cache | Sets the value associated with a key and assigns a TTL (the default is 5 seconds). Overwrites the value and TTL if the key already exists. |
CounterIncrement(key string, amount int64) |
Counter | Increments a counter by the designated amount. Returns the new value of the counter or an error. |
CounterGet(key string) |
Counter | Fetches the current value of a counter. Returns zero if the counter isn't found. |
FlagGet(key string) |
Flag | Fetches the current Boolean value of a flag. If the flag hasn't yet been set, the default value is false . |
FlagSet(key string, value bool) |
Flag | Sets the Boolean value of a flag. |
SetGet(set string) |
Set | Fetch the items currently in the specified set. Returns an empty string set ([]string ) if the set isn't found. |
SetAdd(set, item string) |
Set | Adds an item to the specified set and returns the resulting set. |
SetRemove(set, item string) |
Set | Removes an item from the specified set and returns the resulting set. Returns an empty set isn't found or is already empty. |
KVGet(key string) |
KV | Gets the value associated with a key or returns a not found error. The value is currently just a byte array payload but could be made more complex later (e.g. a payload plus a content type, metadata, etc.). |
KVPut(key string, value *Value) |
KV | Sets the value associated with a key, overwriting any existing value. |
KVDelete(key string) |
KV | Deletes the value associated with a key or returns a not found error. |
There are currently three backends available for Purple:
Backend | Explanation |
---|---|
Disk | Data is stored persistently on disk using the Badger library. Each service (cache, KV, etc.) is stored in its own separate on-disk DB, which guarantees key isolation. |
Memory | Data is stored in native Go data structures (maps, slices, etc.). This backend is blazing fast but all data is lost when the service restarts. |
Redis | The Purple server stores all data in a persistent Redis installation. Each service uses a different Redis database, which provides key isolation. |
To try out Purple locally, you can run the Purple gRPC server in one shell session and some example client operations in another session:
git clone https://github.com/purpledb/purple && cd purple
# Start the gRPC server...
go run examples/grpc-server/main.go
# And then in a different session...
go run examples/grpc-client/main.go
To install the Purple gRPC server:
# Executable
go install github.com/purpledb/purple/cmd/purple-grpc
# Docker image
docker pull purpledb/purple-grpc:latest
Then you can run it:
# Executable
purple-grpc
# Docker image
docker run --rm -it -p 8081:8081 purpledb/purple-grpc:latest
You should see log output like this:
2019/07/27 14:37:09 Starting up the server on port 8081
To install the purple HTTP server:
# Executable
go install github.com/purpledb/purple/cmd/purple-http
# Docker image
docker pull purpledb/purple-http:latest
Then you can run it:
# Executable
purple-http
# Docker image
docker run --rm -it -p 8080:8080 purpledb/purple-http:latest
To use the Go client in your service or FaaS function:
go get github.com/purpledb/purple
To instantiate a client:
import "github.com/purpledb/purple"
// Supply the address of the purple gRPC server
client, err := purple.NewGrpcClient("localhost:8081")
if err != nil {
// Handle error
}
// Now you can run the various data operations, for example:
if err := client.CacheSet("player1-session", "a1b2c3d4e5f6", 120); err != nil {
// Handle error
}
import "github.com/purpledb/purple"
client, err := purple.NewHttpClient("http://localhost:8080")
if err != nil {
// Handle error
}
count, err := client.CounterIncrement("player1-points", 100)
There are two configuration files in the deploy
directory that enable you to run the purple gRPC and HTTP servers, respectively, on Kubernetes. Both use the default
namespace.
kubectl apply -f deploy/purple-grpc-k8s.yaml
kubectl apply -f deploy/purple-http-k8s.yaml
Once you've deployed purple on Kubernetes, you can access it in your local environment using port forwarding:
# gRPC
kubectl port-forward svc/purple 8081:8081
# HTTP
kubectl port-forward svc/purple 8080:8080