Code Monkey home page Code Monkey logo

sled's Introduction

sled

Build Status crates.io documentation

A modern embedded database.

extern crate sled;

let tree = sled::Config::default()
  .path(path)
  .tree();

// set and get
tree.set(k, v1);
assert_eq!(tree.get(&k), Some(v1));

// compare and swap
tree.cas(k, Some(v1), Some(v2));

// scan forward
let mut iter = tree.scan(k);
assert_eq!(iter.next(), Some((k, v2)));
assert_eq!(iter.next(), None);

// deletion
tree.del(&k);

features

  • ordered map API
  • fully atomic single-key operations, supports CAS
  • zstd compression (use the zstd build feature)
  • cpu-scalable lock-free implementation
  • SSD-optimized log-structured storage

goals

  1. don't make the user think. the interface should be obvious.
  2. don't surprise users with performance traps.
  3. don't wake up operators. bring reliability techniques from academia into real-world practice.
  4. don't use so much electricity. our data structures should play to modern hardware's strengths.

plans

  • MVCC, transactions, merge operators and snapshots provided via a higher-level Db versioned-key interface
  • beat LSM trees for reads and traditional B+ trees for writes
  • forward-compatible binary format
  • auto-tuning of internal parameters to squeeze out every drop of performance with zero operator effort
  • bindings for other languages

warnings

  • quite young, there are lots of tests but don't bet a billion dollar business on it yet!
  • the C API is likely to change rapidly
  • the on-disk format is going to change in non-forward compatible ways before the 1.0.0 release! after that, we will always support forward migrations.
  • has not yet received much attention for performance tuning, it has an extremely high theoretical performance but there is a bit of tuning to get there. currently only around 200k operations per second with mixed workloads, and 7 million/s for read-only workloads on tiny keys. this will be improving dramatically soon!

contribution welcome!

want to help advance the state of the art in open source embedded databases? check out CONTRIBUTING.md!

architecture

lock-free tree on a lock-free pagecache on a lock-free log. the pagecache scatters partial page fragments across the log, rather than rewriting entire pages at a time as B+ trees for spinning disks historically have. on page reads, we concurrently scatter-gather reads across the log to materialize the page from its fragments.

the system is largely inspired by the Deuteronomy architecture, and aims to implement the best features from RocksDB as well.

References

sled's People

Contributors

spacejam avatar pmuens avatar dotcypress avatar matthiasn avatar

Watchers

James Cloos avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.