Code Monkey home page Code Monkey logo

dtc's Introduction

DTC

DTC - Distributed Table Cache

ubuntu&gcc-4.9

Overview

DTC is a high performance Distributed Table Cache system designed by JD.com that offering hotspot data cache for databases in order to reduce pressure of database and improve QPS.

The DTC system consists of the following components:

  • Agent - Provides key consistent hash routing in order to reduce connections and improve performance.
  • Dtcd - Provides hot data caching service.
  • Connector - Provides connection and communication between cache and persistent storage database such as MYSQL.

Feature

  • Database Protection
    • protection for null node, prevent cache breakdown.
    • provide long-term data caching, and prevent cache penetration.
    • data source thread available, protect the database with a limited number of connections.
    • Estimated timeout policy to reduce invalid database requests.
  • Data consistency
    • write-through policy, ensure cache and database data consistent.
    • barrier policy to prevent update requests lost while concurrcy.
  • Performance
    • integrated memroy allocation policy to avoid frequent system calls.
    • I/O multiplexing to handle concurrcy requests.
    • multiple data structure models to improve memory performance.
  • Scalability
    • cache node expands horizontally to enhance cache capacity.
    • cache node expands vertically, supports slave reading, and solve the bottleneck of hot keys.
    • provide sharding, supports for persistent storage scalable.

Performance

  • DTC can process 90,000 QPS of query requests at single-core cpu & single dtc instance.
  • DTC can provide above 3,000,000 QPS query capability with above 99.9% hit rate and less than 200 μs response time in actual distributed scenarios.
  • Layered Storage is able to provide about 1,000 QPS write capability with above 99.9% per single instance.

How to Build

DTC provides docker images for quick start:

  • Start server docker:
docker pull dtc8/server:latest
docker run -i -t --name dtc-server -p 127.0.0.1:20015:20015 dtc8/server:latest

Depending on 3rd-party sql parsing engine hsql . For more compile information, click Building.
Trying a demo, visit QuickStart.

License

JD.com © Copyright 2023 JD.com, Inc.
Apache 2.0. Visit LICENSE for more details.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.