Code Monkey home page Code Monkey logo

paraclickhouse's Introduction

ParaClickHouse

A very crude implementation of CUDA Heterogeneous ClickHouse Aggregate Functions.

概述

  • 本项目尝试将ClickHouse聚合函数的计算进行CUDA异构加速
  • 硬件:Intel CORE i5; GeForce RTX 3060
  • 操作系统:Ubuntu22.04.2
  • ClickHouse版本:ClickHouse 22.12.1.1
  • 编译器版本:LLVM/Clang 15.0.0
  • CMake版本:3.24.2
  • CUDA Driver/Runtime版本:12.1/12.1
  • 项目展示了在ClickHouse源码基础上作了改动的目录及文件。parallelization文件夹是新增的,其余改动都标记了"kai mod"注释或"ifdef ENABLE_CUDA"宏;

核心**

  • 多线程并发启动网格级核函数并行,通过异步流形成Cpy-Kernel-Cpy(CKC)三级流水。
  • 在ClickHouse解析器确定有Sum聚合查询时,预分配主机端内存空间/设备端内存空间/流,以达到复用效果。
  • 扩大了ClickHouse批处理大小,以掩盖PCI-e传输的额外开销。
  • 因为没有利用锁页内存,所以该异构算法受限于H2D传输。
  • 在Star Schema Benchmark的lineorder表上,聚合480,003,275行数据可以达到1.14~1.66倍提速。

限制

  • 只能作 “select sum(filed) from table;"操作。

本人C++/CUDA/CMake/大型项目的开发经验为零,项目阅览可以权当图一乐。

paraclickhouse's People

Contributors

yktian-x2b avatar

Stargazers

 avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.