Code Monkey home page Code Monkey logo

simple-log's Introduction

simple-log

简易C++异步日志(学习用)

license

Introduction

简易的C++异步日志,采用了日志缓冲区的方式提高效率,写入压力大时,缓冲区个数可自动扩展
通常情况工作时为双缓冲区
Init初始化时创建一个后台flush线程,用于写入log文件
性能测试,已进行了单线程测试,多线程测试
增加同步写入测试
增加多生产者线程写入测试

Analysis & Optimization

  • 一开始时,经测试,发现同步竟然比异步写入要快!非常奇怪,经研究后发现是因为fwrite函数已经起到了缓冲作用,先是写到缓冲区,所以两者都有缓冲,性能差距不大,如果换成write就应该明显了。
    而本程序异步写入在fwrite之前还多了一次memcpy到Buffer的操作,所以更慢。
    因此,同步写入和异步写入的fwrite之后增加flush操作刷盘,就可以实现异步更快于同步了。
  • 优化性能,减少localtime()调用次数,当秒数不变时只需调用gettimeofday()更新毫秒就可以了,性能提高60%
  • 优化性能,去除logline初始化,因为snprintf会覆盖logline,所以无需初始化,性能提高17%
  • 优化性能,复用同一秒内的年月日时分秒的字符串,减少snprintf()中太多参数格式化的开销,性能提高30%
  • 多线程测试时,性能下降,猜测是加锁的原因,待优化
  • 优化多线程性能,每个线程独立拥有写入缓冲区,减少锁开销(只有map索引和queue时有锁),性能提高150%
  • 对比了存储线程缓冲区的数据结构map和unordered_map,最终使用map,map效率高40%,可能是因为数据量少时map效率更好
  • map的锁应该可以改为读写锁,性能应该提升会很大,之后有时间再改吧 暂时先这样吧

Envoirment

  • CPU: AMD Ryzen Threadripper 2990WX 32-Core Processor
  • MEM: 128GB ddr4
  • STORE: 500GB NVME
  • OS: Ubuntu 16.04
  • kernel: 4.15.0-72-generic (# uname -a)
  • Complier: gcc version 5.4.0 20160609

Build and Run

  • 进入目录,执行: $ g++ -g -Wall -O2 -pthread -std=c++11 logtest.cpp logger.cpp -o logtest
  • 运行: $ ./logtest

Simple Performance Test

单线程:1百万条log写入,耗时453ms,220w logs/second
多线程:4线程各1百万条log写入,耗时1134ms,352w logs/second

Other Ref Blog

Ring Log

Enjoy!

simple-log's People

Contributors

chenshuaihao avatar

Stargazers

 avatar deltamaya avatar  avatar Pisces avatar  avatar  avatar Luosu avatar  avatar  avatar  avatar  avatar  avatar xky avatar  avatar  avatar Caliyang avatar netaudio avatar Ginger avatar 剑指offer avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.