Code Monkey home page Code Monkey logo

hdrhistogram-swift's Introduction

codecov Address sanitizer Thread sanitizer

Histogram

Histogram is a port of Gil Tene's High Dynamic Range (HDR) Histogram to native Swift. It provides recording and analyzing of sampled data value counts across a large, configurable value range with configurable precision within the range. The resulting "HDR" histogram allows for fast and accurate analysis of the extreme ranges of data with non-normal distributions, like latency.

Histogram supports the recording and analyzing of sampled data value counts across a configurable integer value range with configurable value precision within the range. Value precision is expressed as the number of significant digits in the value recording, and provides control over value quantization behavior across the value range and the subsequent value resolution at any given level.

For example, a Histogram could be configured to track the counts of observed integer values between 0 and 3,600,000,000 while maintaining a value precision of 3 significant digits across that range. Value quantization within the range will thus be no larger than 1/1,000th (or 0.1%) of any value.

This example Histogram could be used to track and analyze the counts of observed response times ranging between 1 microsecond and 1 hour in magnitude, while maintaining a value resolution of 1 microsecond up to 1 millisecond, a resolution of 1 millisecond (or better) up to one second, and a resolution of 1 second (or better) up to 1,000 seconds. At it's maximum tracked value (1 hour), it would still maintain a resolution of 3.6 seconds (or better).

Histogram is designed for recording histograms of value measurements in latency and performance sensitive applications.

Measurements show value recording times as low as 3-4 nanoseconds on Apple Silicon CPUs (M1).

The Histogram maintains a fixed cost in both space and time. A Histogram's memory footprint is constant, with no allocation operations involved in recording data values or in iterating through them.

The memory footprint is fixed regardless of the number of data value samples recorded, and depends solely on the dynamic range and precision chosen. The amount of work involved in recording a sample is constant, and directly computes storage index locations such that no iteration or searching is ever involved in recording data values.

This port contains a subset of the functionality supported by the Java implementation.

The current supported features are:

  • Generic histogram class parametrized by type used for bucket count
  • All iterator types (all values, recorded, percentiles, linear, logarithmic)

Users are encouraged to read the documentation from the original Java implementation, as most of the concepts translate directly to the Swift port. Additional Thanks to the maintainers of the Rust port for a nice introduction to the package that we've largely borrowed.

Adding dependencies

To add to your project:

dependencies: [
    .package(url: "https://github.com/HdrHistogram/hdrhistogram-swift", .upToNextMajor(from: "0.1.0"))
]

and then add the dependency to your target, e.g.:

.executableTarget(
  name: "MyExecutableTarget",
  dependencies: [
  .product(name: "Histogram", package: "hdrhistogram-swift")
]),

Usage

The Histogram API follows that of the original HdrHistogram Java implementation, with some modifications to make its use more idiomatic in Swift.

Documentation for the classes and API are hosted by the SwiftPackageIndex

Simple example

(see implementation in Sources/HistogramExample)

import Histogram

let maxValue: UInt64 = 3_600_000_000 // e.g. for 1 hr in usec units

var histogram = Histogram<UInt64>(highestTrackableValue: maxValue, numberOfSignificantValueDigits: .three)

// record some random values
for _ in 1...100 {
    histogram.record(UInt64.random(in: 10...1000))
}

// record value n times
histogram.record(UInt64.random(in: 50...200), count: 10)

// record value with correction for co-ordinated omission
histogram.recordCorrectedValue(1_000, expectedInterval: 100)

// iterate using percentile iterator
for pv in histogram.percentiles(ticksPerHalfDistance: 1) {
    print("Percentile: \(pv.percentile), Value: \(pv.value)")
}

print(String(repeating: "-", count: 80))

// print values for interesting percentiles
let percentiles = [ 0.0, 50.0, 80.0, 95.0, 99.0, 99.9, 99.99, 99.999, 100.0 ]
for p in percentiles {
    print("Percentile: \(p), Value: \(histogram.valueAtPercentile(p))")
}

print(String(repeating: "-", count: 80))

// general stats
print("min: \(histogram.min)")
print("max: \(histogram.max)")
print("mean: \(histogram.mean)")
print("stddev: \(histogram.stdDeviation)")

print("\(histogram)")

hdrhistogram-swift's People

Contributors

dimlio avatar finestructure avatar github-actions[bot] avatar hassila avatar heckj avatar ordo-ci avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

hdrhistogram-swift's Issues

feature request: add Hashable conformance to Benchmark.IterationValue

I was working with the internal values returned from Histogram, and it ended up being slightly awkward - specifically not easily being able to extend an internal type on a specialized version. Or... maybe I just don't know how to do that correctly, but in either case, I wanted to ask: Would there be any concern about making Benchmark.IterationValue conform to Hashable?

It's internally conforming to Equatable already, but it would be extra lovely if I could safely drop it into a dictionary. At the moment, I'm working around this by mapping all the values into my own replica of that type, except that includes compiler-derived Hashable conformance.

I'd be happy to submit a PR to make the change.

chore: Align log format with upstream implementation

Should kill the dependence on TextTable as part of that (sorry, mea culpa) - will also make the package more palatable to depend on as there will be fewer dependencies (should probably remove SPI manifest checker bundle too).

Also, we seem to have a truncating error in the value output?

If looking at the upstream sample file

    Value   Percentile   TotalCount 1/(1-Percentile)

    0.016     0.000000            1         1.00
    4.455     0.100000       109166         1.11
   10.367     0.200000       218441         1.25
   14.383     0.300000       327522         1.43
   19.471     0.400000       436590         1.67
   30.447     0.500000       545740         2.00
   39.519     0.550000       600381         2.22
   52.191     0.600000       654888         2.50
   68.223     0.650000       709558         2.86
   87.807     0.700000       763927         3.33

The value seems to be interpolated for the percentiles? We always get '000'.

        Value     Percentile TotalCount 1/(1-Percentile)
------------ -------------- ---------- ----------------
      11.000 0.008333333333          1             1.01
      56.000 0.150000000000         18             1.18
     109.000 0.200000000000         24             1.25
     213.000 0.300000000000         36             1.43
     305.000 0.400000000000         48             1.67
     441.000 0.500000000000         60             2.00
     471.000 0.550000000000         66             2.22
     528.000 0.600000000000         72             2.50
     548.000 0.650000000000         78             2.86
     601.000 0.700000000000         84             3.33
     700.000 0.750000000000         90             4.00

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.