This library designed for fast and efficient calculate basic statistical data of a big sequence of numbers, without loading all data into memory.
It directly calculate:
-
min
-
max
-
sum
-
mean
-
variance
-
standard deviation
And use t-digest algorithm to approximately calculate quantiles and build a chart.
T-digest is optional. Without t-digest, library computing statistic few times faster. As example six seconds for a one billion numbers.
$go test -bench . -benchmem goos: linux
goarch: amd64
pkg: github.com/lynxzp/seq_stats
cpu: Intel(R) Core(TM) i7-4790 CPU @ 3.60GHz
BenchmarkStat/input_size_1000-8 276945 3612 ns/op 0 B/op 0 allocs/op
BenchmarkStat/input_size_1000000-8 328 3700582 ns/op 0 B/op 0 allocs/op
BenchmarkStat/input_size_1000000000-8 1 3643973531 ns/op 0 B/op 0 allocs/op
PASS
ok github.com/lynxzp/seq_stats 6.370s
import "github.com/lynxzp/seq_stats"
const tDigestEnabled = true
s := seq_stats.New[int64](tDigestEnabled)
for i := 0; i < 1000000; i++ {
s.Add(int64(i))
}
fmt.Println(s.Min())
fmt.Println(s.Quantile(0.5))
fmt.Println(s)
writer := bytes.NewBuffer([]byte{})
const pointCount = 1000
_ = s.DrawPNG(writer, pointCount)
result:
0
499999.5
min: 0, max: 999999, avg: 499999.5, rms: 577349.8, stddev: 288675 sum: 4.999995e+11 amount: 1000000
0.1%: 999.5, 1%: 9999.5, 10%: 99999.5, 50%: 499999.5, 90%: 899999.5, 99%: 989999.5, 99.9%: 998999.5