Segment writes blocked when taking a snapshot. about raft HOT 3 CLOSED

One thing that would be nice to have is some form of continuous benchmarking, so we can track changes in performance over time and have some feedback about improvements or deterioration.

There are helpers like:

https://github.com/marketplace/actions/continuous-benchmark

that could be useful to that extent.

from raft.

We have this https://github.com/canonical/go-dqlite/actions/workflows/daily-benchmark.yml , which is a bit rudimentary and we don't assert results as the results can vary pretty wildly from run to run. We think this is because we don't control the runner on which the tests run and don't know what else is running on the physical system while we use it. Ideally we want to run benchmarks on the same physical machine every day, so that we get comparable results. Cole and me have some dedicated hardware to do that, but we still need to set the benchmarking up.

Will check out the link, thanks.

from raft.

We have this https://github.com/canonical/go-dqlite/actions/workflows/daily-benchmark.yml , which is a bit rudimentary

Ah I didn't know about that. Seems a good starting point.

and we don't assert results as the results can vary pretty wildly from run to run.

Right, using Github Actions to run the benchmarks is of course sub-optimal, but given enough runs it should give an idea of the trend (improvement or regression). It couldn't reliably be used for assertions.

We think this is because we don't control the runner on which the tests run and don't know what else is running on the physical system while we use it.

Most probably that's the reason.

Ideally we want to run benchmarks on the same physical machine every day, so that we get comparable results. Cole and me have some dedicated hardware to do that, but we still need to set the benchmarking up.

That'd be better than using Github Actions runners indeed. I believe it's possible to tell Github Actions to use an external runner.

Will check out the link, thanks.

I didn't look at it in detail, but I'd expect that it should be possible to combine it with the benchmark code that we already run in the go-dqlite workflow you linked and get graphs out of it.

from raft.