Random IO throughput about goaio HOT 5 CLOSED

So I cobbled together implementation of of the randread tool with a new mode that uses the goaio library. It is available here: https://gist.github.com/traetox/0d9b482067780933f1df0856e3911904 The results match what I was seeing in my other use case, which is basically that the added overhead of mutexes, managing buffers, and added complexity of the aio syscall interface with the goaio package completely overwhelms any benefits from the asynchronous IO with lots of small reads. If you extend out the size of the reads (say 1MB) the additional overhead fades and it gets faster. Some rough numbers: 4K reads - roughly 5X slower than the goroutine based parallel read 1MB reads - about 20% faster than the goroutine based parallel read If you reworked the library to remove the mutexes and additional overhead and just managed it locally by hand it might get a little better. The caveats being that its not a really good one to one comaprison due to the way i implemented the test in randread. Traetox

I bring up the mutexes simply because they are showing up in the top 20 in the CPU profiles. They were not the majority of the CPU time spent. As an example, mode 2 spends 96% of its time in the Syscall6 call where mode 3 (AIO) spends only 79.21% of its time in Syscall indicating there are other issues. I also don't have a good method to identify if differences in the Syscall6 vs Syscall are causing additional overhead. It looks like the actual syscalls just take longer. The AIO interfaces in the kernel do a lot of work in condensing read requests and ordering them. So the small frequent out of order reads cause a lot of internal "stress" on those routines that overwhelm the benefits of letting the kernel do more work for you. The paradigm used in that benchmark just doesn't really fit the use case for the AIO library and I doubt it will be competitive in this use case. Traetox