Performance comparison between s3fs and riofs about riofs HOT 12 CLOSED

Definitely interested in this too.

from riofs.

I did not know about riofs and ended up writing goofys, partially motivated by poor performance of s3fs. I have some rudimentary benchmark against s3fs using https://github.com/kahing/goofys/blob/master/bench.sh which I think covers most common operations. I try hard to measure uncached performance because that's the most interesting use case for me.

btw glancing at the code it looks like flush() in riofs does nothing? That's cheating ;-)

from riofs.

flush() has no semantics on AWS S3, hence we left it intentionally unimplemented (leaky abstraction). Can you explain why you think this should be implemented?

from riofs.

apologizes about the lack of flush(), like I commented on HN I didn't go through the code enough and that was the first thing I saw. I hadn't heard of riofs previously so I just assumed it wasn't in a stage that's ready yet. Sorry!

from riofs.

No worries, riofs is in production use internally at Skoobe for over 18 months now and I also heard of some others using and liking it, but we haven't spend any energy making it more popular.

I would be very interested in seeing performance numbers such that we learn more about which underlying strategies really work. Unfortunately, I am not with Skoobe anymore and don't have enough cycles to work on this right now...

Getting caching right took us most time, but I believe the non-caching part is also pretty neat.

from riofs.

I am trying to run some benchmarks, but numbers seem too good to be true. On closer examinations looks like rfuse_release() does not wait for the S3 response before returning. It's probably okay for most use cases but that makes it hard to do apples and apples comparisons.

from riofs.

Just want to give you a hint how did I to test RioFS: start two instances of RioFS and mount the same bucket. Using some script create a file (or many files / folders) and place into the first mounted folder, then check how soon it appears in the second mounted folder. It should give you some ideas about the performance.

from riofs.

Here's the result. I didn't put any effort in optimizing the number of concurrent connections, so maybe that explains why the read/write perf isn't up to par. riofs config here

(*) indicates the number of outliers removed
(†) see the above caveat regarding release()

from riofs.

Nice work and interesting figures indeed, your critique regarding rfuse_release sounds valid, but the configuration is certainly not ideal for a benchmark (capped at max. 2/2/4 concurrent connections), I would increase it substantially for a production setup. Most likely the bottleneck hasn't been RioFS. Can you try again with x10 and x100 for all numbers?

from riofs.

I can but I don't want to, because then I would have to find the optimal configuration for s3fs as well. I challenge you guys to default to a reasonable value instead of having everyone independently figure that out.

from riofs.

Understood, you are probably highlighting an issue as we haven't done a good job at thinking from a user's perspective here.

from riofs.

This should be re-opened because it has not been resolved. It was accidentally closed because PR#137 used #6 in the title

from riofs.