Hi there 👋 First of all - thanks for all your great work, I appreciate it!
I just wanted to ask something I can't wrap my head around. It seems steno
is way way faster than regular fs
. But then I just saw it is internally using the fs
itself, and it got me wondering - well how is that possible?
And I'm actually thinking the benchmark is comparing apples to oranges. From benchmark.ts:
// To avoid race condition issues, we need to wait
// between write when using fs only
for (let i = 0; i < 1000; i++) {
await writeFile(fsFile, `${data}${i}`)
}
console.timeEnd(fsLabel)
console.time(stenoLabel)
// Steno can be run in parallel
await Promise.all(
[...Array(1000).keys()].map((_, i) => steno.write(`${data}${i}`)),
)
console.timeEnd(stenoLabel)
What I see here is that the first one actually does write into a file a thousand times. It waits for each write to finish and then runs the next one.
The second benchmark runs all writes in parallel (as stated). That's the first issue that makes me think comparison is not proper. But the second one is even more important and it comes from the lock
implementation (index.ts):
async write(data: string): Promise<void> {
return this.#locked ? this.#add(data) : this.#write(data)
}
The way this locking mechanism is implemented is that it would "save" (#add
) data for later usage, if a write is currently ongoing. When the first write is done, it will check if there's any next data to be written to the file, and if so - makes the call:
if (this.#nextData !== null) {
const nextData = this.#nextData
this.#nextData = null
await this.write(nextData)
}
Since #nextData
is a single property that gets overridden (as stated in the code), you have just one next data to write, and you make a second file write call. And that's the end of the chain of writes.
What I'm saying is that from what I can see, you compare a thousand writes to a file, to just two writes to a file. Which is of course way way slower 😃
In no way I'm pointing fingers here, I'm just trying to figure out if I've missed something, or it's just a wrong statement.
Thanks again, good luck!