We now have a Deno implementation, however it doesn't implement the full spec yet (as far as I'm available).
In order to have all implementations up-to-date someone should implement the full spec

Add a Scala 3 implementation with a focus on the JVM.

Early tests found that the Elixir implementation is incredibly slow and utilizes a lot of resources in comparison to the other implementations.

This is likely a programming issue and should be resolvable, because it can't be sooo slow.

When running the Dart implementation an Exception is raised on every connection. Please ignore this exception.

We should specify what an implementation is supposed to log and in what form.

I propose that we at least require an "Server started listening" log entry.
The client and the scripts should log more extensively.

Also, each log should be prepended with a context, e.g. [Client] Message.

Currently, the client panics whenever anything goes wrong at all.

Two problems that occur often are:

1. Server response is invalid
2. Server closes unexpectedly

In these cases (or others), the client should log that an error occurred and then exit gracefully with a non-zero exit code.

This error shows up 50 times or more when running the benchmark. Occurs both for Swift and Dart:

thread 'tokio-runtime-worker' panicked at 'called 'Result::unwrap()' on an 'Err' value: Os { code: 49, kind: AddrNotAvailable, message: "Can't assign requested address" }', src/benchmark.rs:47:53

Introduce a Zig based implementation.

Current Problem

Right now all actions required in the specification are IO bound tasks. The most compute intensive task is the parsing of the incoming command.

Solution proposal

Add some of the following actions, they do not require much developing time but also benchmark computation capability:

• GETKN key-pattern get the number of keys containing the given key pattern at least once
• GETVN value-pattern get the number of values containing the given value pattern at least once

To automate the benchmarking process, the client should not die just because the server is not yet online, it should rather wait until a connection can be established before beginning the benchmark process.

Introduce a JVM focused Kotlin implementation.

Right now there are two commands that initiate a simple protocol consisting of several steps that have to be followed by both the server and the client: UPLOAD and DOWNLOAD.

However it is not fully specified what the server should do in case it either gets no answer when it's expecting one (e.g. after an unexpected disconnect) or when it gets a wrong answer (e.g. when the client responds with neither OK nor ERROR in step 4 of the UPLOAD protocol, but with something else).

I suggest that the server should cancel an ongoing operation when there was a disconnect or when it gets an unexpected response from the client (for the UPLOAD command this means deleting the created file).
It could also be worth adding a timeout to the protocols, i.e. both the server and the client should only wait a certain amount of time (e.g. 2s) for a response before cancelling the protocol.

The DUMPINTERVAL command is missing concrete specification:

• What should be returned by the server after a DUMPINTERVAL command?

The Go implementation currently returns xxhxxmxxs, corresponding to the supplied interval. This should at least be changed to correspond to the specified duration format.

• Should the old interval be cleared and the new one immediately started?
  • There are multiple ways this could be handled:
    • Wait until the last interval runs out and schedule the next dump with the new interval
    • Cancel the last interval, create a dump immediately and schedule the next dump with the new interval
    • Cancel the last interval and schedule the next dump with the new interal

The go implementation currently uses the third approach (correct me if I'm wrong) and I think this should be the way to go.

Please change my name, see single line comment

Originally posted by @TecTrixer in #18 (review)

I'm already working on this.

Introduce a Nim implementation.

When running the Dart implementation an Exception is raised on every connection. Please ignore this exception.

Add a Kotlin implementation focused on the Kotlin Native backend.

There will be a new client, since the old one is very performant but very useless. Please tell me, what you want to see in the new client.

Add a Haskell implementation.

A Scala 2 implementation focused on the JVM.

Introduce an implementation written in server-side Dart.

With the new spec, we have to update all implementations

• #47
• #48
• #49
• #50
• #51
• #52
• #53
• #54
• #58
• #62
• #55

For better performance the C implementation should be adapted to employ POSIX fd's instead of the stdio.h FILEs.

It would be also very reasonable to improve performance through the implementation and drastically reduce the number of mallocs and frees.

Introduce a Scala 2 implementation focused on the Scala Native backend.

Since we now have a Dart implementation (#42) it should be updated as well.

Introduce a JavaScript implementation focused on running in Node.

We need some sort of script that runs all benchmarks automatically, generating the result files, which can then be used by the make_plots script to generate fancy graphic results.

Refer to the parent issue #46.

We need Dockerfiles for every implementation, and for the client.

Each Dockerfile should build the server executable and the container should expose the server at port 8080.

Reference Dockerfile in the Java implementation.

For better comparability, please also make sure that all runner containers run on alpine:latest and preferably use static linking.

• C
• Crystal
• Elixir
• Java
• Python
• Ruby
• Rust
• Swift
• Go
• Client

Currently, the specification demands that a SHA512/265 hash is returned after and UPLOAD command. However, this kind of hash is not widely supported, thus some implementations could suffer from this.

I propose changing this to a plain SHA512 hash, which is available in virtually every language.

We should add more language implementations.

I would love to see:

• #26
• #27
• #28
• #29
• #30
• #31
• #32
• #34
• #33
• #35
• #36
• #61

Introduce a JavaScript/TypeScript implementation focused on running in Deno.

Currently, the specification requires that a parsed duration shall match the following RegEx in total:

([0-9][0-9])h-([0-9][0-9])m-([0-9][0-9][0-9])s

This seems wrong, since normally seconds are only written using two digits.
Additionally, it allows second and minute values larger than 59.

Therefore, I suggest that the requirement is changed to this RegEx instead:

([0-9][0-9])h-([0-5][0-9])m-([0-5][0-9])s