API tests about secp256k1 HOT 4 OPEN

API tests should explicitly perform tests with variously unaligned input/output buffers.

(not likely to currently be a source of bugs except through compiler screwup, but if anyone starts going around adding SIMD code it would rapidly become a potential source of extremely hard to spot bugs).

from secp256k1.

From all the existing work, what comes probably closest is #744 .

This should be less "test" and more "mechanical verification that the interface we present to callers isn't unintentionally changed".

@gmaxwell Does fuzzing match this idea, or did you have something else in mind?

from secp256k1.

I would describe that test as the opposite of an api test: All it tests is the payloads, the usage of the API is hardcoded into the harness.

An API test does stuff like calls with a null context or a number of pubkeys = 0, it acts like callers that fail to check errors or which call functions out of order... the goal is to validate the contract implied by the header documentation and maybe a little beyond it.

from secp256k1.

It seems to me one of the first steps here is to separate the library code from the test code better. (Compile the library, then compile tests that link to that library from code in a separate directory)

Broadly I see:

• Conformance tests - Like OpenSSL to ensure that the cyrpto is correct, these are very good today
• Performance tests - ensure that the operations are performant (handed by bench*.c and friends), these exist
• Client tests - Build and sanity check that a consumer can accomplish it's task, needs work
• (NEW) API tests - Do all the things that a client shouldn't do and make sure its gracefully handled. Because of the nature of this it seems like the wise idea might be to use a language like python/ruby and FFI the API surface because it's easier to create a large matrix of tests this way. It also tests that FFI works correctly (as much as I hate FFI 😆)

from secp256k1.