Backwards compatibility tests. about c-toxcore HOT 3 OPEN

It shouldn't try to connect to wild nodes. Instead, it should build and run a node from N time ago and test against that.

from c-toxcore.

There are two parts to this, one easy part and one more involved part:

1. Easy: fetch a git commit from the support horizon (e.g. 1 month ago) and replace the test suite from the to-be-tested clone (i.e. the one cloned by Travis CI) with the old one. This validates that we're source-compatible with the version of a month ago (including internal APIs, but perhaps we could just replace the tox* test files, not the internal unit tests).
2. Harder: fetch a git commit from the support horizon and build a test program with it. Build the same test program with the current code and run both of them. This test program is yet to be written, and should run some simple tests like adding each other and sending messages.

from c-toxcore.

2 is the reason I opened this issue. (Because you're right, 1 would be near trivial)

Perhaps we should consider caching the tests now, with version numbers, then scan the version numbers, and try to get them to interact.

That wouldn't be nearly as hard, and would be a better test of what would happen in the real world. I.e. it might be using an older version of libsodium which would be what the previous version of clients would use.

We should also consider limiting it to major or minor version changes, only. As they should be the only ones affected.

from c-toxcore.