goblin / rdbl-code Goto Github PK

I'm looking for a nice way to add redundancy to the output of rdbl_enc.

Ideally I'd like to add as little data as possible to allow for a couple of substitution, insertion or deletion errors. Processing of input in rdbl_dec would stop as soon as there's enough data to ensure the whole text is error-free (that is, rdbl_dec would read the text, the checksum, and then NOT read any extra redundancy if everything is error-free; if it isn't, read up to only as many extra words as are needed to recover from the errors).

To use an existing FEC library such as https://github.com/quiet/libfec, I guess I'd need to packetify the data (perhaps making each line a packet), and add a checksum and redundancy to each packet. That could work, but IMO would add quite a lot of overhead.

One simple solution that comes to mind is to simply add a 4-byte checksum, encoded separately and appended (or prepended) to the output of current rdbl_enc. If the checksum matches, great. If not, do a simple bruteforce scan checking for a single substitution error somewhere in the text (prioritizing by key locations i.e. it's common to swap "u" for "i" but uncommon to swap "u" for "x"), and see if the checksum works then. If it still doesn't, give up, report an error, and tell the user to look for the mistake herself.

https://arxiv.org/pdf/1602.06820v2.pdf might be useful.