Code Monkey home page Code Monkey logo

back-of-napkin's Introduction

back-of-napkin's People

Contributors

a-f-g-u-c avatar

Stargazers

 avatar

Watchers

 avatar

back-of-napkin's Issues

"Known Issue: Ensuring same dictionary" - suggestion

From your section containing:
"...A common issue with compression schemes involving pre-shared dictionaries is ensuring that all parties use the same version of the dictionary.

It is expected that, as the CoT ecosystem develops, new patterns will emerge, prompting further revisions of the shared dictionary. In that case, it is important to understand that the use of mismatched dictionaries will lead to data loss or corruption, therefore it is critical to ensure that all parties in the communication use the exact same version of the dictionary.

A mechanism for solving this issue is outside the scope of this study, and therefore remains an open problem for the application developer to solve..."

I'd propose reserving up to 2 bytes as a dictionary version number using a psuedo Huffman Tree at the beginning of the string (last dictionary bit must be zero). This would allow immediate dictionary mismatch detection, and allow more conservative use of future dictionary definition space, since you ony really need to add new entries to a dictionary, you don't need to depreciate entire dictionaries. A 16-bit space would give 16 dictionary versions (yes, limited) but would also not require the additional bit-length until later version of the dictionary are made available with unique dictionary entries from all previous versions - the first release would be "0", second "10", etc. You would thus only use a similar amount of additional code space in controlling versions as you would a true greedy Huffman Tree.

Being able to detect the dicitonary mismatch would allow a return query to the sender for the specific dictionary entry used for the recently sent message, further economizing on the necessary data in transit. As an example, if version 1 ("0") was already in a release, and it received a packet with a version 4 ("1110"), it would only need the version 4 dictionary contents until it receives another version in the wild. Version 2 and 3 dictionaries wouldn't need to be sent from the sender to the receiver. The request for a dictionary's contents should be a one-time thing, meaning that you gain effectively some temporal channel compression. The now updated receiver would have another dictionary to regex against, potentially improving the return-path compression.

Having the dictionary version and mismatch detection tightly coupled would allow for the oportunity of packing a dictionary request in the as-of-yet-decompressed packet's ACK to the sender.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.