cabo / cbor-diag Goto Github PK

Hello, thank you for your tools !

I've tried a few way to run the scripts but with no success.

Can I please ask you for an command-line example how to use the tools?

I want to use cbor2diag.rb, but I run it, i give the string, nothing happens.

I've never used ruby, maybe that's why is so hard.

Thank you for your time.
antoni

$ pretty2diag.rb
.../3.0.0/gems/cbor-diag-0.7.6/lib/cbor-diagnostic.rb:73:in `to_json': source sequence is illegal/malformed utf-8 (JSON::GeneratorError)
        from .../3.0.0/gems/cbor-diag-0.7.6/lib/cbor-diagnostic.rb:73:in `cbor_diagnostic'
        from .../3.0.0/gems/cbor-diag-0.7.6/lib/cbor-diagnostic.rb:83:in `block in cbor_diagnostic'
        from .../3.0.0/gems/cbor-diag-0.7.6/lib/cbor-diagnostic.rb:83:in `map'
        from .../3.0.0/gems/cbor-diag-0.7.6/lib/cbor-diagnostic.rb:83:in `cbor_diagnostic'
        from .../3.0.0/gems/cbor-diag-0.7.6/lib/cbor-diagnostic-helper.rb:32:in `cbor_diagnostic_output'
        from .../3.0.0/gems/cbor-diag-0.7.6/bin/pretty2diag.rb:17:in `<top (required)>'
        from .../3.0.0/bin/pretty2diag.rb:25:in `load'
        from .../3.0.0/bin/pretty2diag.rb:25:in `<main>'

Originally reported by @Cody119 at cbor/cbor.github.io#83

$ echo '"\u00f6"' | diag2pretty.rb
62 # text(2)
c3b6 # "\xC3\xB6"
$ echo '"ö"' | diag2pretty.rb
62 # text(2)
c3b6 # "\xC3\xB6"
$ echo '"ö\u00f6"' | diag2pretty.rb
/Volumes/nar/Users/cabo-rescue/lib/ruby/gems/3.0.0/gems/cbor-diag-0.7.1/lib/cbor-diag-parser.rb:1042:in `join': incompatible character encodings: UTF-8 and ASCII-8BIT (Encoding::CompatibilityError)

(Kudos to @cyberphone for reporting this.)

(Comments work in h'...' byte strings.)

cbor-pure (and cbor.me) could use a mode that decodes a leading data item, ignoring trailing additional bytes.

(Reporter: Malte Kuhn)

On cbor.me, "🍎" is correctly converted into

64          # text(4)
   f09f8d8e # "\xF0\x9F\x8D\x8E"

(well, the comment could possibly be improved), but on the way back, this turns into "\u{1F34E}", which is not valid CBOR diagnostic notation. This bug does not occur in a more conventionally configured environment; the code appears to be a dependent on Encoding.default_external to be Encoding::UTF_8.

In the CBOR playground at [1], it would be useful to have one more selectable checkbox on the right side.

If that is selected, the right frame would show only the binary serialization of the CBOR item/sequence provided on the left, while including no comments, no spaces and no line breaks.

[1] http://cbor.me/

A more friendly result would be desireable when feeding cborseq2diag with the contents of CBOR encoded flash.
The flash's erase values is 0xff, which is I think, why the CBOR stop has that value.
This means that a program can continue adding to the flash (extending the map, or array easily) without erasing first.

I'm testing this, and with my input of:

obiwan-[RIOT/tests/driver_cfg_page](2.6.6) mcr 10743 %hexdump -C CONFIG.bin
00000000  d9 d9 f8 da 52 49 4f 54  43 42 4f 52 00 19 c6 3b  |....RIOTCBOR...;|
00000010  bf ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff  |................|
00000020  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff  |................|
*

I get:

55800(1380536148(h'424F52')), 0, 50747, {}, Traceback (most recent call last):
8: from /home/mcr/.rvm/gems/ruby-2.6.6/bin/ruby_executable_hooks:24:in <main>' 7: from /home/mcr/.rvm/gems/ruby-2.6.6/bin/ruby_executable_hooks:24:in eval'
6: from /home/mcr/.rvm/gems/ruby-2.6.6/bin/cborseq2diag.rb:23:in <main>' 5: from /home/mcr/.rvm/gems/ruby-2.6.6/bin/cborseq2diag.rb:23:in load'
4: from /cache/mcr/.rvm/gems/ruby-2.6.6/gems/cbor-diag-0.6.1/bin/cborseq2diag.rb:16:in <top (required)>' 3: from /cache/mcr/.rvm/gems/ruby-2.6.6/gems/cbor-diag-0.6.1/lib/cbor-pure.rb:91:in decode_with_rest'
2: from /cache/mcr/.rvm/gems/ruby-2.6.6/gems/cbor-diag-0.6.1/lib/cbor-pure.rb:315:in decode_item_with_rest' 1: from /cache/mcr/.rvm/gems/ruby-2.6.6/gems/cbor-diag-0.6.1/lib/cbor-pure.rb:273:in decode_item'
/cache/mcr/.rvm/gems/ruby-2.6.6/gems/cbor-diag-0.6.1/lib/cbor-pure.rb:258:in `decode_item_streaming': break stop code outside indefinite length item (RuntimeError)
[3] 21649 exit 1 ./maketest0

And I'd just like something nicer.
Maybe I should write some ruby and catch this error and print my own thing?
I don't think that this is the same as #10, unless I just don't understand that issue.

It seems there are no way to keep information on indefinite strings/array/maps etc.

$ echo "{_ 3:4}" | diag2cbor.rb | cbor2diag.rb
{3: 4}

The definition in Section G.4 of RFC8610 allows whitespace-separated tstr and bstr fragments to be treated as one whole concatenated value. When I attempt to use this with the tool diag2pretty I get an error shown below. This also happens if the whitespace includes newlines, which is what I was originally trying to do.

*** can't parse "test" "this"
*** Expected one of [ \t\n\r], "/" at line 1, column 8 (byte 8) after 

The CBOR
5f 5f 41 00 ff ff
decodes to
(_ h'00')

It should error out per this text:

If any item between the indefinite-length string indicator
(0b010_11111 or 0b011_11111) and the "break" stop code is not a
definite-length string item of the same major type, the string is not
well-formed.

My decoder had an issue with this too.

These go well with the binary features of CBOR.

Hi, I came across the following inconsistency:

cbor2diag ignores width of floating-point numbers and thus is not bijective, as one would expect.

The RFC specifies that the width should be encoded using _n suffix, but it's not, therefore the information is lost, as demonstrated by the following example:

perl -e' print "\xfb\x00\x00\x00\x00\x00\x00\x00\x00"' | cbor2diag.rb | diag2cbor.rb | hexdump

prints

0000000 f9 00 00                                       
0000003

instead of \xfb\x00\x00\x00\x00\x00\x00\x00\x00.

Moreover, the parser doesn't seem to support parsing the _ width suffixes at all:

pavel@dt3:~|⇒  echo 1.0_2 | diag2cbor.rb
*** can't parse 1.0_2
*** Expected one of [0-9], [Ee], [ \t\n\r], "/" at line 1, column 4 (byte 4) after 

This in general goes back to the design of Diagnostic Notation which doesn't specify the exact rules for encoding 'logical' values to CBOR items. The same problem occurs with e.g. lengths of all items, where this implementation also chooses the 'canonical' encoding over preserving the full information:

pavel@dt3:~|⇒  perl -e' print "\x98\x00"' | cbor2diag.rb | diag2cbor.rb | hexdump  
0000000 80                                             
0000001

If these are the intended semantics of DN, then the standard should be amended with unambiguous specification of 'Canonical' CBOR, which would render section 6.1 unnecessary.

Otherwise the implementation should be changed to at least allow parsing all valid DN inputs. In this case, cbor2diag 'composed' with diag2cbor can never be an identity unless DN is extended to allow specifying width in all constructs where multiple choices produce valid and logically equivalent item.

can cbor2diag be educated to just continue decoding if there are more bytes?
that would let CBOR sequences "just work"

What is the intended open-source license for this software?

I would like to use it, but I could not find any indication in the README or in the source code.

$ echo -n '\x7f\xff' | cbor2diag.rb
(_ )
$ echo -n '\x5f\xff' | cbor2diag.rb
(_ )

This should output ""_ or ''_, because the (_ ) does not contain the information about the major type any more.

{{{
dooku-projects/pandora/fountain mcr 10006 %cbor2diag.rb tmp/ff1.cbor
/usr/local/rvm/gems/ruby-2.3.0/gems/cbor-diag-0.2.5/lib/cbor-pure.rb:193:in `take': Out of bytes to decode: 290 (CBOR::OutOfBytesError)
}}}

So something has truncated my CBOR, pretty sad for me, and I'm debugging it.
I wonder if there is someway that cbor2diag could show me what it does have, and at which point the shortage of data has occured?

Currently the diag2pretty tool seems to treat whitespace within hex-encoded bytestrings differently than for base64-encoded bytestrings. An example of bad behavior is below. It seems that the where the space is in the base64 sequence doesn't matter. Any space causes this issue.

Section G.1 of RFC 8610 indicates that all whitespace is ingored. Its examples uses hex encoding but my interpretation it it should also apply to base64 encoding.

[STRIP ~]$ echo "h'6865  6C6C 6F'" | diag2pretty.rb
45            # bytes(5)
   68656c6c6f # "hello"
[STRIP ~]$ echo "b64'aGVsbG8='" | diag2pretty.rb
45            # bytes(5)
   68656c6c6f # "hello"
[STRIP ~]$ echo "b64'aGVs bG8='" | diag2pretty.rb
*** can't parse b64'aGVs bG8='
*** Expected one of [0-9a-zA-Z_\-+/], [=], "'" at line 1, column 9 (byte 9) after b64'aGVs