Invalid utf-8 in EDID decode Extra descriptor about ddcutil HOT 5 OPEN

I have modified the EDID report output to replace any character having value 127 with the string "", where HH is the hex value of the character.

from ddcutil.

I should add that if edid text fields are always suppose to decode to ASCII and not UTF-8, perhaps any invalid ASCII characters could just be filtered out or replaced.

from ddcutil.

As per the VESA ENHANCED EXTENDED DISPLAY IDENTIFICATION DATA STANDARD (Defines EDID Structure Version 1, Revision 4) Release A, Revision 2 September 25, 2006 the contents of Alphanumeric Data String Descriptor Definition (tag #FE), which ddcutil refers to as the Extra Descriptor, consists of ASCII characters. This is also documented as code page 437 in the EDID Display Descriptors section of the Wikipedia EDID page.

This descriptor is reported in the EDID summary for purely informational purposes. Usually it is blank. If we are occasionally seeing non-ASCII characters in the string, the simplest solution woud be to simply not report the field.

from ddcutil.

Further, slightly more awake comments. What the EDID spec considers as ASCII is defined in its Appendix E - ASCII Reference Tables. Table ASCII-II, which defines code points x80..xFF, indeed appears to be identical to Microsoft Code Page 437, as stated in the Wikipedia spec. In that table, x80 is defined as C-Cedilla. The string in question is not UTF-8 encoded. x80 would only appear as the 2nd, 3rd or 4th byte of a multi-byte character encoding. In ISO 8859-1, x80 is undefined and in a range reserved for control characters.

In any event, the x80 is probably just someone's sloppiness or an attempt at a clever encoding that only their software understood. ddcutil could emit a character sequence such as "" for characters in the range x80..xFF, but as I noted for our purposes the Extra Descriptor is just a curiosity and not worth a lot off effort. On the other hand, if a character in the range x80..xFF appeared in a model name or serial number that would be problematic.

As it happens, there was a time I swam in the character encoding swamp. I attended several of the early Unicode Technical Committee meetings on behalf of the Research Libraries Group for which I did work at the time. Their cross-library database allowed for cataloging materials using most of the world's scripts (Latin, Cerillic, Hebrew, Arabic, Chinese, Japanese, and Korean, IIRC). The notable exception was Devanagari.

from ddcutil.

Because I've dealt with this in vdu_controls, I don't have any issues with what the ddcutil command produces.

For ddcutil-dbus-server I'm currently filtering text fields to remove bad UTF-8, but I will simplify that to filter non-printable ASCII.

As it happens, there was a time I swam in the character encoding swamp. I attended several of the early Unicode Technical Committee meetings on behalf of the Research Libraries Group for which I did work at the time. Their cross-library database allowed for cataloging materials using most of the world's scripts (Latin, Cerillic, Hebrew, Arabic, Chinese, Japanese, and Korean, IIRC). The notable exception was Devanagari.

Down here in NZ, many of my clients had not even progressed beyond uppercase EBCDIC/ASCII.

from ddcutil.