What is the intended Debug escape behavior for the BStrs that contain the Unicode replacement character? about bstr HOT 4 CLOSED

The CharIndices iterator is also used by:

• ByteSlice::to_lowercase_into
• ByteSlice::to_uppercase_into

They are not impacted by this bug, but I think adding a comment like this would be a good idea:

            // This branch coaleces invalid UTF-8 bytes and legitimate
            // occurrences of the Unicode replacement character. This is
            // acceptable since the replacement character does not lowercase
            // into a different character
            if ch == '\u{FFFD}' {

from bstr.

The current implementation of Debug will always output the replacement character as three byte escapes. Is this intended?

I would say probably not. It's a good point. I think something like your approach would work well: check if the bytes correspond to the UTF-8 encoding of the replacement codepoint, and if so, escape it like any other codepoint. Otherwise, right the raw bytes.

from bstr.

As an aside, I now know that \uFFFD is the replacement character after diving through the bstr source, but there is a core::char::REPLACEMENT_CHARACTER constant that I think would be nicer to use instead. I'm not sure if using it has MSRV implications.

from bstr.

@lopopolo Yeah, using the constant is probably the better thing to do. If you work with Unicode and UTF-8 as much as I do, then \uFFFD just becomes as natural as breathing! Hah. But yeah, a named constant would be clearer. It would be fine to use, since that constant was introduced in Rust 1.9. bstr's MSRV is much higher than that.

from bstr.