Adding more Rune unit tests.

ustring design is amazing because it allows you to bring your own string. That said, the default stack should at least contain an implementation that does allocation-less string concatenation by holding a chain of ustrings internally.

class ChainUString : ustring {
    ustring this;
    ustring next;
}

1. IDisposable implementation is hidden to ustring API users, thus IntPtrUString will always end up being finalized, unless people explicitly Dispose the string.

2. IntPtrUString design assumes that the parent string outlives the child string, failing code would look like:

IntPtr mem = grab_native_string(out int count);
var parent = ustring.Make (mem, count, str => release_native_string (str));
var child = parent [0, 5]; // Let's assume these values are correct
// somewhere here GC happens, parent is reclaimed
child.ToString(); // accessing bad memory

Maybe a SafeHandle mechanism is better here than an opaque pointer? The wrappers can get reclaimed while the SafeHandle still lives, so that's 👍.

That would make the IntPtrUString slightly larger - Marshal.SizeOf(SafeHandle) + sizeof(int) for an offset.

This patch and this patch seem to have messed up column widths. E.g. the Unicode character "𝔹" (U+1D539) is counted as double-width. Before these patches, it did not fall in any of the double-width ranges. However, after these patches, gethexaformat conflates U+1D539 with U+D539 (픹), which is considered as a wide character.

I don't understand what the purpose of gethexaformat is -- it seems to just be performing the bitwise operation x & 0xFFFF, but using text parsing? Why are only the least significant 16 bits considered when determining whether a character is wide? This seems like it would cause many characters beside my example to have the wrong width...

From my understanding you have to list each platform and what it actually need: https://github.com/jamesmontemagno/mvvm-helpers/blob/master/MvvmHelpers.nuspec#L18-L46

Is what I do. Else installing your library does something like this:

Resulting in:

This was just into a xamarin android app. Perhaps I am doing something wrong or misunderstanding the changes they made.

A tool to build ustrings.

Currently InvalidIndex does not treat ReplacementChar as an invalid rune, so we can enable the tests that scan for it.

Wonder if there is any interest in being able to write a literal ustring that may contain embedded non-UTF16 friendly bytes? I was thinking that if you provide a parser for a FormattableString, you can use some Format Item format code to intermix literal character strings with raw bytes.

Something like:

var us = ustring.Make($"Sample {0xff::} Embedded");

This would yield a ustring (us) with the content:

Might need a new subclass, that would ensure that we never deallocate the parent if the child is still alive.

Test release

Hello,

Can someone explain me the difference between System.Rune (NStack) and System.Text.Rune.

Thanks

Just like terminal.gui

For code like this:

ustring.Make (new List () {.....})

Nbsp (160U) is printable character (space) , but Rune.ColumnWidth returns -1.

https://github.com/migueldeicaza/NStack/blob/8481cc1eb5b06c0e173d95351bcdd23b8235e54d/NStack/unicode/Rune.ColumnWidth.cs#L90-L91

I think we need this small change to fix it:

if (rune1 < 32U || rune1 >= (uint) sbyte.MaxValue && rune1 < 160U)
    return -1

Thanks,
Peter

Currently there is no way of getting a range of runes 3 to 10 and no API to find how those runes map to the offsets inside the byte string without manual work.

Is there any way to obtain the number of columns occupied by a Rune other than by the range of Unicode characters as currently done by the ColumnWidth method?

Hide IComparable implementation via explicit implementation and implement IComparable so we have compile-time failure not runtime failure.

For ustring.

This is an issue I filed in gui.cs a while ago.
I took a look in NStack and I think I found the reason, so I'll open this issue to be able to fix it with a PR.

Issue

I am currently using some unicode signs like in gui.cs: ≡ ⌂ ↑ ↓ …

While rendering them in a ListView or TextField, I got a IndexOutOfRangeException. In Labels, everything is okay.

   at System.Rune.bisearch(UInt32 rune, UInt32[,] table, Int32 max)
   at System.Rune.ColumnWidth(Rune rune)
   at Terminal.Gui.ListView.ListWrapper.RenderUstr(ustring ustr, Int32 col, Int32 line, Int32 width)
   at Terminal.Gui.ListView.ListWrapper.Render(Boolean marked, Int32 item, Int32 col, Int32 line, Int32 width)
   at Terminal.Gui.ListView.Redraw(Rect region)
   at Terminal.Gui.View.Redraw(Rect region)
   at Terminal.Gui.Window.Redraw(Rect bounds)
   at Terminal.Gui.View.Redraw(Rect region)
   at Terminal.Gui.Application.Redraw(View view)
   at Terminal.Gui.Application.Begin(Toplevel toplevel)
   at Terminal.Gui.Application.Run(Toplevel view)
   at grrui.Program.Main(String[] args) in D:\Develop\GitHub\RepoZ\grrui\Program.cs:line 43

It turns out that "\u2261" for example, returns Rune 8801. Rune.ColumnWidth(8801) throws.

Scenario:

				SetText (text [0, point]);

If point = 0, this makes the operation ont set it to the empty string, but the whole text

To clean up the current API, so that not everything is surfaced at once.

Something like Unicode.Category.Letter for example, and the same for Scripts.

Try pasting "漢語" on the input line, that takes up 4 characters, but we are not aware of this.

Should be possible.

https://tehgm.net/blog/docfx-github-actions/

Need to surface this one, based on Rune.ColumnWidth, to easily compute the console-visible length of a ustring.

Perhaps ustring should be "u8string" to indicate the utf8 nature of it, and we introduce a "ustring" that is rune-indexable, which is a List

Currently, there is:
ustring Make (IntPtr block, int size, Action<ustring, IntPtr> releaseFunc = null)

The option to avoid allocating here is to make the caller cache the lambda at callsite, this can lead to lots of static delegates being declared, and possibly captures, etc. I'm not sure there is a better way to handle this though, so the users will have to deal with the delegate allocations.

It's not necessary to know the ustring that's being released. I don't have enough information to understand why we also pass in the ustring that's being reclaimed. Done this way, the user can still shoot themselves in the foot, since they don't know the implementation details of IntPtrUString, thus if they decide to use the string after free-ing the IntPtr, it'll lead to invalid memory accesses.

Maybe trim down the Action parameters to IntPtr? That way, you can use built-in methods like Marshal.FreeHGlobal and use it as a method group at call-site?