cometkim / unicode-segmenter Goto Github PK

The original Rust library implements the DoubleEndedIterator trait, which, unlike JavaScript's standard iterator, allows for reverse traversal (next_back())

The original goal was to be a lightweight alternative to Intl.Segmenter, so I intentionally removed it. But as I can see some issues in the ecosystem like flmnt/graphemer#10, it is one of the most common requirements in scenarios such as implementing an editor. Because, it would be crazy to re-traverse the entire text forward to find just the previous cursor. To avoid that, the user would have to implement their own cursor history, which is inefficient.

However, that significantly inflates the bundle size, which the generator cannot represent.

I will improved the class version implementation and expose it as a separate bundle entry (e.g unicode-segmenter/advanced/double-ended/grapheme)

Fuzzing using Intl.Segmenter (which is based on ICU) was a neat idea, but directly utilizing the Unicode official break tests will make the package more trustworthy.

I expected the Intl.Segmenter to behave based on the provided locale parameter and some dictionary per it. However there seems to be no change in behavior depending on the locale, and only the basic algorithm specified in Unicode is implemented.

So there's no reason why I couldn't completely polyfill it. It still has a chance to be useful for Korean, unlike other CJK, it has strict word spacing.