Type/enter "ctrl del" -> Console message: "Unknown input type: deleteWordBackward".

Steps to reproduce

1. Create an ordered list with 2 list items.
2. Indent the 2nd one.
3. Toggle its list type by toggling the unordered list button.

This can also be done by reversing the ordered -> unordered list types.

Outcome

What did you expect?

I'd expect only the unordered list button to be highlighted, as it's the active one at this level of indentation.

What happened instead?

Both list buttons are highlighted. This is a bit confusing, as I'd expect the highlighted ordered list button to remove the external list type, 'reversing' the previous action, as most highlighted action buttons work, but instead it toggles the list type for the internal list items.

Screenshot

This screenshot shows what it looks like at the moment, with the 'Now ordered' list item being selected.

Seeing how WYSIWYG is effectively turning into a Message Composer Driver for Element clients (at this time), I wonder if it's possible to support markdown as a composition method, as an alternate mode to rich text.

The kind of presentation (and/or hinting) I'm thinking about would be something like the following, where text would be stylised according to the effects that the markdown syntax would have on that particular piece of text. This specific image comes from Discord, which recently implemented this in their desktop/web composer:

As you can see, this could enrich the typing experience of people who are primarily used to markdown, and may not wholly work well with only-rich-text.

I think matrix-WYSIWYG is uniquely positioned to allow this composition method to flourish in matrix, if it were to offer this input mode alongside rich text.

If we had something like:

<b>Hell</b>|

And then we disabled bold formatting and we typed an ‘o', the whole ‘Hello’ word would be replaced by a plain text 'Hello’, removing the <b> tags.

This happens because on Android, in InterceptInputConnection we get messages from the Android OS telling us that we should replace the whole ‘composition’ (aka the word the cursor is currently in) with a new text, so it would be something like “replace range 0-4 with ‘Hello'” instead of just 'add an “o” at the end of the word’.

On the other hand, if we had:

<b>Hel</b>l|

And we typed “o”, it would apparently make the whole “Hello” word bold.

To workaround this issue on Android, we have 2 options:

1. On InterceptInputConnection.setComposingText get the current composition length (N) and remove the first N characters from the text that should replace it. The rest of the text is what should be inserted at the end of the composition (this only works for typing, autocomplete/autocorrect uses a different method and we probably don’t have to worry too much about them). We’d have to double check if this could affect RTL or languages with UTF16/32 symbols.
2. Create a function in Rust that given a selection and a text, it replaces the underlying text in the Dom while keeping the styles intact. This might be way more complex than the previous approach, but it should be safer.

Converting between html and markdown for formatting functions seem to work well.

For some of the newer functionality there is a bit of inconsistency that means we converting html => markdown => html is not consistent and we see some odd behaviour. Examples taken from element web.

Lists (doesn't reach a steady state, list items keep shuffling along):

Code blocks (reaches a steady state, but note it becomes inline code):

Found when implementing lists in web.

Writing something, inserting a linebreak, then clicking on list puts a linebreak in the first list item, making it appear that the cursor is not in the list item.

Recording demonstrating this can be seen here:

Example test to recreate this:

#[test]
fn new_list_after_linebreak() {
    let mut model = cm("start|");
    model.enter();
    model.unordered_list();

    assert_eq!(tx(&model), "start<ul><li>~|</li></ul>");
    // this will fail, result is "start<ul><li>~<br />|</li></ul>"
}

With rich text editor labs flag enabled
Open a thread, and type a message
The keyboard focus is now in the main timeline composer.

I would expect the keyboard focus to stay in the same composer where I typed my message (in the thread).

Having:

let mut model = cm("{asd}|");
model.bold();
model.bold();
model.bold();
model.bold();
model.undo();
model.undo();
model.undo();
model.undo();

The several undo should render the text as bold half of the time. However, the rendered text is always plain text.

Can be reproduced in both Web and Android, so it seems to be a Rust issue.

If a link is highlighted or just part of it (following google doc's behaviour), and a new link is added it should just update the current existing link.

Seen in the element web on 19 Jan.

Steps to reproduce:

• with the current text editor, highlight a message in the timeline, click the three dots in the top corner, then click Quote on the menu that appears
• the composer will have the message, translated into markdown, inserted into it
• switch to the rich text editor
• repeat the first step to click on Quote on a message in the timeline
• nothing appears in the composer

Expected behaviour:

• as a guess, I'd say that the rich text editor should probably display the quoted message inside a <blockquote> tag, similarly to the current behaviour (but not in markdown)

• Type a line of text
• Press Enter
• Type another line of text

Note that the 2 lines are joined together, so I only have 1 line.

They should remain as 2 separate lines.

I copied a portion of a Google Doc which included links and a bulleted list and pasted it into the rich text editor hoping it would copy all the formatting so I wouldn't need to convert manually to Markdown. It unfortunately lost all formatting and ended up with raw text.

Pasting the same text into e.g. Thunderbird or a GitHub comment box (mostly) copies the formatting.

Steps to reproduce

• Checkout 921adc7
• Toggle list on
• Toggle list off
• Type a character
• Backspace
• Toggle list on
• Crash

Failing test

#[test]
fn toggle_list_then_type_text() {
    let mut model = cm("|");
    model.unordered_list();
    model.unordered_list();
    
    // State is now bad: "<ul></ul>~|"
    
    model.replace_text(utf16("A"));
    model.backspace();
    model.unordered_list(); // panic
}

Screen recording

Steps to reproduce

1. Type some text
2. Select it form back to front.
3. Select link and put in a url
4. Press OK

This can also be done by reversing the ordered -> unordered list types.

Outcome

What did you expect?

A link would be added

What happened instead?

A link was not added, the same text remained.

Screenshot

Given:

If I try to delete the 1st character with a backspace press in any list item, InterceptInputConnection.deleteSurroundingText(beforeLength: Int, afterLength: Int) will be called by the OS with beforeLength being 1 (the character before the selection) and afterLength being the length of the rest of the word (4 in this case), so effectively the OS is telling us to delete the whole word for some reason.

This is not what should happen and we need to either investigate why this is happening and fix it or replace the current implementation of that method with one that just deletes whatever the current selection is, although this might lead to some other side effects in unknown conditions.

Mis-spell a word (in Web).
Choose the correct suggested spelling.
Type some more characters.

Observe that your wrong spelling re-appears.

And also for ZWSP chars, and line breaks.

Since such a link is now created but is not visible to the user, a selection that contains only these elements (whitespaces, line breaks and ZWSP) should replace them with new text if a new links is added.

https://element-io.atlassian.net/browse/PSU-1036

Inserting a line break doesn't apply/remove pending formats, only replace_text does.

let mut model = cm("{Test}|");
model.bold();
model.select(Location::from(4), Location::from(4));
model.bold(); // This writes bold into toggled_format_types
model.enter(); // This should apply pending formats (e.g. remove bold)
model.enter();
model.enter();
model.replace_text("T"); // Bold is only removed here
model.replace_text("e");
model.replace_text("s");
model.replace_text("t");
assert_eq!(tx(&model), "<strong>Test<br /><br /><br /></strong>Test|");

Some users have mentioned that the automatic spell checking is not working.

• vector-im/element-android#7463

Confirmed in other rich text editors:

let mut model = cm("before <em>{italic}|</em> after");
model.create_unordered_list();
assert_eq!(tx(&model), "<ul><li>before <em>{italic}| after</em></li></ul>");

Note: this works as expected if you don’t have the italic part already.

The list should operate on the whole paragraph, which I think means expanding to cover all non-structural nodes before applying

Steps to reproduce

• Type a line of text
• Type several new lines
• Place the cursor at the end of the line of text
• Toggle quote on
• Note that the quote contains an extra line break

Failing test

#[test]
fn apply_quote_to_simple_text_with_line_breaks() {
    let mut model = cm("Some text|<br />Next line");
    model.quote();
    assert_eq!(
        tx(&model),
        "<blockquote>~Some text|</blockquote><br />Next line"
    )
    // Actual: "<blockquote>~Some text|<br /></blockquote>Next line"
}

Screen recording

When using Mac, pressing and holding a letter such as 'e' raises the options to add an accented version of the letter:

When selecting an accented letter in rich mode, both the original letter and the accented letter are displayed:

When selecting an accented letter in plain mode, only the accented letter is displayed

I think that the latter behaviour is the better behaviour of the two, and that the behaviour should at least be consistent

Pretty much the same behaviour as Google Doc

Steps to reproduce

• Create a list
• Backspace the list to clear it
• Create a list or start typing
• Crash

Failing test

#[test]
fn backspacing_in_empty_list_then_creating_a_new_list() {
    let mut model = cm("<ol><li>~|</li></ol>");
    model.backspace();
    assert_eq!(tx(&model), "|");
    model.unordered_list();
    assert_eq!(tx(&model), "<ol><li>~|</li></ol>");
}

Video

To integrate the editor to an Android application written using Jetpack Compose, the Android library should expose a Composable editor, i.e. a method annotated with @Composable.

Will be useful for vector-im/element-x-android#1, however this is not blocking us.

Tasks

• #754
  • Theming and dark mode
• #755
• #758
• #760
  • Add usage documentation

Out of scope

• Feature parity with View library

Describe the bug
When making an unordered list from text that is formatted in two different ways, the list will be created from the part of the text that has first been formatted with the second part of the text appearing on a separate line below. For example, this text wqweaedaw would become:

• wqw
  eaedaw

Expected behavior
The entirety of the text forming a list

Test Case
let mut model = cm("sadsaww{daswdaswd}|");
model.italic();
model.select(Location::from(7), Location::from(7));
model.underline();
model.select(Location::from(7), Location::from(7));
model.underline();
model.underline();
model.italic();
model.select(Location::from(7), Location::from(7));
model.strike_through();
model.select(Location::from(0), Location::from(7));
model.unordered_list();
assert_eq!(tx(&model), "

• {sadsaww}|

Hej :-),

Using the wysiwyg crate, this code generates a really sub-optimal DOM tree:

use wysiwyg::ComposerModel;
use widestring::Utf16String;

let model = ComposerModel::<Utf16String>::from_html(
    "aa&lt;strong&gt;bbbb&lt;/strong&gt;cc",
    0,
    0,
);

dbg!(&model.state.dom);

It generates this DOM tree:

Dom {
    document: Container(
        ContainerNode {
            name: "",
            kind: Generic,
            attrs: None,
            children: [
                Text(
                    TextNode {
                        data: "aa",
                        handle: DomHandle {
                            path: Some(
                                [
                                    0,
                                ],
                            ),
                        },
                    },
                ),
                Text(
                    TextNode {
                        data: "<",
                        handle: DomHandle {
                            path: Some(
                                [
                                    1,
                                ],
                            ),
                        },
                    },
                ),
                Text(
                    TextNode {
                        data: "s",
                        handle: DomHandle {
                            path: Some(
                                [
                                    2,
                                ],
                            ),
                        },
                    },
                ),
                Text(
                    TextNode {
                        data: "trong",
                        handle: DomHandle {
                            path: Some(
                                [
                                    3,
                                ],
                            ),
                        },
                    },
                ),
                Text(
                    TextNode {
                        data: ">",
                        handle: DomHandle {
                            path: Some(
                                [
                                    4,
                                ],
                            ),
                        },
                    },
                ),
                Text(
                    TextNode {
                        data: "b",
                        handle: DomHandle {
                            path: Some(
                                [
                                    5,
                                ],
                            ),
                        },
                    },
                ),
                Text(
                    TextNode {
                        data: "bbb",
                        handle: DomHandle {
                            path: Some(
                                [
                                    6,
                                ],
                            ),
                        },
                    },
                ),
                Text(
                    TextNode {
                        data: "<",
                        handle: DomHandle {
                            path: Some(
                                [
                                    7,
                                ],
                            ),
                        },
                    },
                ),
                Text(
                    TextNode {
                        data: "/",
                        handle: DomHandle {
                            path: Some(
                                [
                                    8,
                                ],
                            ),
                        },
                    },
                ),
                Text(
                    TextNode {
                        data: "strong",
                        handle: DomHandle {
                            path: Some(
                                [
                                    9,
                                ],
                            ),
                        },
                    },
                ),
                Text(
                    TextNode {
                        data: ">",
                        handle: DomHandle {
                            path: Some(
                                [
                                    10,
                                ],
                            ),
                        },
                    },
                ),
                Text(
                    TextNode {
                        data: "c",
                        handle: DomHandle {
                            path: Some(
                                [
                                    11,
                                ],
                            ),
                        },
                    },
                ),
                Text(
                    TextNode {
                        data: "c",
                        handle: DomHandle {
                            path: Some(
                                [
                                    12,
                                ],
                            ),
                        },
                    },
                ),
            ],
            handle: DomHandle {
                path: Some(
                    [],
                ),
            },
        },
    ),
}

What should we expect? I guess this:

Dom {
    document: Container(
        ContainerNode {
            name: "",
            kind: Generic,
            attrs: None,
            children: [
                Text(
                    TextNode {
                        data: "aa<strong>bbbb</strong>cc",
                        handle: DomHandle {
                            path: Some(
                                [
                                    0,
                                ],
                            ),
                        },
                    },
                ),
            ],
            handle: DomHandle {
                path: Some(
                    [],
                ),
            },
        },
    ),
}

Note: If HTML entities (&…;) are removed, the DOM tree is now optimal.

In #225, I'm replacing html5ever (in Rust) by DOMParser (in JavaScript) for the Wasm module of wysiwyg-wasm. Using DOMParser generates the second, expected, DOM tree. And suddenly, the following tests with a TODO saying to_example_format has a bug, are fixed: https://github.com/matrix-org/matrix-wysiwyg/blob/b2dcc603e34860a6b989cb8c8ed73e8e80270c61/platforms/web/lib/useTestCases/utils.test.ts#L231-L330

And here is the commit that fixes those tests by the way, dcfc602 (it's part of #225).

So where are we? The bug seems to be fixed in wysiwyg-wasm because the tree is now optimal. However, it is not fixed on other platforms that use html5ever. And the bug may appear with another DOM tree even in wysiwyg-wasm I guess.

There are 2 things here:

1. The DOM is sub-optimal here, and we should merge TextNode when it's possible,
2. There is a bug in to_example_format when it needs to deal with HTML encoded characters, maybe, not sure.

When a code block is created before a new line, text is not inserted to the right place.

Steps to reproduce

• Create a new line
• Move the cursor to position 0
• Toggle code block on
• Type some text
• Note text is created outside the code block

Failing test

#[test]
fn create_code_block_before_new_line() {
    let mut model = cm("|<br />");
    model.code_block();
    model.do_replace_text(utf16("hello"));
    model.enter();
    assert_eq!(tx(&model), "<pre>~hello<br />|</pre><br />");
    // actual: "<pre>~</pre>hello<br />|<br />"
}

Screenshot

Note the model after the example in this video is:

<pre>~</pre>text is outside code block<br />and this is too|<br /><br />

These are real useful when sharing code with people

We currently have a slight issue with converting html to markdown in the rust model. Issue is as follows:

• html can support whitespace inside tags eg <em>hello </em> is valid html and if a user were to click italic, type 'hello ' and then click italic again to disable the italic mode, this is the html they would have input
• when we convert this to markdown, the output is _hello _
  • this is not valid markdown due to the space before the final underscore
  • this means that if we try to parse that markdown back into html we get the output _hello _

In summary, converting html => markdown => html at the moment is not perfect due to the difference in how the formats handle whitespace at the end of a 'node'.

steps:

1. Type asdf

   let mut model = cm("{asdf}|");

2. Ctrl-backspace, or whatever the equivalent is on Mac

   let mut model = cm("|asdf");

   This is not the expected behavior! It should be cm("|"), or cm(""). The problem becomes apparent during the next step
3. Start typing anything. asdf comes back

(Tested on the web interface)

Seen in the example app on 17 Jan.

Steps to reproduce:

• write text on three separate lines
• highlight the middle line
• select code block
• note the appearance of another linebreak above the codeblock

Expected behaviour:

• inserting a code block in this way should only add the code block and not the extra linebreak

Notes:

• Looking at the video I actually wonder if it's just moving the second linebreak above the pre tag, so this might be an ordering issue as opposed to unexpectedly adding an element

Unbold some text next to some other text. Note that we end up with two adjacent text nodes. They should have been merged.

Failing test (despite indent button/action being marked as available):

#[test]
fn can_indent_on_last_char_of_list_item() {
    let mut model = cm("<ol><li>abc</li><li>def|</li></ol>");
    model.indent();
    assert_eq!(
        tx(&model),
        "<ol><li><p>abc</p><ol><li>def|</li></ol></li></ol>",
    )
}

If moving the cursor one character to the left it does work.

This is an example of what happens when applying a link

Original
<ul><li>Te{st</li><li>Bo}|ld</li></ul>
Result
<ul><li>Te</li><a href="https://element.io"><li>{st</li><li>Bo}|</li></a><li>ld</li></ul>

The list is broken and two more list elements are created.
This because the link is always set to be the outermost container node, however this should be avoided for block nodes.

Seen in the example app on 17 Jan.

Steps to reproduce:

• Start a list
• enter three items
• select the second item
• insert a new item
• start typing, text appears on a different line
• also, subsequent presses of enter appear not to add a list item
  This can be seen in the video below:

Expected behaviour:

• user can enter list items in the middle of the list in the same way as they can currently add items to the end of the list

Reported by Valere:

Bug on the rich editor
1: Type 'bar'
2: Select Bold
3: Type 'foo'
4: Select Italic
5: Type 'baz'
6 press space
=> Resets all editing on previous text

In the current implementation of lists on web, to exit a list, a user has to press enter in an empty list item to exit a list.

This issue is being raised to discuss some issues resulting from this behaviour:

1. It is not possible to add empty bullet points to a list
2. Previous experience (Slack, googledocs) led me to expect pressing backspace in an empty list item would remove the item and exit the list (actual behaviour removes the item, but puts the cursor at the end of the previous list item, shown in video)
3. Should pressing the down arrow exit a list? I think it should but this could just be my expectation being odd, I think this probably comes from time using Slack (as it's the composer I've used the most)

Video of pressing backspace in an empty list item, then enter twice to exit
https://user-images.githubusercontent.com/56027671/212375062-e7fdbca3-a9ea-4707-8c3e-f1d0db23dfe7.mov

Discussion of this suggested expected behaviour might be something along the lines of:

• have an empty list item
• press backspace
• cursor goes to start of line, exiting the list
• press backspace
• cursor goes to end of last list item (inside the list)
• up or down arrows can be used to enter/exit the last list item (web only)

Notes:

• Slack and Googledocs behaviour is close to the expected behaviour above
• Slack actually achieves this behaviour using custom classes on list items, ie a list is a single list node and everything else inside is a list item

• Implementing nested lists inside other lists may present issues on mobile clients

The deleteSoftLineBackward event can be fired on mac by doing cmd + backspace (expect it would be ctrl + backspace on windows).

Expected
Pressing cmd/ctrl + backspace should remove a line

Actual
Pressing cmd/ctrl + backspace appears to remove the line, but then type anything and the line reappears

This can be seen in the sample app, screen grab below:

Enabling the Rich Text Editor (RTE) stops /rainbow autocompletion and rainbowfying messages.

Steps:

1. With RTE disabled, start typing /rainbow (autocompletes to show the explanation of the rainbow command)
2. Continue typing so your message reads /rainbow _message_ and press ENTER (message appears in the room coloured as a rainbow
3. Go to Labs and enable RTE
4. Start typing /rainbow followed by a message and press ENTER

Result:

• With RTE disabled the autocomplete explaining the rainbow command appears and any message that follows is formatted coloured as a rainbow.
• With RTE enabled the autocomplete explanation does not appear and what gets output is /rainbow message in plain, unformatted text

Tested this on Fedora Linux 37 with Element Desktop 1.11.16 installed via Flatpak from Flathub.org. I also replicated this on Element Android 1.5.10 on a Pixel 7 running GrapheneOS Tiramisu, Element iOS 1.9.13 on an iPhone 12 Mini running iOS 16.1.2 and staging.element.io on Firefox 107.0.1 on Fedora.

Here is a screenshot of the autocomplete with RTE disabled:

Here is a screenshot of the message formatting, the first grouping of messages is with RTE enabled, the last message is with me entering /rainbow message with RTE disabled:

I'd be happy to provide any other information that is useful.

Steps to reproduce

• Type four lines of text
• Place the cursor at the end of the second line
• Toggle code block on
• Note lines two and three are swapped

Failing test

#[test]
fn code_block_preserves_line_order() {
    let mut model = cm("A<br />B|<br />C<br />D");
    model.code_block();
    assert_eq!(tx(&model), "A<br /><pre>~B|</pre><br />C<br />D");
    // actual: "A<br />C<pre>~|B</pre><br />D"
}

Screen recording

Seen in the example app on 17 Jan.

Steps to reproduce:

• Start a code block
• enter some text
• press enter
• note cursor appears to still be on same line
• start typing
• new text appears on a new line
  This can be seen in the video below:

Expected behaviour:

• pressing enter moves the cursor to a new line and this can be seen by the user

Notes:

• I'm not sure if this is purely a web issue, but I think it could be so have labelled it as such
• This bug is a bit annoying combined with our decision to make two consecutive presses of Enter exit the code block, as there's no visual indication of the first press
• Same behaviour can be seen in Quotes

• Center the microphone icon vertically, since it seems to be a bit nearer to the bottom than expected.
• When the full screen mode is collapsed, don't hide the keyboard.

Seen in the example app on 18 Jan.

Steps to reproduce:

• write some text (perhaps not necessary)
• click the quote button
• write some text
• press enter (note same issue as code blocks where cursor does not appear to move, raised in issue #491)
• note that linebreak appears inside the blockquote tag
• press enter again to stop editing the quote
• cursor moves outside the codeblock (good) but linebreak moves from inside the blockquote to the containing element

Uploading Screen Recording 2023-01-18 at 15.56.49.mov…

Expected behaviour:

• On second press of the enter key I think that the blockquote should have the final br node removed
• The cursor is then placed after the blockquote

Seen in the example app on 18 Jan.

Steps to reproduce:

• click the inline code button
• note the state of the buttons, inline code shows as active, but disabled buttons do not appear disabled
• type a single letter
• now the disabled buttons appear disabled
• click on inline code again, note some buttons still show as disabled
• type a single letter, the buttons display the expected states

Expected behaviour:

• on clicking the inline code button to start, the disabled formatting buttons should immediately be disabled
• on clicking the inline code button to finish, the disabled formatting buttons should immediately be enabled

Notes:

• As we will just lookup if we are inside a <code>, when we hit the button with no selection we only create the node when we start typing. We need to lookup both the current parent nodes & the pending formats to apply.

When toggling list off and removing the list entirely, the selection is not properly updated to reflect the ZWSP removal.

#[test]
fn toggling_list_updates_selection() {
    let mut model = cm("Test|");
    assert_eq!(model.get_selection(), sel(4, 4));
    model.ordered_list();
    assert_eq!(model.get_selection(), sel(5, 5));
    model.ordered_list();
    assert_eq!(model.get_selection(), sel(4, 4)); // Fails because selection is not decremented again
}

It's possible to create a bad state by selecting a position immediately after a zero width space and then backspacing it.

Steps to reproduce

Tested on Android...

• Create a few list items
• Tap the first half of the first character of a list item to place the cursor before that character but after the zero-width space
• Press backspace
• Start editing
• Observe the broken state (missing zero-width spaces)

Failing test

#[test]  
fn select_and_backspace_start_of_a_list_item() {  
    let mut model = cm("<ul><li>~A</li><li>~B</li></ul>|");
    
    model.select(Location::from(3), Location::from(3));
    assert_eq!(tx(&model), "<ul><li>~A</li><li>~|B</li></ul>"); // Pass
    
    model.backspace();
    assert_eq!(tx(&model), "<ul><li>~|</li><li>~B</li></ul>"); // Fail
    // actual "<ul><li>~A|</li><li>B</li></ul>"
    // Note the missing "~" (the debug symbol for ZWSP)
}

Solutions

We could handle this specific edge case for backspacing in this selection state. Though it might be worth rethinking the use of zero-width spaces at the beginning of list items.

May be a rust bug, may not be.

Steps to reproduce

• In the example app, click on a button to start a list
• note that the button does not show as active
• write a list item
• add another list item
• as soon as the second list item appears, the button will show as active

Expected behaviour

• on clicking the list button, it should show as active

Notes

• If you write some text, add a line break and then start a list, the button state will display as expected

let mut model = cm("text|");
model.inline_code();
model.replace_text("text");
model.inline_code();
model.inline_code();
model.replace_text("text");
assert_eq!(tx(&model), "text<code>text</code>text|");

Found issue while fixing #495
It seems that when inlineCode is added to toggled_format_types in order to disable it on next input it cannot properly be removed again. Hitting the button twice should have re-enabled the inline code and further text should have been inside the <code>.

Note: other "format" like e.g. bold work fine for this use case.