[BUG] Getting text_markdown_v2 from message does not always work correctly when the message contains blockquote entities about python-telegram-bot HOT 5 CLOSED

Hi. Thanks for reaching out. We are aware of this behavior, but this is accepted and documented in the docs of text_markdown_v2. If you are interested in more background story, please have a look at the discussion in #4038.
As alternative, Telegram recommends that you work directly with Message.entities.
I expect that my answer will be unsatisfactory for you - as it is for me - but this is the current status...

from python-telegram-bot.

Hi. Thanks for reaching out. We are aware of this behavior, but this is accepted and documented in the docs of text_markdown_v2.

Thanks for the quick response and sorry, I don't know how I read the documentation and didn't see that warning.

from python-telegram-bot.

Hi. Thanks for the additional investigation. TBH I can't really form a opinion atm if this would affect any other special cases/introduce different problems. At least one point that I see is that simply checking for endswith("\n") is not enough, since the text could actually end on an escaped \\n.
Frankly, since TG officially discouraged us from providing these proporties, I'm not very motivated to try and cover every edge case … This is also why we included the note in the docs.

from python-telegram-bot.

If you are interested in more background story, please have a look at the discussion in #4038.

Hi again, I took some time and looked through the discussion and the code responsible for text_markdown_v2 and found the following:

insert = ">" + "\n>".join(escaped_text.splitlines())

/n at the end of escaped_text will be ignored by splitlines() that is what causes behavior which I described in the issue.
When I added insert += "\n" if escaped_text.endswith("\n") else "" my issue was solved.

But there is another, and it is more about Telegram Desktop only:
If the blockquote is immediately followed by text without \n, the text will be attached to the blockquote, and if blockquote is immediately after text the telegram.error.BadRequest: Can't parse entities: character '>' is reserved and must be escaped with the preceding '\' is raised.
Telegram Desktop allows writing blockquote like this

And after sending, blockquote is shown on separate line, but there is no \n at all.

Since we know where quote actualy starts and ends, I think, it can be fixed, by adding check if there is \n in the beginning and ending of blockquote, and adding it if needed.

I changed localy:

            elif entity.type == MessageEntity.BLOCKQUOTE:
                # Check if there's no \n before blockquote and blockquote is not at the beginning of message
                if message_text[entity.offset - 1] != "\n" and entity.offset != 0:
                    insert = "\n>" + "\n>".join(escaped_text.splitlines())
                    insert += "\n" if escaped_text.endswith("\n") else ""
                else:
                    insert = ">" + "\n>".join(escaped_text.splitlines())
                    insert += "\n" if escaped_text.endswith("\n") else ""
                # Check if blockquote is not the last thing in message
                if entity.length + entity.offset != len(message_text):
                    # Check if there's an \n after blockquote
                    if message_text[entity.length + entity.offset] != "\n" and not escaped_text.endswith("\n"):
                        insert += "\n"

And everything works as expected.

Maybe adding \n which was not there before is bad, but it should be there according to Telegram's recommendations as in that, mentioned earlier, warning:

Telegram recommends that entities of type BLOCKQUOTE and PRE should start and end on a new line

Hope this will help resolve the issue.

from python-telegram-bot.

Ok.
Thank you for your time.

from python-telegram-bot.