Consider revealing `:smile:` -> 😀 in the AST about djot HOT 4 CLOSED

I just pushed a change that does the resolution in the parser.
But this can always be reverted. I see both sides of it. Note that we could still say that parser implementations aren't required to implement emoji lookup -- the renderer could still do its own lookup on the alias provided.

from djot.

The reason I did it this way is that renderers that consume the AST might make different decisions about how to render the emoji. For example, a djot renderer might want :smile:, while in HTML you'd probably want 😄

Another reason isl to avoid requiring the parser to have the big emoji table. So, for example, :oeu: gets parsed as an emoji with alias oeu; the renderer will look this up, not find it, and have to figure out what to do.

from djot.

I can see why it would be convenient to have the entity resolved to unicode in the AST.
(That would free the consumer of the need to do the substitution, and why not? since we have the entity table in djot.)

from djot.

might make different decisions about how to render the emoji.

This I think is addressed by having both text and raw.

Another reason isl to avoid requiring the parser to have the big emoji table

This sounds very convincing to me. And, if we don't substitute emojis, substituting em,en dashes, quotes and ellipsis doesn't make sense (they are a drop in the bucket).

I guess what would help here is to have a canonical table with substitution as a part of the spec, so that:

• users have an expected amount of subsitutions which work the same across implementations
• implementations can be easily consistent by building of the same source of truth.

and why not? since we have the entity table in djot.)

I think a good argument to not do this is to make sure that consumers actually can work with implementations which doesn't bundle an emoji table. That is, if the ref impl does this because "why not?", then every other impl would be pushed to do this anyway.

from djot.