Allow any valid HTML4 identifier string to be a djot identifier string about djot HOT 2 OPEN

Currently the syntax for attributes (undocumented except in code comments) is

 attributes <- '{' whitespace* attribute (whitespace attribute)* whitespace* '}'
 attribute <- identifier | class | keyval
 identifier <- '#' name
 class <- '.' name
 name <- (nonspace, nonpunctuation other than ':', '_', '-')+
 keyval <- key '=' val
 key <- (ASCII_ALPHANUM | ':' | '_' | '-')+
 val <- bareval | quotedval
 bareval <- (ASCII_ALPHANUM | ':' | '_' | '-')+
 quotedval <- '"' ([^"] | '\"') '"'

So we don't allow . in an identifer. I can't recall whether there was a specific reason for this.
XML identifiers are more restrictive than this (must start with letter or underscore). HTML4 identifiers are less restrictive, and HTML5 identifiers are much less restrictive.

Class names have more restrictions (at least if they're to be used with CSS).

EDIT: Anyway, I'm open to making this less restrictive, but some thought needs to go into what would be a reasonable restriction.

from djot.

At first glance, it seems like djot has a principal to not distinguish between the first character and other characters in ids, possibly for simplicity of implementation? Which dictates that . can't appear in ids because '.' name indicates a class? Or possibly it's just that classes and ids follow the same pattern, and class name in djot may not contain periods (which I agree is a good decision).

HTML4 identifiers are less restrictive

As I understand it HTML4 ids are generally extremely restrictive, because they follow the SGML rules laid out ISO 8879:1986. #1, #:, and #ß are all invalid HTML4 identifiers or class names, but valid djot identifiers because they don't start with [A-Za-z].

The only case I see where djot is more restrictive than HTML4 is that "foo.bar" is a valid HTML4 identifier but an invalid djot identifier because it contains a .This difference prevents a lot of pretty basic ascii-encoded HTML4 from being able round-trip through djot back to HTML.

I have one firm proposal, which is to disentangle the identifier and class rules to allow non-initial identifier characters to be periods. I.e.:

 identifier <- '#' nameChar Maybe[subsequentIdChar+]
 class <- '.' nameChar+
 nameChar <- (nonspace, nonpunctuation other than ':', '_', '-')
 subsequentIdChar <- (nonspace, nonpunctuation other than ':', '_', '-', '.')

My goals would be served equally well by the parser accepting periods on ids in any position but requiring them to be escaped (\.). But that feels uglier.

I don't have opinions about any larger related changes, though I do like how unicode characters can be id and class names in djot.

Just for context, I should possibly say that my interests here are not primarily in writing in DJOT, but in getting things into djot's AST, which is much nicer to work with than pandoc's for my purposes.

from djot.