mmarkdown / mmark Goto Github PK
View Code? Open in Web Editor NEWMmark: a powerful markdown processor in Go geared towards the IETF
Home Page: https://mmark.miek.nl
License: Other
Mmark: a powerful markdown processor in Go geared towards the IETF
Home Page: https://mmark.miek.nl
License: Other
For I-Ds you may want to add a draft sequence number, which can be done as such: [@?I-D.blah#06]
There was some special code in mmark1 that hasn't been ported yet.
References including target attributes on reference blocks do not have a target attribute in the outputted xml. It seems this issue was also reported in a comment on issue #53 but never fixed.
2.0.38
mmark ./test.md > /tmp/x.xml
%%%
Title = "Test doc"
date = 2019-02-19T00:00:00Z
[seriesInfo]
name = "Internet-Draft"
value = "draft-test-00"
status = "informational"
[[author]]
initials="J."
%%%
{mainmatter}
# Introduction
Test [@!IANA]
{backmatter}
<reference anchor='IANA' target='https://www.iana.org/assignments/media-types/media-types.xhtml'>
<front>
<title abbrev='IANA'>IANA Media Types</title>
<author>
<organization>IANA</organization>
</author>
<date month='February' year='2019'/>
</front>
</reference>
Figure out what to do with asides in 7749 output.
Subfigures in mmark are sorta supported by prefixing a paragraph with F>
. This is a bit annoying. The scholary markdown (discontinued) solution seems better:
Do this? http://scholarlymarkdown.com/Scholarly-Markdown-Guide.html#floating-env
V2 has dropped Parts, it may make sense to bring that back (not that difficult either).
See https://github.com/miekg/mmark/wiki/Syntax#parts for the syntax.
Bibliography is stored in a map and when we range
over it we get the elements in random order; this should be sorted so a single source markdown file always maps to the same output.
Add indexInclude attribute to the TOML block: https://tools.ietf.org/html/rfc7991#section-2.45.4
Ident all lists when generating markdown, so
* item1
* item2
will become
* item1
* item2
this works for kind of listst
I'm trying to use hugo + mmark + katex. Documentation around the web about math support in mmark is sparse and seems to be partly / mostly outdated.
Using mmark as a markdown processor in hugo, $$ E = mc^2 $$
is rendered to <span class='math'>\( E = mc^2\)</span>
which is valid TeX, but doesn't seem to be compliant with MathJax as is written in most docs, but rather seems to be MathJS syntax (?).
Trying to run
$("span[class='math']").replaceWith(
function(){
var tex = $(this).text();
return "<span class=\"inline-equation\">" +
katex.renderToString(tex) +
"</span>";
});
over it fails because KaTeX cannot deal with the delimiters \(
and \)
.
Is there more documentation on math support in mmark that I have missed and is it possible to configure how the math is rendered?
In the manual output footnotes are outputted in a "Notes" section, we can do the same in the xml/xml2 renderers, same code (almost) can even be used.
This would be inserted just before the {backmatter}
similar to the place of the bibliography, but on the opposite site.
(re-) implement example lists.
Follow the Pandoc implementation, but only support the alfanum implementation (@good)
is allowed, but not (@)
(as you can't reference those).
Referencing happens with the same syntax (@good)
for rfc7991 output we can't output images as artwork, as a image element is not an (block)level artwork, this means we must inhibit outputting <t>
s, but that is hard. Also using the title for the name attribute looks weird.
A figureblock works, but must be present in the markdown code.
Now that there is a -markdown
renderer which output markdown again, we can work on the AST and transform it and then output "new" markdown.
I.e. you can write a super tiny go program that rewrites the AST: for instance to transform all emphasis to strong. (inline are easiest).
But you can also have a more sophisticated one that looks at a code block and if the language matches exec-dot
executes dot
and converts the whole code block into an dot-generated image.
Unable to include a link of a code sample. I commonly do this—and see this—on StackOverflow.
2.0.40, and xml2rfc 2.22.0
$ mmark a.md > a.xml
$ xml2rfc --v3 a.xml
Parsing file a.xml
a.xml(18): Error: Did not expect element t there, at /rfc/middle/t
a.xml(3): Error: Invalid document before running preptool.
Cannot continue, quitting now
$ cat a.md
%%%
title = "Model designer"
abbrev = "ODESIGN"
area = "Internet"
workgroup = "Offscale.io"
keyword = ["platform-independent"]
#date = 2019-12-07T00:00:00Z
[seriesInfo]
name = "RFC"
value = "ODESIGN0.0.1"
status = "informational"
[[author]]
surname="Marks"
fullname="Samuel Marks"
organization = "Sydney Scientific"
[author.address]
email = "[email protected]"
%%%
{mainmatter}
You MAY use [`localStorage`](https://developer.mozilla.org/en-US/docs/Web/API/Window/localStorage) and/or similar built-ins.
{backmatter}
Need to double check if tables are rendered correctly, esp. when they have a caption.
Also tests.
Thanks, using your new release now. Installed directly. Can you release a new version also?
The use of non-breaking spaces is useful for better typography and indispensable in some languages, furthermore when writing long texts (documentation, books, or just blog posts).
However, inserting
in all files is not ideal as it is tiresome and can bring some brittleness (it only works as long as the target rendering is HTML).
Could we think of a universal shortcut that would be rendered appropriately?
See Fig. 3
.Processing time: 25 ms
.:
, ;
, ?
and !
should be preceded with a non-breaking space: Et il déclara : « Que c'est pénible d'écrire correctement ! Faut-il systématiquement saisir six caractères pour une espace ? »
{nbsp}
variable. It is as verbose as writing
but rules out the brittleness as it can be treated differently depending on the renderer.\
) or a tilde (~
) as a non-breaking space. For instance: 42\ ms, 1337~ms
.> this is a quote quote with *empp* and more
>
hallo more tetg hallo more tetg hallo more tetg hallo more tetg hallo more tetg hallo more
> tetg hallo more tetg hallo more tetg
>
> : ahllo
>
more1
>
> : more
>
is rendered wrong, the >
is tripped from the prefix, because we cut the entire prefix which is too much.
Add a 'safeInclude' flag that only allows relative includes and no ..
in the paths. This is a simple check.
A more advanced safe include would be to only allow files below the initial file being work on.
Now that gomarkdown/markdown support an IncludeFunc
we can implement all the bits we need in mmark.
Can't build an RFC without warnings:
514.xml(5): Warning: Expected explicit values for year, month, and day, but found None, None and None
3514.xml(3): Warning: Expected a <link> with rel='convertedFrom' providing the datatracker url for the origin draft.
We need a convertedFrom in the toml, maybe 7749 can use it as well.
See examples in the rfc dir
This is not rendered correctly in the markdown output.
[I'm an inline-style link](https://www.google.com)
[I'm an inline-style link with title](https://www.google.com "Google's Homepage")
[I'm a reference-style link][Arbitrary case-insensitive reference text]
[I'm a relative reference to a repository file](../blob/master/LICENSE)
[You can use numbers for reference-style link definitions][1]
Or leave it empty and use the [link text itself].
URLs and URLs in angle brackets will automatically get turned into links.
http://www.example.com or <http://www.example.com> and sometimes
example.com (but not on Github, for example).
Some text to show that the reference links can follow later.
[arbitrary case-insensitive reference text]: https://www.mozilla.org
[1]: http://slashdot.org
[link text itself]: http://www.reddit.com
mmark logs Attribute "start" not supported for list style="numbers"
and makes the 2.
element of a list start over at 1.
.
v2.0.46
mmark -2
1. Process the internal representation into a field element s as follows:
_some unindented code block here_
2. Return the canonical little-endian encoding of s.
xml2rfc --v3
complained with Warning: Expected a 'docName' attribute in the <rfc/> element, but found none.
, and https://datatracker.ietf.org/submit/ rejected the XML.
Manually adding the docName
attribute fixed the issue.
2.0.46
mmark
%%%
Title = "The ristretto255 Group"
abbrev = "ristretto255"
ipr = "trust200902"
category = "info"
area = "Internet"
workgroup = "Crypto Forum Research Group"
[seriesInfo]
status = "informational"
name = "Internet-Draft"
value = "draft-hdevalence-cfrg-ristretto-01"
stream = "IETF"
[...]
<name>
needs to be directly after <figure>
, right now it comes after the artwork. Simple change and we can look how xml2/renderer.go does it.
It might make sense to add v2 XML output as well to mmark2, as the v3 tool chain for the IETF is still in development. As a new render is relatively easy to add this should be doable. And a lot of stuff could be copied from mmark1 (https://github.com/miekg/mmark)
Is it possible to support multiple references to a footnote content? Seems that blackfriday has implemented the feature through russross/blackfriday#366.
Sentence one[^1]. Sentence two[^1]. Sentence three[^1].
[^1]: They should all refer to here.
As of Hugo 0.49, whose blackfriday engine outputs one line of footnote while mmark does 3 lines of them with incremental numbers.
Reference is missing organization in the author block. I am not familiar with writing programs in Go, However, looking at the previous fix, it seems that the test is written to pass if the organization is missing.
2.0.39
mmark test.md > test.xml
Using the same snipped as #61.
%%%
Title = "Test doc"
date = 2019-02-19T00:00:00Z
[seriesInfo]
name = "Internet-Draft"
value = "draft-test-00"
status = "informational"
[[author]]
initials="J."
%%%
{mainmatter}
# Introduction
Test [@!IANA]
{backmatter}
<reference anchor='IANA' target='https://www.iana.org/assignments/media-types/media-types.xhtml'>
<front>
<title abbrev='IANA'>IANA Media Types</title>
<author>
<organization>IANA</organization>
</author>
<date month='February' year='2019'/>
</front>
</reference>
From the XML, here's just the reference section. Note the organization in the markdown above does not appear in the XML.
<back>
<references><name>Normative References</name>
<reference anchor="IANA" target="https://www.iana.org/assignments/media-types/media-types.xhtml">
<front>
<title>IANA Media Types</title>
<author>
<organization></organization>
</author>
<date year="2019" month="February"></date>
</front>
</reference>
</references>
We can include svg, but not(?) other figures... need to double check what rfc7991 says about this.
Currently callouts in manual output are supported, but it lacks the option to specify comments to be detected, this does happen in the other renderers and should be added here as well.
If the little Toml header is not correctly set the manual page output will crash on index out of range.
Sometimes you need to fallback to html or xml; currently all blocklevel elements except <reference>
are removed from the final output. This may be sub optimal.
I use go get to install
-html
master
bash
linux
mmark -html ~/go/src/github.com/mmarkdown/mmark/rfc/3514.md > x.html
result:
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x40 pc=0x53d7b3]
goroutine 1 [running]:
github.com/mmarkdown/mmark/render/mhtml.bibliographyItem(0x5cfde0, 0xc000114930, 0xc000114690, 0xc000114901)
/home/king/go/src/github.com/mmarkdown/mmark/render/mhtml/hook.go:102 +0x183
github.com/mmarkdown/mmark/render/mhtml.RenderHook(0x5cfde0, 0xc000114930, 0x5d1d00, 0xc000114690, 0x1, 0x5cfde0, 0xc0000d3718)
/home/king/go/src/github.com/mmarkdown/mmark/render/mhtml/hook.go:37 +0x38c
github.com/gomarkdown/markdown/html.(*Renderer).RenderNode(0xc0000e8b40, 0x5cfde0, 0xc000114930, 0x5d1d00, 0xc000114690, 0x1, 0x0)
/home/king/go/src/github.com/gomarkdown/markdown/html/renderer.go:914 +0x16ce
github.com/gomarkdown/markdown.Render.func1(0x5d1d00, 0xc000114690, 0x4ff401, 0xc0000e8b40)
/home/king/go/src/github.com/gomarkdown/markdown/markdown.go:63 +0x61
github.com/gomarkdown/markdown/ast.NodeVisitorFunc.Visit(0xc00000bd20, 0x5d1d00, 0xc000114690, 0x1, 0xc0000d38e8)
/home/king/go/src/github.com/gomarkdown/markdown/ast/node.go:547 +0x43
github.com/gomarkdown/markdown/ast.Walk(0x5d1d00, 0xc000114690, 0x5d0060, 0xc00000bd20, 0x0)
/home/king/go/src/github.com/gomarkdown/markdown/ast/node.go:519 +0x70
github.com/gomarkdown/markdown/ast.Walk(0x5d1ca0, 0xc0001148c0, 0x5d0060, 0xc00000bd20, 0x0)
/home/king/go/src/github.com/gomarkdown/markdown/ast/node.go:530 +0x1ba
github.com/gomarkdown/markdown/ast.Walk(0x5d1280, 0xc000114000, 0x5d0060, 0xc00000bd20, 0x0)
/home/king/go/src/github.com/gomarkdown/markdown/ast/node.go:530 +0x1ba
github.com/gomarkdown/markdown/ast.Walk(0x5d1220, 0xc000064480, 0x5d0060, 0xc00000bd20, 0xc00000bd20)
/home/king/go/src/github.com/gomarkdown/markdown/ast/node.go:530 +0x1ba
github.com/gomarkdown/markdown/ast.WalkFunc(0x5d1220, 0xc000064480, 0xc00000bd20)
/home/king/go/src/github.com/gomarkdown/markdown/ast/node.go:553 +0x4b
github.com/gomarkdown/markdown.Render(0x5d1220, 0xc000064480, 0x5d0580, 0xc0000e8b40, 0x5a9adc, 0x13, 0x5a861f)
/home/king/go/src/github.com/gomarkdown/markdown/markdown.go:62 +0xd8
main.main()
/home/king/go/src/github.com/mmarkdown/mmark/mmark.go:171 +0x67
A markdown table:
\ | \ | \
------------|---------------|-----------
`break` |`default` |`func`
is rendered:
| |
----------|------------|---------
`break` | `default` | `func`
It's off because it does not account for the backticks (or any other span markings)
they have no content
Stuff can be copied from codeblock or quotes, just needs to be done.
On the latest version of mmark, if a <reference>
has multiple authors, only the last one is rendered in the XML. This includes mmark/rfc/3514.md
.
try to build mmark from source and got following error
# github.com/mmarkdown/mmark github.com\mmarkdown\mmark\mmark.go:90:12: undefined: parser.ParserOptions
failed to build from source after cloning the repo.
check gomarkdown commit
2.0.37
go get && go build
Replication steps (mmark 2.0.40, xml2rfc 2.22.0):
$ mmark index.md > index.xml
$ xml2rfc index.xml -o index.html
Error: Unable to parse the XML document: index.xml
<string>: Line 24: Opening and ending tag mismatch: middle line 17 and
section
<string>: Line 24: Opening and ending tag mismatch: middle line 17 and
section
<string>: Line 28: Opening and ending tag mismatch: rfc line 3 and section
<string>: Line 30: Extra content at the end of the document
More info:
https://mailarchive.ietf.org/arch/msg/xml2rfc/OPep-8cpM-NZFqQZzX_whWpU9iw
Citations can have a suffix. This is not implemented. In pandoc this is signaled using a comma as the separation character we now use a space. We should use a comma as well and implement the suffix use in citation rendering.
Cut and paste from blackfriday and/or gomarkdown and flesh out a full document
I might be nice to allow toml syntax for reference, so that we unify the preamble (which is in toml) with the references. Downside is that all references come in XML format, so you need to reformat that. We already support the XML from the references natively; so not sure how much functionally this adds.
xml2 output for the IETF has already(?) been deprecated and the RFC editor has moved to some form of 7991 XML as the canonical form.
I'll try to get some authoritative source to say so and then delete this xml2 code in Mmark.
When rendering <cite>
s they look a bit off, need to double check if they are ok or what is wrong here.
<figure title="Scenic Routing Option Layout ">
<artwork anchor="fig-scenic-routing-option-layout">
the anchor
attribute need to be on the <figure>
tag
This is not supported as of yet:
!---
![Array versus slice](fig/array-vs-slice.png)
!---
Figure: An array versus a slice.
The issue #56 was fixed 10 days ago, but this software FTBFS. A stable upstream library or dependency management would be nice, but for now a release would fix this also.
Current release FTBFS
Current Release
go build
FTBFS
This should be relatively easy. Support mmark indices: https://mmark.nl/post/syntax/#indices
This needs upstream support in parser/inline.go (add a new file index.go or something) and a bunch of tests to see if we can parse it.
upstream is https://github.com/gomarkdown/markdown
With a long-form xref like the XML output is
[]<xref target="RFC2119"></xref>
and the HTML is
[]<a href="#RFC2119"></a>
I expected no [] in the output.
2.0.10
echo '[](#RFC2119)' >xref.md
~/go/bin/mmark xref.md
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.