Code Monkey home page Code Monkey logo

witiko / markdown Goto Github PK

View Code? Open in Web Editor NEW

This project forked from jgm/lunamark

321.0 8.0 31.0 7.75 MB

:notebook_with_decorative_cover: A package for converting and rendering markdown documents in TeX

Home Page: http://ctan.org/pkg/markdown

License: LaTeX Project Public License v1.3c

Makefile 0.72% TeX 58.25% Shell 0.22% M4 0.26% Dockerfile 0.41% Lua 0.05% Perl 0.01% Python 2.25% Jupyter Notebook 37.83%
tex latex latex-package context lua luatex lualatex luametatex markdown markdown-parser

markdown's Introduction

Markdown

license ci release docker-witiko/markdown matrix discord chatgpt

The Markdown package converts CommonMark markup to TeX commands. The functionality is provided both as a Lua module, and as plain TeX, LaTeX, and ConTeXt macro packages that can be used to directly typeset TeX documents containing markdown markup. Unlike other converters, the Markdown package does not require any external programs, and makes it easy to redefine how each and every markdown element is rendered. Creative abuse of the markdown syntax is encouraged. 😉

Your first Markdown document

Using a text editor, create an empty directory named workdir/. In it, create a text document named workdir/document.tex with the following content:

\documentclass{book}
\usepackage{markdown}
\markdownSetup{pipeTables,tableCaptions}
\begin{document}
\begin{markdown}
Introduction
============
## Section
### Subsection
Hello *Markdown*!

| Right | Left | Default | Center |
|------:|:-----|---------|:------:|
|   12  |  12  |    12   |    12  |
|  123  |  123 |   123   |   123  |
|    1  |    1 |     1   |     1  |

: Table
\end{markdown}
\end{document}

Next, run the LaTeXMK tool from our official Docker image on document.tex:

docker run --rm -v "$PWD"/workdir:/workdir -w /workdir witiko/markdown \
  latexmk -pdf -shell-escape -silent document.tex

Alternatively, you can install TeX Live (can take up to several hours) and use its LaTeXMK tool:

latexmk -cd -pdf -shell-escape -silent workdir/document.tex

A PDF document named workdir/document.pdf should be produced and contain the following output:

banner

Congratulations, you have just typeset your first Markdown document! 🥳

Use Markdown for continuous integration

Can't live without the latest features of the Markdown package in your continuous integration pipelines? It's ok, you can use our official Docker image as a drop-in replacement for the texlive/texlive:latest Docker image! The following example shows a GitHub Actions pipeline, which will automatically typeset and prerelease a PDF document:

name: Typeset and prerelease the book
on:
  push:
jobs:
  typeset:
    runs-on: ubuntu-latest
    container:
      image: witiko/markdown:latest
    steps:
      - uses: actions/checkout@v2
      - run: latexmk -lualatex document.tex
      - uses: marvinpinto/action-automatic-releases@latest
        permissions:
          contents: write
        with:
          title: The latest typeset book
          automatic_release_tag: latest
          prerelease: true
          repo_token: ${{ secrets.GITHUB_TOKEN }}
          files: document.pdf

In fact, this is how we automatically produce the latest documentation for the Markdown package.

Peek under the hood

Remember how we said that the Markdown package converts markdown markup to TeX commands? Let's see what that means and what we can do with this knowledge.

Using a text editor, create an empty text document named document.md with the following markdown content:

Hello *Markdown*! $a_x + b_x = c_x$

Next, run the Lua command-line interface (CLI) from our official Docker image on document.md:

docker run --rm -i witiko/markdown markdown-cli hybrid=true < document.md

We will receive the following output, where the markdown markup has been replaced by TeX commands:

\markdownDocumentBegin
Hello \markdownRendererEmphasis{Markdown}!
$a\markdownRendererEmphasis{x + b}x = c_x$
\markdownDocumentEnd

We can see right away that the Markdown package has incorrectly interpreted _x + b_ as an emphasized text. We can fix this by passing in the underscores=false option:

docker run --rm -i witiko/markdown markdown-cli hybrid=true underscores=false < document.md
\markdownDocumentBegin
Hello \markdownRendererEmphasis{Markdown}!
$a_x + b_x = c_x$
\markdownDocumentEnd

Much better! If the Markdown package ever surprises you, use the Lua CLI to peek under the hood and inspect the results of the conversion.

Further information

For further information, consult one of the following:

  1. The user manual for either the released version or the latest development version, which can be produced by interpreting the markdown.ins file using a Unicode-aware TeX engine, such as LuaTeX (luatex markdown.ins). The manual will reside in the file markdown.md and the CSS stylesheet markdown.css.
  2. The technical documentation for either the released version or the latest development version, which can be typeset by running the LaTeXMK tool on the markdown.dtx file (latexmk markdown.dtx) after installing the Markdown package. LaTeXMK should be included in your TeX distribution. The typeset documentation will reside in the file markdown.pdf.
  3. Tutorials and example documents by Lian Tze Lim at Overleaf:
  4. Journal articles published by TUGboat:
  5. Journal articles published by CSTUG Bulletin (in Czech and Slovak):
  6. Talks:
  7. Theses:

Acknowledgements

Logo Acknowledgement
I gratefully acknowledge the funding from the Faculty of Informatics at the Masaryk University in Brno, Czech Republic, for the development of the Markdown package in projects MUNI/33/12/2015, MUNI/33/1784/2020, MUNI/33/0776/2021, MUNI/33/1654/2022, and MUNI/33/1658/2022.
Extensive user documentation for the Markdown package was kindly written by Lian Tze Lim and published by Overleaf.
Support for content slicing (Lua options shiftHeadings and slice) and pipe tables (Lua options pipeTables and tableCaptions) was graciously sponsored by David Vins and Omedym.
Fixes for issues #359, #368, #401, #424, #440, and #474 were graciously sponsored by the International Software Testing Qualifications Board (ISTQB).

Contributing to the Development of Markdown

Apart from the example markdown documents, tests, and continuous integration, which are placed in the examples/, tests/, and .github/ directories, the complete source code and documentation of the package are placed in the markdown.dtx document following the literate programming paradigm. Some useful commands, such as building the release archives and typesetting the documentation, are placed in the Makefile file for ease of maintenance.

When the file markdown.ins is interpreted using a Unicode-aware TeX engine, such LuaTeX (luatex markdown.ins), several files are produced from the markdown.dtx document. The make base command is provided by Makefile for convenience. In markdown.dtx, the boundaries between the produced files are marked up using an XML-like syntax provided by the l3docstrip plain TeX package.

Running the LaTeXMK tool on the markdown.dtx file (latexmk markdown.dtx) after the Markdown package has been installed typesets the documentation. The make markdown.pdf command is provided by Makefile for convenience. In markdown.dtx, the documentation is placed inside TeX comments and marked up using the ltxdockit LaTeX document class. Support for typesetting the documentation is provided by the doc LaTeX package.

To facilitate continuous integration and sharing of the Markdown package, there exists an official Docker image, which can be reproduced by running the docker build command on Dockerfile (docker build -t witiko/markdown .). The make docker-image command is provided by Makefile for convenience.

Contributed Software

Links to contributed third-party software for the Markdown package are available in the contributions/ directory. The intention is to show interesting tools for the Markdown package and to give them wider exposure without taking responsibility for their development or maintenance.

Citing Markdown

When citing Markdown in academic papers and theses, please use the following BibTeX entry:

@article{novotny2017markdown,
  author  = {V\'{i}t Novotn\'{y}},
  year    = {2017},
  title   = {Using {M}arkdown Inside {\TeX} Documents},
  journal = {TUGboat},
  volume  = {38},
  number  = {2},
  pages   = {214--217},
  issn    = {0896-3207},
  url     = {https://tug.org/TUGboat/tb38-2/tb119novotny.pdf},
  urldate = {2020-07-31},
}

Alternatively, you can use the Novotny:2017:UMI key from the tugboat.bib BibTeX file that is included in your TeX distribution like this:

\cite{Novotny:2017:UMI}
\bibliography{tugboat}

markdown's People

Contributors

daurnimator avatar davidluptak avatar drehak avatar github-actions[bot] avatar jgm avatar kensanata avatar khaledhosny avatar lostenderman avatar mbertucci47 avatar texhackse avatar tst2005 avatar witiko avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

markdown's Issues

Error in Markdown bulleted lists: "Too deeply nested. \markdownRendererUlBeginTight"

First I want to say that I love the Markdown package. Thanks for putting this up here.

I'm a big fan of using bulleted lists in Markdown to write outlines. I'm trying to add an outline to a paper that I'm working on in Latex, but I keep getting the same error whenever I try to render it. Here's a complete example of what Texstudio will complain about:

003fc04eb7c4a930393129e07112a107.md.tex error line 19: Too deeply nested. \markdownRendererUlBeginTight

I dug around a little, and apparently there's a hard limit for list nesting depth built into TeX at some deep level. According to various sources (such as http://tex.stackexchange.com/questions/41408/a-five-level-deep-list) the way to work around it is to use the enumitem package. I tried adding some enumitem stuff to the preamble of my document:

\usepackage{enumitem}
\setlistdepth{20}
\renewlist{itemize}{itemize}{20}

% use dots for all levels
\setlist[itemize]{label=$\cdot$}

only now Texstudio is reporting a different set of errors:

003fc04eb7c4a930393129e07112a107.md.tex error line 2: Package enumitem Error: Undefined label. \markdownRendererUlItem 7
003fc04eb7c4a930393129e07112a107.md.tex error line 3: Package enumitem Error: Undefined label. \markdownRendererUlItem r
003fc04eb7c4a930393129e07112a107.md.tex error line 4: Package enumitem Error: Undefined label. \markdownRendererUlItem b
...

I think the enumitem solution is correct, but I'm such a Latex novice that I'm getting the syntax /placement wrong somewhere. Any suggestions on how to fix my fix for list depth?

If you give me a couple of pointers, I'd also be glad to get a pull request together so that deeper lists can be turned on as an option in the Markdown packaged itself, if you'd like.

How highlighting a code source ?

Hi,
Thanks for this package.
I don't understand how highlighting a code source in the generated PDF.

Exemple in my markdown I have a code block with xml "label" and in my pdf I have:
Result PDF

The xml code isn't highlighting and the format is not correct.

Thanks for your help.

TF.

`\markdownReadAndConvert` contains an unsolicited `\par`

Steps to reproduce bug

Create a text document named test.tex with the following content:

\input markdown
\input lmfonts
\markdownBegin
$\sqrt{-1}$ *equals* $i$.
\markdownEnd
\def\markdownOptionHybrid{true}
\markdownBegin
$\sqrt{-1}$ *equals* $i$.
\markdownEnd
\bye

Afterwards, execute LuaTeX:

$ luatex test.tex

Expected behavior

Executing LuaTeX should produce a PDF document named test.pdf with the following content:
correct

Actual behavior

Executing LuaTeX produces a PDF document named test.pdf with the following content:
wrong

Add support for TeX-like rich paragraphs

Introduction

The content model of markdown is modeled after the content model of HTML, which considers a paragraph to be an element that may only contain inline elements – such as emphasis, or inline code examples – but not block-level elements such as lists or code listings. [1]

The original markdown spec by Gruber [2] is not entirely explicit about this, noting only that

A paragraph is simply one or more consecutive lines of text, separated by one or more blank lines.

but the later CommonMark v0.28 spec [3] (which our Lua parser predates and not entirely complies with) makes this explicit:

We can think of a document as a sequence of blocks—structural elements like paragraphs, block quotations, lists, headings, rules, and code blocks. Some blocks (like block quotes and list items) contain other blocks; others (like headings and paragraphs) contain inline content—text, links, emphasized text, images, code spans, and so on.

Unlike HTML and markdown, TeX is happy to include vertical content inside paragraphs [4]. As a consequence, one can naturally include lists inside paragraphs in LaTeX:

The content model of both
\begin{itemize}
\item HTML, and
\item markdown
\end{itemize}
makes it impossible to put block-level elements inside a paragraph.

latex

In markdown, the following document will consist of a paragraph, an unordered list, and another paragraph:

The content model of both

- HTML, and
- markdown

makes it impossible to put block-level elements inside a paragraph.

After typesetting the document using the LaTeX implementation of the Markdown package, we get the following (wrong) result:
latex

A temporary solution in LaTeX is to add \noindent macros at the beginning of the markdown top-level blocks that do not correspond to actual paragraphs. Below, I will propose a more permanent solution.

The `testfiles/PHP_Markdown/email-auto-links.test` testfile fails with TL 2018 pretest

$ ./test.sh testfiles/PHP_Markdown/email-auto-links.test
Testfile testfiles/PHP_Markdown/email-auto-links.test
  Format templates/context/
    Template templates/context/input.tex.m4
      Command texexec  --passon="--shell-escape --8bit --interaction=nonstopmode" test.tex
      Command context                                              --nonstopmode  test.tex
    Template templates/context/verbatim.tex.m4
      Command texexec  --passon="--shell-escape --8bit --interaction=nonstopmode" test.tex
      Command context                                              --nonstopmode  test.tex
  Format templates/latex/
    Template templates/latex/input.tex.m4
      Command pdflatex --shell-escape                  --interaction=nonstopmode  test.tex
*** /dev/shm/4838/test-expected.log     2018-04-09 16:56:50.201620942 +0200
--- /dev/shm/4838/test-actual.log       2018-04-09 16:56:52.337640679 +0200
***************
*** 5,11 ****
  END link
  interblockSeparator
  BEGIN link
! - label: help@tūdaliņ.lv
! - URI: mailto:help@tūdaliņ.lv
  - title: 
  END link
--- 5,11 ----
  END link
  interblockSeparator
  BEGIN link
! - label: help@t\unhbox \voidb@x \bgroup \let \unhbox \voidb@x \setbox \@tempboxa \hbox {u\global \mathchardef \accent@spacefactor \spacefactor }\accent 22 u\egroup \spacefactor \accent@spacefactor dali\unhbox \voidb@x \setbox \z@ \hbox {n}{\lineskiplimit -\maxdimen \unhbox \voidb@x \vtop {\baselineskip \z@skip \lineskip .25ex\everycr {}\tabskip \z@skip \halign {##\crcr \unhbox \z@ \crcr \hskip \hideskip \char 24\hskip \hideskip \crcr }}}.lv
! - URI: mailto:help@t\unhbox \voidb@x \bgroup \let \unhbox \voidb@x \setbox \@tempboxa \hbox {u\global \mathchardef \accent@spacefactor \spacefactor }\accent 22 u\egroup \spacefactor \accent@spacefactor dali\unhbox \voidb@x \setbox \z@ \hbox {n}{\lineskiplimit -\maxdimen \unhbox \voidb@x \vtop {\baselineskip \z@skip \lineskip .25ex\everycr {}\tabskip \z@skip \halign {##\crcr \unhbox \z@ \crcr \hskip \hideskip \char 24\hskip \hideskip \crcr }}}.lv
  - title: 
  END link

Remove the Lua text buffering routine

No advantage results from having two implementations of the \markdownReadAndConvert command (one in plain TeX and the other using the LuaTeX process_input_buffer hook). By keeping only the plain TeX implementation, the ConTeXt Mark IV module (which fails to expose the process_input_buffer hook) will no longer require shell access and the overall code will become markedly more readable.

Hybrid + latex comments => bug

Hi,
I have found out, when using hybrid mode, that the lines below comments disappear. Here is MWE

\documentclass{article}

\usepackage[hybrid]{markdown}

\begin{document}
\begin{markdown}
% Comment
Hello _world_.

Second line.
\end{markdown}
\end{document}

Desired output

Hello world.
Second line.

Actual output

Second line.


It is a really annoying bug, because there is no error nor warning if the next line after the comment is not blank.
Is there an easy workaround in the lua code before the bug can be properly patched? Some latex construct are dependant on comments, so they can't just be stripped off.

https://www.sharelatex.com/read/ythksyknnwqb

Add `blankBeforeList` Lua option

Currently, the markdown package recognizes lists only when they are preceded by a blank line (or some other block-finishing element, I suppose). However, the list is not recognized, if it directly follows a regular text. There is no consensus about the correct Markdown behavior in that case: Should there be a list? Or should the text be continued? The markdown package continues the text.

Example:

Some text
- List item 1
- List item 2

It would be great to have an option such as blankBeforeBlockquote, but blankBeforeList to control this behavior.

Here are a few notes by Vit just to keep them in the record:

As for the leading blank line before lists, there is no consensus between different implementations of Markdown on this issue 1 2, and the Markdown spec leaves the behavior unspecified 3.

problem with # sign in url and hyperref

I hade a problem with the # sign in urls and the hyperref package. I asked for help at tex.stackexchange: https://tex.stackexchange.com/questions/418726/markdown-package-problem-with-sign-in-url-and-hyperref

Here the problem in short:

\usepackage[	
		blankBeforeBlockquote,
		blankBeforeCodeFence,
		codeSpans,
		citations,
		definitionLists,
		fencedCode,
		hybrid,
		underscores,
		inlineFootnotes,
		hashEnumerators	]{markdown}

\usepackage{hyperref}

\begin{document}
\begin{markdown}

Here's a link to [a website](https://developer.android.com/training/testing/fundamentals.html#testing-pyramid).  

\end{markdown}
\end{document}

If I compile the above code, I get following error message.

Illegal parameter number in definition of \Hy@tempa.

Ulrike Fischer had a quite nice solution for me. Maybe it is easy for you to incorporate this solution in your package.

Her solution, with a little tweak of mine:

{\catcode35=12 [a website](urlwith#sign)}

\begin{markdown} env not working with latex

In document.tex:

\documentclass{article}
\usepackage{markdown}
\begin{document}
	\begin{markdown}
		Hello *world*!
	\end{markdown}
\end{document}

Either pdflatex --shell-escape document.tex or lualatex document.texresults in document.pdf

Hello *world*!

But if I try

\documentclass{article}
\usepackage{markdown}
\begin{document}
\markdownInput{document.md}
\end{document}

with"Hello *world*!" written in document.md , both pdflatex and lualatex work pretty well rendering

Hello world!

My sys info:

$ pdflatex --version
pdfTeX 3.14159265-2.6-1.40.20 (TeX Live 2019)
kpathsea version 6.3.1
Copyright 2019 Han The Thanh (pdfTeX) et al.
There is NO warranty.  Redistribution of this software is
covered by the terms of both the pdfTeX copyright and
the Lesser GNU General Public License.
For more information about these matters, see the file
named COPYING and the pdfTeX source.
Primary author of pdfTeX: Han The Thanh (pdfTeX) et al.
Compiled with libpng 1.6.36; using libpng 1.6.36
Compiled with zlib 1.2.11; using zlib 1.2.11
Compiled with xpdf version 4.01
$ lualatex --version
This is LuaTeX, Version 1.10.0 (TeX Live 2019)

Slice without explicit html attributes

It is possible to use the slice option without using the explicit html attributes?

The reason I ask is that I have a markdown file I want to include in a latex document, but I can't make changes to the markdown file, since it is also used elsewhere, where the special syntax for html attributes is not supported.

I did find issue #28, but that syntax doesn't work (as is expected, since it's not in the user manual).

Rewrite technical documentation from LaTeX to Markdown

Since 665af33, it is possible to use the Markdown package to typeset documentation. Rewrite the existing technical documentation from LaTeX to Markdown at the user-manual branch. Typeset the documentation using the Markdown package itself. Note that not only will this make the documentation more readable, but it will also allow us to share text between the technical documentation and the user manual developed as a part of #19.

Incorrect vertical spaces between list elements

Hi, thanks to all for this extension! It's wonderful. Nevertheless, I've got a problem with it.
Here's a MWE :

\documentclass{report}
\usepackage{markdown}
\usepackage{lipsum}
\begin{document}
\begin{itemize}
	\item \lipsum[1][1]
	\item \lipsum[1][2]
	\item \lipsum[1][3]
\end{itemize}
\begin{markdown*}{hybrid}
- \lipsum[1][1]
- \lipsum[1][2]
- \lipsum[1][3]
\end{markdown*}
\end{document}

The space between list elements is not the same outside and inside the markdown environment. I'm using Lualatex with Texlive 2018.20190227 (stock Texlive from Kubuntu 19.04).

error with beamer using this package

I use pdflatex to compile a tex file using this markdown package, but it gives a parse error.

/usr/share/texlive/texmf-dist/tex/latex/markdown/markdown.sty:247: Package keyval Error: citations undefined.

See the keyval package documentation for explanation.

It located at \end{markdown}.

I use the tex file from here:

\documentclass{beamer}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{mathabx}
\usepackage{mathpazo}
\usepackage{eulervm}
\usepackage{natbib}

%% Load the markdown package
\usepackage[citations,footnotes,definitionLists,hashEnumerators,smartEllipses,tightLists=false,hybrid]{markdown}
\markdownSetup{rendererPrototypes={
 link = {\href{#2}{#1}},
 headingThree = {\begin{frame}\frametitle{#1}},
 headingFour = {\begin{block}{#1}},
 horizontalRule = {\end{block}}
}}


\usetheme{Dresden}
\usefonttheme{serif}
\usecolortheme{rose}


\title{Writing Beamer Slides with Markdown}
\author{LianTze Lim}
\institute{Overleaf}

\begin{document}

\maketitle

\frame{\tableofcontents}

\begin{markdown}

# How?

### We can do that?

- Yeah to some extent, with \texttt{markdown} package :-)
    - __$\hash$__ and __$\hash\hash$__ for section and subsection headers (in ToC)
    - Redefine __$\hash\hash\hash$__ to start a frame and frametitle
    - (Nested) bullet and numbered lists
    - Text formatting (*italic*, **bold becomes italic + alerted**) 
    - Redefine __$\hash\hash\hash\hash$__ to start a block with title \linebreak
      and __\texttt{-{}-{}-{}-}__ to end the block
    - ___Compile with \texttt{-{}-shell-escape}___ (Overleaf does this already)
- (Alternative approaches: Pandoc, wikitobeamer)

\end{frame}

%%%%%%%%%%%%%%%%%%%%%%

### Caveats

- Nothing too complicated! 
- No verbatim or fragile stuff!
- No $\hash$ and \textunderscore{} characters!\linebreak 
  (I used `$\hash$` and `\textunderscore`)
- Can't pass options to frames
- __Need to write \texttt{\textbackslash end\string{frame\string}} manually!__

\end{frame}

%%%%%%%%%%%%%%%%%%%%%%


%%% # and ## are still sections and subsections
# Example

## Proposed Menus

%%% ### starts a frame + frametitle
### Breakfast Menu

%%% bulleted lists as usual 

- Eggs
    * scrambled
    * sunny-side-up
- Coffee
    * Americano
    * Long black
- Tea
    * Darjeeling
    * English Breakfast

%%% Due to the complicatedness of beamer frames, \end{frame} MUST appear in the source code itself and cannot be "hidden" in another command

\end{frame}

%%%%%%%%%%%

### Lunch Menu

- Spaghetti
    * Bolognese
    * Aglio olio
- Sandwiches
    * Egg
    * Ham
    * Tuna

\end{frame}

%%%%%%%%%%%

## Budgeting

### Projected Profit

1. And the answer is...
2. $f(x)=\sum_{n=0}^\infty\frac{f^{(n)}(a)}{n!}(x-a)^n$
    #. How do we _know_ that?
    #. __Maths!__

\end{frame}

### Testing blocks

#### This is a block!

- Here is some content.
- Here's more contents.

---

\end{frame}


### Citations

- This is a book [@BookKey]
- This is an article [@ArticleKey]

\end{frame}

\end{markdown}

\begin{frame}
\renewcommand{\bibfont}{\footnotesize}
\frametitle{Bibliography}

\bibliographystyle{apalike}
\bibliography{refs}

\end{frame}


\end{document}

Add a separate `experimentalOptions` table

Introduction

Since the release of Markdown v2.0.0, a large number of new options was added. However, to retain backwards compatibility, most of these options are disabled by default. This makes it difficult to enable everything back again for a user that is not concerned with backwards compatibility and wants to use the Markdown package with all the bells and whistles.

Proposed solution

Add a separate experimentalOptions table that will have the same structure as the current defaultOptions table, but that will enable the majority of new options and syntax extensions. Add a Lua option experimental that will make the Lua parser use the options in the experimentalOptions table as the defaults.

Typo in File markdown.lua

Hey all,

I am using the markdown package with MikTeX, using the editor TexStudio.

While compiling my tex-file, I got no errors, but once I opened the pdf, where the Markdown-Output was supposed to be, there was only the message "[path]/MiKTeX/2.9/tex/luatex/markdown/markdown.lua:1212: attempt to call upvalue ’B’ (a nil value)" instead.

Checking line 1212 of markdown.lua revealed that 'B' was indeed nowhere defined at all. Instead I found a lot of instances of the function 'C' instead.
And lo, once I just tried changing the 'B' to 'C', it works perfectly (after ~10 tries with different files).
I have honestly no idea what 'C' does.

Now, I admit openly that I'm not well versed with Lua, scratch that, I have no idea at all about Lua, but this must be the most amusing and puzzling bug I've ever seen. For the moment, I'm just gonna put it down to a typo. It would be interesting to see if anyone else has/had this problem. Maybe it also has to do with how I interpreted the markdown.ins file? No idea, really.

Lines 1211-1214 of markdown.lua:

parsers.between = function(p, starter, ender)
local ender2 = B(parsers.nonspacechar) * ender
return (starter * #parsers.nonspacechar * Ct(p * (p - ender2)^0) * ender2)
end

Changed to:

parsers.between = function(p, starter, ender)
local ender2 = C(parsers.nonspacechar) * ender
return (starter * #parsers.nonspacechar * Ct(p * (p - ender2)^0) * ender2)
end

The ConTeXt Mark II module mishandles active characters from `\enableregime`

\enableregime[…] is the ConTeXt equivalent of the LaTeX \usepackage[…]{inputenc}; both make characters above the ASCII range active. Currently, the scanning routine of \startmarkdown … \stopmarkdown is not prepared for this and expands these characters. This produces unwanted macros in the input for Lunamark if the Mark II format is used.

\usemodule[t][markdown]
\enableregime[utf]
\usetypescript[modern][ec]
\setupbodyfont[10pt,rm]
\starttext
\startmarkdown
Příliš žluťoučký kůň úpěl ďábelské ódy.
\stopmarkdown
\stoptext

scrot

The ConTeXt module should redefine \markdownMakeOther like the LaTeX package does to temporarily switch these characters to the category of other during the scanning.

sections/headings numbering

This is a feature request, rather than an issue.

It seems to me that, with the present implementation, markdown sectioning commands (#, ##, etc) yield a numbered heading in the LaTeX conversion (Chapter 1, Section 1.1, etc).

Can you please provide an option so that there is no numbering, that is, to have the starred version of the sectioning commands (like \chapter*{})?

Make enumerators before headers optional

I would like titles to not begin with numbers.
I didn't find a way to achieve this reading through the documentation. Would that be possible ?

Here is an example :

\begin{markdown}
## H1
\end{markdown}

Gives this :

1 H1

Where I'd like

H1

File transclusion

Recently released iA Writer has interesting extension to Markdown, they call it content blocks. This enables easy inclusion of images, tables, source code and child documents. For example:

/Savings Account.csv 'Recent Transactions'
/Example.swift
/Lorem Ipsum.txt

would create the table from file Savings Account.csv, include Example.swift as source code and include contents of Lorem Ipsum.txt processed with Markdown.

More details can be found in this post, there is also in depth description.

I guess that it wouldn't be easy to support for example the syntax highlighting and CSV inclusion in all TeX engines, but I like the idea. What do you think?

Write a user manual

The current documentation is too technical to be useful for the average end user. A separate document, which illustrates the use of the package by example, needs writing.

Support for markdown tables

Thank you for providing this package! I tried to insert a markdown table into a latex document, the table below was not rendered.

Letters Numbers
a 1
b 2

Here is the tex file I wrote.

\documentclass{article}

\begin{document}

\begin{markdown}

## A table

| Letters | Numbers |
| ----    | ---     |
|   a     |  1      |
|   b     |  2      |

\end{markdown}

\end{document}

The output came out like this | Letters | Numbers | | —- | — | | a | 1 | | b | 2 | when I ran pdflatex -shell-escape file.tex.

Thanks again

How can I have two separated quotes?

If I want to separate two quotes, I usually insert into my markdown files an empty html comment <!-- -->.

For example

> quote 1
<!-- -->
> quote 2

This will give me

quote 1

quote 2

How can I do this in my latex document?

\documentclass{article}
\usepackage{markdown}

\begin{document}
\begin{markdown}
> quote 1
<!-- -->
> quote 2
\end{markdown}
\end{document}

My current workaround is

\documentclass{article}
\usepackage{markdown}

\begin{document}
\end{markdown}
> quote 1
\end{markdown}
\begin{markdown}
> quote 2
\end{markdown}
\end{document}

However, I would prefer to have a consistent experience of my markdown data. I do not like to adapt it for a tex file inclusion.

Image resizing

As discused in #49 about the image example.

I have a lot of images in my markdown files but when I try to resize them the parser does nothing. Reading the documentation the parser only takes the name of the file and adjust the size to \textwidth. Is there any adittional option to include in order to resize any included image in markdown?
As far as I know an image can be resized with:

![example image](example-image.png "An example image from Martin Scharrer's mwe package" =250x250)

Superscript and Subscript

I am using version 2.5.5 of this package with Overleaf, and I can't for the google of me figure out the syntax for superscript and subscript. Pandoc markdown syntax such as, "water is H~2~O" and "21^st^ Century" do not work. The ~ get ignored, and the ^ throws a compile error. Any thoughts?

Allow Jekyll Frontmatter

Jekyll Frontmatter (A YAML block before actual markdown text) is commonly used with github pages and markdown.

It would be great to be able to

  • have Frontmatter in included Markdown files and maybe
  • set a callback/command to process the contents.

Background: I have some people giving me tex files that includes things like authors, title, etc. I really would like to have that as markdown, but cannot deal with the metadata.

E.g., currently I get

\author{John Doe}
\title{The Theory of Everything}
\maketitle
\begin{abstract}
foo
\end{abstract}

Lorem \emph{ipsum} ...

But I would like

---
author: John Doe
title: The Theory of Everything
abstract: >
  foo
---

Lorem *ipsum* ...

And then turn the latter into the forme

MacTeX ! Package markdown Error: I can not access the shell.

Hello,

I'm using MacTeX and installed the mardown package via tlmgr install markdown

When I attempt to print a simple PDF with embedded markdown, I receive and error:
! Package markdown Error: I can not access the shell.

I've been searching for an answer in any similar projects, but have not found anything helpful. Do you have any suggestions?

Thank you!

Nested ordered lists

One feature I am seeking for constantly is nested ordered lists. It seems to me that any list such as

1. test
    1. dodo
    2. gaga
2. more test

is always redered exactly as above. What I would like to achieve, is that the indented list is redered as

1. test
    a) dodo
    b) gaga
2. more test

I believe LaTeX should do this automatically, but the generated temporary code hardcodes the numbering by \markdownRendererOlItemWithNumber{1}.

Is there a way to achive the desired behavior?

And as a side note: I often also would like to have the first level of the enumerated list appear as a), b), c). This is so as to mark alternative options and not a consecutive number of steps to take. It would be perfect, if I could also achieve that...

Thanks for you help!

Integrate with jgm/pandoc

Currently, a user of the Markdown package is restricted in their choice of syntax extensions to the ones provided by the Lua parser implemented in markdown.lua. To provide experimental ground for implementing new syntax extensions, support for the internal abstract syntax tree (AST) format of the jgm/pandoc converter will be added.

Currently, jgm/pandoc can be used to provide conversion from various input formats to Markdown:

pandoc -f docx -t markdown input.docx -o input.md

The Markdown package can then be used to convert the Markdown document to the TeX abstract syntax tree format (TeX AST) produced by the Markdown package:

texlua /path/to/markdown-cli.lua input.md input.tex

This representation can then be typeset. This is useful, but limited to the Markdown syntax extensions supported by our Lua parser.


The plan is to provide a jgm/pandoc Lua writer (see jgm/pandoc issues 4341 and 1541 for futher information) that will directly convert the jgm/pandoc AST to the TeX AST, circumventing the Lua parser altogether:

pandoc -f docx -t /path/to/markdown-pandoc_writer.lua input.docx -o input.tex

Adding initial support for a new syntax extension already supported by jgm/pandoc will then be as easy as adding a new procedure to the writer and defining the corresponding \markdownRenderer… macros. Full support can be added later by extending our Lua parser.


TODOs:

Add a command-line interface

Until now, the conversion from Lua to TeX would typically be performed transparently by an upper-level TeX interface. To provide finer control over the conversion and to simplify debugging, a command-line Lua interface is now also provided:

texlua /path/to/markdown-cli.lua outputDir=. -- input.md output.tex

The use of the command-line interface makes it unnecessary to provide shell access to the TeX engine.

The current implementation of the Lua command-line interface is documented in sections 2.1.3 (interface) and 3.1.7 (implementation) of the technical documentation.

Add support for content cherry-picking

Introduction

Users typesetting proceedings, almanacs, and other kinds of composite documents from external markdown documents would benefit from an option to cherry-pick only select portions of a markdown document. I am considering adding the sliceBegin and slideEnd Lua parser options, which would each accept a string in the following format (given in BNF): {[^|$]ATX_HEADER|^|$}. The individual values would carry the following meaning:

  • ^ – the beginning of the document (the default value of sliceBegin),
  • ^ATX_HEADER – the beginning of section ATX_HEADER,
  • ATX_HEADER – corresponds to ^ATX_HEADER for sliceBegin and to $ATX_HEADER for sliceEnd,
  • $ATX_HEADER – the end of section ATX_HEADER,
  • $ – the end of the document (the default value of sliceEnd).

Examples

Consider the following example document document.md:

Chapter 1
========
Some text.

Section 1.1
-----------
Some other text.

Section 1.2
-----------
Yet another text.

Below are example outputs of the \inputMarkdown macro for various values of the sliceBegin and sliceEnd options:


\inputMarkdown[sliceBegin=^, sliceEnd=$]{document.md}

Chapter 1

Some text.

Section 1.1

Some other text.

Section 1.2

Yet another text.


\inputMarkdown[sliceBegin=## Section 1.1]{document.md}

Section 1.1

Some other text.

Section 1.2

Yet another text.


\inputMarkdown[sliceEnd=## Section 1.1]{document.md}

Chapter 1

Some text.

Section 1.1

Some other text.


\inputMarkdown[sliceBegin=^# Chapter 1, sliceEnd=^## Section 1.1]{document.md}

Chapter 1

Some text.


\inputMarkdown[sliceBegin=^# Chapter 1, sliceEnd=$## Section 1.1]{document.md}

Chapter 1

Some text.

Section 1.1

Some other text.

Improve logging

The logging support is rudimentary:

  1. All messages go directly into the standard output of the TeX compiler.
  2. Lua errors are not caught and get propagated to the user in different ways depending on the compiler:
    • LuaTeX will properly catch the errors, but
    • with shell bridge, the Lua errors will manifest themselves in the typeset output.
  3. The creation of auxiliary files and directories is not properly logged, so external tools like LaTeXMK may not do a proper clean-up.

The testfile `testfiles/lunamark/unicode.test` testfile fails with TL2018 pretest

$ ./test.sh testfiles/lunamark/unicode.test
Testfile testfiles/lunamark/unicode.test
  Format templates/context/
    Template templates/context/input.tex.m4
      Command texexec  --passon="--shell-escape --8bit --interaction=nonstopmode" test.tex
      Command context                                              --nonstopmode  test.tex
    Template templates/context/verbatim.tex.m4
      Command texexec  --passon="--shell-escape --8bit --interaction=nonstopmode" test.tex
      Command context                                              --nonstopmode  test.tex
  Format templates/latex/
    Template templates/latex/input.tex.m4
      Command pdflatex --shell-escape                  --interaction=nonstopmode  test.tex
*** /dev/shm/19996/test-expected.log    2018-04-09 17:22:13.813482591 +0200
--- /dev/shm/19996/test-actual.log      2018-04-09 17:22:15.621494380 +0200
***************
*** 1,5 ****
  BEGIN link
! - label: Beiß nicht in die Hand, die dich füttert.
  - URI: http://en.wikiquote.org/wiki/German(underscore)proverbs
  - title: 
  END link
--- 1,5 ----
  BEGIN link
! - label: Bei\OT1\ss  nicht in die Hand, die dich f\unhbox \voidb@x \bgroup \let \unhbox \voidb@x \setbox \@tempboxa \hbox {u\global \mathchardef \accent@spacefactor \spacefactor }\accent 127 u\egroup \spacefactor \accent@spacefactor ttert.
  - URI: http://en.wikiquote.org/wiki/German(underscore)proverbs
  - title: 
  END link

Add support for package documentation

Currently, the package cannot be used for package documentation using the doc LaTeX package and similar, since it does not support stripping percent signs from the beginning of lines. Package documentation a place where an easy-to-read format such as Markdown would shine. Also in relation to the user manual (see #19), which is being written in Markdown, support for Markdown in package documentation would make it possible to easily share text between the user manual and technical documentation.

HTML entities with the `html` Lua option enabled produce a Lua parser error

Steps to reproduce bug

Create a text document named test.tex with the following content:

\input markdown
\def\markdownOptionHtml{true}
\markdownBegin
&#33;&excl;&#x00021;
\markdownEnd
\bye

Afterwards, execute LuaTeX:

$ luatex test.tex
This is LuaTeX, Version 1.0.4 (TeX Live 2017) 
 restricted system commands enabled.
(./test.tex (/home/witiko/texmf/tex/generic/markdown/markdown.tex)/home/witiko/
texmf/tex/luatex/markdown/markdown.lua:493: bad argument #1 to 'char' (number e
xpected, got nil)
stack traceback:
        [C]: in function 'char'
        /home/witiko/texmf/tex/luatex/markdown/markdown.lua:493: in function </home/wi
tiko/texmf/tex/luatex/markdown/markdown.lua:491>
        [C]: in function 'match'
        /home/witiko/texmf/tex/luatex/markdown/markdown.lua:1293: in function 'parse_b
locks_toplevel'
        /home/witiko/texmf/tex/luatex/markdown/markdown.lua:1930: in function 'transfo
rm'
        /home/witiko/texmf/tex/luatex/markdown/markdown.lua:117: in function 'cache'
        /home/witiko/texmf/tex/luatex/markdown/markdown.lua:1929: in function 'convert
'
        [\directlua]:1: in main chunk.
\markdownLuaExecute ...local print = tex.print #1}
                                                  
...arkdownInput \markdownOptionInputTempFileName 
                                                  \markdownEnd 
l.5 \markdownEnd

Expected behavior

Executing LuaTeX should produce a PDF document named test.pdf containing three exclamation marks (i.e. !!!).

Actual behavior

A Lua error is displayed and no output is produced.

Improve ConTeXt interface

The ConTeXt interface would benefit from a proper user interface such as the one LaTeX has. Being an academician first and a TeX enthusiast second, I have only had cursory exposure to ConTeXt. Therefore, I don't feel I understand the ConTeXt philosophy and syntax conventions enough to design a user interface for it. This leaves the ConTeXt users of the Markdown package with the plain TeX interface, which is not ugly per se, but which does feel out of place in a ConTeXt document.

See also the accurate description of the status quo by on the ConTeXt mailing list by @mojca:

Hi,

There's a really nice module for processing markdown:
https://github.com/Witiko/markdown
that has been presented during the TUG meeting and is included in TeX Live.

It theoretically works for ConTeXt (the output is perfectly usable),
but it needs a cleanup of the user interface to turn the options on
and off.

For example, it would be nice to use something like

\setupmarkdown
    [fencedCode=true]

rather than

\def\markdownOptionFencedCode{true}

Is anyone willing to look into the code and help the author fix the interface?

(I can add it to our distribution if needed.)

Mojca

I will appreciate any suggestions on how the user interface for ConTeXt should be redesigned to better fit in with the rest of ConTeXt and which ConTeXt mechanisms should be used to implement the suggested redesign.

Fenced code blocks are being misparsed as nested lists

The library incorrectly detects fenced code blocks beginning with either a hyphen or an asterisk as nested lists. A document such as

This is a list:

 - This is a fenced code block:
    ```infostring
 - some text beginning with either - or *
    ```

will be parsed as two lists.

Tabs are stripped even with the `preserveTabs=true` Lua option enabled

Steps to reproduce bug

Create a text document named test.tex with the following content:

\input markdown
\input lmfonts
\catcode`\ =13
\catcode`\^^I=13
\def {\ (SPACE)\ }
\def^^I{\ (TAB)\ }
\input{optionfalse}

\input{optiontrue}
\bye

Using a text editor, create a text document named content.md with the following content. Replace the graphical tab characters (↹) with actual tab characters:

These words are separated by spaces and these↹are↹separated↹by↹tabs.

Afterwards, execute LuaTeX:

$ texlua `kpsewhich markdown-cli.lua` -- content.md optionfalse.tex
$ texlua `kpsewhich markdown-cli.lua` preserveTabs=true -- content.md optiontrue.tex
$ luatex document.tex

Expected behavior

Executing LuaTeX should produce a PDF document named test.pdf with the following content:

These (SPACE) words (SPACE) are (SPACE) separated (SPACE) by (SPACE) spaces (SPACE) and
(SPACE) these (SPACE) are (SPACE) separated (SPACE) by (SPACE) tabs.

These (SPACE) words (SPACE) are (SPACE) separated (SPACE) by (SPACE) spaces (SPACE) and
(SPACE) these (TAB) are (TAB) separated (TAB) by (TAB) tabs.

Actual behavior

Executing LuaTeX produces a PDF document named test.pdf with the following content:

These (SPACE) words (SPACE) are (SPACE) separated (SPACE) by (SPACE) spaces (SPACE) and
(SPACE) these (SPACE) are (SPACE) separated (SPACE) by (SPACE) tabs.

These (SPACE) words (SPACE) are (SPACE) separated (SPACE) by (SPACE) spaces (SPACE) and
(SPACE) these (SPACE) are (SPACE) separated (SPACE) by (SPACE) tabs.

Support for inline footnotes

It would be nice to add support for inline footnotes. As usual in Markdown world, several forms exists:

Here is an inline note.^[Inlines notes are easier to write, since
you don't have to pick an identifier and move down to type the
note.]

Some text with a footnote[^This is the footnote itself.].

I met Jim [^](My old college roommate) at the station.

As Pandoc is the de facto Markdown standard, it seems like the best idea to support this variant.

I've tried to add support by myself, but failed. This is my code:

  function self.inline_note(note)
    return function()
      return writer.note(parse_inlines(note))
    end
  end
  -- I've tried two variants of the pattern
  -- local InlineNote = (circumflex * lbracket) * tag / self.inline_note
  local  InlineNote = circumflex * lbracket * C((anyescaped - rbracket)^1) * rbracket / self.inline_note

I've added the InlineNote to Inline grammar. The converted result is:

\markdownRendererCircumflex{}[footnote text]

Strange is, if I tried to support the iA Writer variand ([^footnote text]) with

InlineNote = lbracket * circumflex * C((anyescaped - rbracket)^1) * rbracket / self.inline_note 

then it works, but this definition causes normal footnotes to fail, so it is not correct as well.

There is also one issue I faced, there are too many local variables in the parsing function, this is why I used function self.inline_note, when I tried local function inline_note, I've got this runtime error:

error loading module markdown from file ./markdown.lua:
./markdown.lua:1245: too many local variables (limit is 200) in function at line 413 near '='

This means that it would be quite hard to add more patterns to the parser, maybe the parsing functions should be stored in some table (I used self, but dedicated table should be better?).

Paragraph after bullet list is not indented correctly

Consider the following code:

\usepackage[hybrid, underscores=false, citations]{markdown}
\begin{markdown}
  - list 1

  paragraph

  - list 2
\end{markdown}

This renders like so:
image

I would expect the second paragraph to be indented in the same way as the first one.

Smart Quote support?

I can not seem to enable smart quotes for my long passages with lots of quotes, and the quotes package for LaTeX is not compatible with the markdown package. Do you have any advice?

newenvironment with markdown

Hi, and thanks a lot for this package!
I'm trying to define a new environment with markdown:

\newenvironment{todo}
   {\begin{markdown}
   }
   { 
   \end{markdown}
   }

But I got this message:

Runaway argument?
! File ended while scanning use of ^^M.

Extract Lua code execution in pdfLaTeX to an external package

The package has the following two responsibilities at the moment:

  • processing and rendering Markdown input and
  • executing Lua code in engines without direct Lua support.

The latter responsibility could be extracted into a separate package. This will decrease the internal complexity of the markdown package and provide this useful functionality to the wider TeX community.

Enhancement - disable underscore intra-word emphasis

I tried to write some hybrid markdown-latex text and I quickly encounter problem with math subscripts. I thought about some ways how to deal with that problem, but the easiest solution I found is to disable intra-word emphasis with underscores. This is the "default" in most implementations and specifications, so there is no deviation from standard either.

Because there are more ways how to do it I am proposing it as an enhancement rather than pull request.

This is my naive sketch how it could be implemented.

parsers.punctuation            = S(",.?!;")
larsers.Emph   = ( parsers.between(parsers.Inline, parsers.asterisk,
                                     parsers.asterisk)
                   + parsers.between(parsers.Inline, (B(parsers.punctuation + parsers.spacing + parsers.asterisk) * parsers.underscore),
                                     (parsers.underscore * #(parsers.punctuation + parsers.spacing + parsers.asterisk + parsers.eof)))
                   ) / writer.emphasis

The same situation is with the strong emphasis.

PS: There is a known bug I don't know how to deal with. It's how to match beginning of string (for look behind), so this version fails when asterisk is the first symbol in the string.

Fix input buffering in ConTeXt MkIV

Since the LuaTeX process_input_buffer callback is frozen in ConTeXt Mark IV, a write18 workaround is used until a proper patch is prepared. Since the file descriptor 18 is no longer defined in LuaTeX 0.87+, this workaround no longer works in TeX Live '16 and needs to be replaced with either:

  • a similar workaround using the Lua os.execute method directly, or
  • a proper patch that makes use of the ConTeXt buffers.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.