Inline with executablebooks/mdit-py-plugins#31 and executablebooks/mdit-py-plugins#32

then eventually move them into a separate package

Myst is designed to be CommonMark compliant, therefore standard Markdown syntax highlighters will work with Myst and handle the common syntaxes. However, it would be desired to have an 'extended' syntax highlighter, to cover the additional syntax introduced by Myst.

The de facto standard for syntax grammar is the TextMate grammar file, with support in Eclipse, Atom, VSCode, Sublime, etc (jupyterlab/jupyterlab#5504 also discusses their potential incorporation). Therefore, it would be good to have a reference implementation (tested somehow?) in this package.

Here are some helpful links:

• https://www.apeth.com/nonblog/stories/textmatebundle.html
• https://benparizek.com/notebook/notes-on-how-to-create-a-language-grammar-and-custom-theme-for-a-textmate-bundle
• https://medium.com/@model_train/creating-universal-syntax-highlighters-with-iro-549501698fd2

Similar to how directive snippets are implemented

In #27 tests were pinned to this version, since there was failures using the latest version.

I should:

• find the newest version that does still and at least update to that,
• work out what the actual issue is with the newest version
• think of the best way to pin to a vscode version, but also point out when this is getting old. Or maybe something like running for a pinned version and runnning for the latest version (but allowing this to fail)

Awaiting microsoft/vscode#121256

Describe the bug

The equations in a math directive do not render, whereas inline and math blocks do.

To Reproduce
Type:

f(x)=1

The result is

Expected behavior
To render as the following does:
$$
f(x)=1
$$

Desktop (please complete the following information):

• OS: macOS Monterey 12.6 (21G115)
• VSCode: 1.75.0 (Universal)
• MyST-Markdown extension: v0.11.0

When using some Linux installation of VSCode or Codium, it's bind to https://open-vsx.org instead of the marketplace.

Open-VSX teams provides publication instructions on https://github.com/eclipse/openvsx/wiki/Publishing-Extensions.

If this require to much work, it's possible to fill a pull request in this repository: https://github.com/open-vsx/publish-extensions/

I am considering to port a long lecture note (about astronomical data reduction coding) in LaTeX to jupyter book using myst.
I have many places where I have to use {numref}`fig-figurename`, but many times I don't remember fig-figurename for hundreds of figures.
So, the feature I'd love is the autocompletion of references.

You can extend the VS Code Markdown previewer, via markdown-it plugins: https://code.visualstudio.com/api/extension-guides/markdown-extension

Originally posted by @chrisjsewell in #18 (comment)

Thanks @CodeCox.
I have to say, I have not touched this project in months (although I do use it daily within VS Code for the syntax highlighting).

Since then, there has been many, many changes to myst; notably (since vs-code uses markdown-it) the creation of markdown-it-py as the backend to myst-parser, the addition of extended support for a number of markdown-it plugins, and the creation of markdown-it-myst.

I hope to circle back around to this project then (when time permits!), to re-appraise it and integrate all these new developments in.

Ling story short, this should fix the bug!

I am super excited to give this a try :-)

however, vscode tells me I can't install it because I'm running the insider version of it. Is it trivial to support insider versions? (I'm in 1.38-insider), or do you recommend I move to the stable branch?

(I think it's fine for me to move to stable if that's the best option, I had switched to insider a few months back because of some improvements in their remote server plugin, but I think those are in stable now)

The preview for the code-block directive needs some improvements. Currently, the pdf output looks quite different:

No caption is shown. No line numbers are shown. No lines are highlighted.

Markdown in directives/blocks like {margin} and {admonition} does not get rendered

Would it be sufficient to add |margin|admonition in this line in the syntax file?

The Markdown Preview is not correctly rendering the MyST enhancements.
The following enhanced markdown is ignored and only the content is rendered as plain text.

```{tip}
this is a MyST tip that fails!
```

The MyST-Markdown extension is enabled since highlighting, intellisense, etc works on the enhanced MyST. However it seems that the enhanced Preview part is either not working or not activated. None of the enhanced MyST markup is previewing as expected. The config setting is ticked and VSCode has been restarted many times. I have even added "myst.preview.enable": true to my settings.json as a hack but with no effect. Also this same file is correctly rendered by jupyter-book (jb build/page ...)

PS. The marketplace advertising blurb of MyST-Markdown shows this feature working but Issue #5 seems related and is still open?

Screenshots

• Screenshot of Extension config in VSCode:extension
  

• Screenshot of Edit and Preview panes in VSCode:
  

• Screenshot from browser with MyST rendered as expected
  

Desktop

• Windows 10, and Windows 10 with WSL2 Ubuntu
• VSCode 1.48.2
• MyST-Markdown v0.10.2

(that's all)

It would need a browser entry-point, plus (the tricky part) all modules packaged to a single file

See https://code.visualstudio.com/api/extension-guides/web-extensions#web-extension-main-file

This may be out of scope of this project or perhaps I am misreading documentation on how to do this, but wanted to check: is it possible for additional installed sphinx extensions to have these rendered?

for example have https://sphinx-panels.readthedocs.io/en/latest/ installed and using "dropdown" directive like below:

which currently renders the unknown directive:

if this is out of scope with this project, curious how others do local development within vscode to help render extensions like this, only other project seen is like: https://docs.restructuredtext.net/

Is your feature request related to a problem? Please describe.
It would be great if we could adapt the code execution for Python chunks using the similar implement as vscode-python. With that extension, you could press Cmd+Enter to run the code within the kernel in the editor. This would give a great level of interactivity while writing in MyST.

Describe the solution you'd like
Add code execution capability to MyST extension.

Describe alternatives you've considered
It may be more desirable to add this directly vscode-python. It currently supports jupyter notebook and python scripts with percent sign code chunk delimiters (%%). perhaps the addition of ```python syntax there would be more applicable.

Describe the bug

@chrisjsewell a few of the quantecon community are reporting issues installing / getting this this extension to work on linux platforms. I will setup a Linux VM to explore the root cause a bit further to make this more useful but thought I would let you know in the meantime.

To Reproduce

Coming

Desktop (please complete the following information):

• OS: [e.g. iOS] Linux (Arch)

Describe the bug
Inline roles in pipeline tables seem to cause the preview engine to freeze.

To Reproduce

1. Download this file and remove the .txt extension. (Github wouldn't let me upload a .md file.)
   roles-in-tables-bug.md.txt
2. Open it in VS Code.
3. Run the command Markdown: Open Preview to the Side

Expected behavior

The contents of the file should be displayed in a markdown preview on the right side.

Actual behavior:

If the preview has never been loaded for the file, the preview pane remains blank.
If the preview has been loaded previously for the same file, it stops updating the new content.

Screenshots

Here is a screenshot with a table including inline roles, where the preview fails to load.

Here is a screenshot of the same file, except without the table that has the inline roles.

Desktop

• OS: MacOS 11.2.3 (20D91)
• VS Code: 1.54.3
• MyST-Markdown: v0.10.2

Additional context

Here is the log output in the Log (Window).

[2021-04-05 12:31:01.964] [renderer1] [error] Cannot read property 'attrPush' of undefined: TypeError: Cannot read property 'attrPush' of undefined
    at Object.exports.addAttrs (~/.vscode/extensions/jebbs.markdown-extended-1.0.19/node_modules/markdown-it-attrs/utils.js:127:13)
    at Object.transform (~/.vscode/extensions/jebbs.markdown-extended-1.0.19/node_modules/markdown-it-attrs/patterns.js:66:15)
    at Array.curlyAttrs (~/.vscode/extensions/jebbs.markdown-extended-1.0.19/node_modules/markdown-it-attrs/index.js:30:19)
    at a.process (/Applications/Visual Studio Code.app/Contents/Resources/app/extensions/markdown-language-features/dist/extension.js:1:142666)
    at h.parse (/Applications/Visual Studio Code.app/Contents/Resources/app/extensions/markdown-language-features/dist/extension.js:1:103780)
    at t.MarkdownEngine.tokenizeString (/Applications/Visual Studio Code.app/Contents/Resources/app/extensions/markdown-language-features/dist/extension.js:1:97452)
    at t.MarkdownEngine.tokenizeDocument (/Applications/Visual Studio Code.app/Contents/Resources/app/extensions/markdown-language-features/dist/extension.js:1:97329)
    at t.MarkdownEngine.parse (/Applications/Visual Studio Code.app/Contents/Resources/app/extensions/markdown-language-features/dist/extension.js:1:97850)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
    at t.default.getRegions (/Applications/Visual Studio Code.app/Contents/Resources/app/extensions/markdown-language-features/dist/extension.js:1:62771)
    at async Promise.all (index 0)
    at t.default.provideFoldingRanges (/Applications/Visual Studio Code.app/Contents/Resources/app/extensions/markdown-language-features/dist/extension.js:1:62620)

VS Code just introduced this feature, which may be if use?

What is the difference between syntax and semantic highlighting?

Syntax highlighting colors the text based on lexical rules. In VS Code the lexical rules are expressed as regular expressions contained in a TextMate grammar.

Semantic highlighting enriches the syntax coloring based on symbol information from a language service that has the full understanding of the project. Based on this understanding each identifier gets colored & styled with the color of the symbol it resolves to. A constant variable name is rendered as constant throughout the file, not just in its declaration. Same for parameter names, property names, class names and so on.

• https://github.com/microsoft/vscode/wiki/Semantic-Highlighting-Overview
• https://github.com/microsoft/vscode-extension-samples/tree/master/semantic-tokens-sample

Is your feature request related to a problem? Please describe.
This great extension could/should also be available in Codium (open-source base of VSCode).

Describe the solution you'd like
Sent also versions to OpenVSX marketplace: https://open-vsx.org/

Describe alternatives you've considered

Additional context
VSCode and Codium have separate marketplaces, but extensions can be present on both.

Is your feature request related to a problem? Please describe.
I love how the colon fence plugin renders directives that use it (for example, the note admonition) in previews. I'd like to see the plugin also perform syntax highlighting in the source.

Describe the solution you'd like
When using the colon fence with a directive, I'd like to see the same highlighting I see when I use the backtick or tilde fence. See the attached screenshot.

Describe alternatives you've considered
N/A

Additional context

See https://github.com/microsoft/vscode-docs-authoring/tree/master/docs-markdown

The Markdown Preview is not correctly rendering the MyST enhancements.
The following enhanced markdown is ignored and only the content is rendered as plain text.

``` {tip} this is a MyST tip that fails! ```

and

``` {note} this is a MyST tip that fails! ```

The MyST-Markdown extension is enabled since highlighting, intellisense, etc works on the enhanced MyST. However it seems that the enhanced Preview part is either not working or not activated. None of the enhanced MyST markup is previewing as expected.

Desktop:
Windows 11,
VSCodeUserSetup-x64-1.83.1
MyST-Markdown 2.0.0

Describe the bug
When I try ext install ExecutableBookProject.myst-highlight in the quick open box, it searches for this name in the marketplace but nothing relevant is found. I tried searching for different parts of the name, but can't find anything that looks like the correct extension. I'm on Linux so possible related to #20? I'm relatively new to VS Code, so it is possible I'm missing something obvious here.

Desktop (please complete the following information):

Version: 1.53.2
Commit: 622cb03f7e070a9670c94bae1a45d78d7181fbd4
Date: 2021-02-11T21:13:58.622Z
Electron: 11.3.0
Chrome: 87.0.4280.141
Node.js: 12.18.3
V8: 8.7.220.31-electron.0
OS: Linux x64 5.11.1-arch1-1

Currently, YAML block are added only for code blocks:

As you can see, they work. However, they will currently be matched at any point in the cell:

This needs to be fixed, so that they are only matched if they start on the first line. Then they can be rolled out for all directives. I'm not sure what the correct grammar if for this

Describe the bug

With the jebbs.plantuml extension and the VS Code internal Markdown preview,
the following code will produce a rendered inline plantUML diagram inside the markdown preview:

# My diagram document

This should render as an inline diagram in VS code markdown preview:

```{uml}
@startuml
A -> B
B -> A
@enduml
```

With myst markdown extension enabled (and myst.preview.enable enabled), the above code block is instead rendered as some text block hovering on the right side of the document preview.

With the setting myst.preview.enable set to false, plantUML code blocks like above are rendered as diagrams again.

To Reproduce

1. Install extension jebbs.plantuml
2. Configure the plantuml rendering mode to "server" ("local" rendering in Markdown preview won't work)
3. Create a new markdown file with the content given above
4. Open Markdown preview to the side

Expected behavior

myst-vs-code should probably forward all "sphinx directives code blocks" for which it doesn't have any special handling to the default markdown renderer. Especially for directives uml, plantuml. However, there are other directives like mermaid for which there might be a special rendering in VS Code.

Screenshots

Desktop (please complete the following information):

• OS: Ubuntu 20.04 Linux
• VS Code Version v1.75.0
• Myst markdown v0.11.0

Initially the language id was set as myst, i.e. defining it as a whole new language.
However, that means you can't utilise existing markdown tools, such as previewers.

So now the language id is markdown, which means that
the whole syntax highlighting of all markdown files are being overriden.

At least for VS Code (not sure about other editors) there is the alternative to write
injection-grammars, that just to override certain parts of the highlighting.
This probably seems a better way to do it, although you may lose the ability to fully control some aspects of the language.

You can also extend the VS Code Markdown previewer, via markdown-it plugins: https://code.visualstudio.com/api/extension-guides/markdown-extension