Code Monkey home page Code Monkey logo

laurentrdc / pandoc-plot Goto Github PK

View Code? Open in Web Editor NEW
211.0 10.0 8.0 2.16 MB

Render and include figures in Pandoc documents using your plotting toolkit of choice

Home Page: https://laurentrdc.github.io/pandoc-plot/

License: GNU General Public License v2.0

Haskell 94.62% PowerShell 0.22% MATLAB 0.01% Python 0.69% Inno Setup 0.95% HTML 2.87% Shell 0.63% Asymptote 0.02%
pandoc pandoc-filter plotting matlab ggplot2 matplotlib plotly-python plotting-toolkits mathematica octave

pandoc-plot's Introduction

pandoc-plot

A Pandoc filter to generate figures from code blocks in documents

license

pandoc-plot turns code blocks present in your documents (Markdown, LaTeX, etc.) into embedded figures, using your plotting toolkit of choice, including Matplotlib, ggplot2, MATLAB, Mathematica, and more.

Overview

This program is a Pandoc filter. It can therefore be used in the middle of conversion from input format to output format, replacing code blocks with figures.

The filter recognizes code blocks with classes that match plotting toolkits. For example, using the matplotlib toolkit:

# My document

This is a paragraph.

```{.matplotlib}
import matplotlib.pyplot as plt

plt.figure()
plt.plot([0,1,2,3,4], [1,2,3,4,5])
plt.title('This is an example figure')
```

Putting the above in input.md, we can then generate the plot and embed it in an HTML page:

pandoc --filter pandoc-plot input.md --output output.html

The resulting output.html looks like this:

<h1 id="my-document">My document</h1>

<p>This is a paragraph.</p>

<figure>
<img src="plots\9671478262050082276.png" />
</figure>

Supported toolkits

pandoc-plot currently supports the following plotting toolkits (installed separately):

  • matplotlib: plots using the matplotlib Python library;
  • plotly_python : plots using the plotly Python library;
  • plotly_r: plots using the plotly R library
  • matlabplot: plots using MATLAB;
  • mathplot : plots using Mathematica;
  • octaveplot: plots using GNU Octave;
  • ggplot2: plots using ggplot2;
  • gnuplot: plots using gnuplot;
  • graphviz: graphs using Graphviz;
  • bokeh: plots using the Bokeh visualization library;
  • plotsjl: plots using the Julia Plots.jl package;
  • plantuml: diagrams using the PlantUML software suite;
  • sageplot: plots using the Sage software system;
  • d2: plots using D2;
  • asymptote: plots using Asymptote.

To know which toolkits are useable on your machine (and which ones are not available), you can check with the toolkits command:

pandoc-plot toolkits

Wish your plotting toolkit of choice was available? Please raise an issue!

Documentation

You can find more information in the documentation, available either in the source repository file MANUAL.md, on the webpage, or via the command pandoc-plot --manual.

Installation

Binaries and Installers

Latest release

Windows, Linux, and Mac OS binaries are available on the GitHub release page. There are also Windows installers.

conda

Conda Version

Like pandoc, pandoc-plot is available as a package installable with conda. Click here to see the package page.

To install in the current environment:

conda install -c conda-forge pandoc-plot

Homebrew

homebrew version

pandoc-plot is available as a package via Homebrew. Click here to see the package page.

To install:

brew install pandoc-plot

winget

You can install pandoc-plot from the Windows Package Manager winget (just like pandoc). To install:

winget install pandoc-plot

Arch Linux

AUR version

You can install pandoc-plot from the archlinux user repository as pandoc-plot-bin. You can install using e.g. yay:

yay -S pandoc-plot-bin

From Hackage/Stackage

Hackage version

pandoc-plot is available on Hackage and Stackage. Using the cabal-install tool:

cabal update
cabal install

From source

Building from source can be done using cabal:

git clone https://github.com/LaurentRDC/pandoc-plot
cd pandoc-plot
cabal install # Alternatively, `stack install`

pandoc-plot's People

Contributors

laurentrdc avatar mgajda avatar sanchayanmaity avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

pandoc-plot's Issues

Mathematica support/testing

Laurent,

I work for Wolfram Research and noticed your comment on this page: https://github.com/LaurentRDC/pandoc-plot/releases : "or if you would like to help test Mathematica support". Let me know if you need any (additional) licenses for Mathematica or Wolfram Engine, or if you're interesting in joining our developer/prerelease program.

Thanks,
Arnoud

using unicode Greek letters causes an error in plotsjl

Julia language accepts unicodes such as Greek letters, subscripted digits. But, using these characters causes an error while running the pandoc. By avoiding this with e10(x)=real(beta0(x)+nu0(x)), it will be OK. I would prefer using ε, β, ν instead. Is this a specification of this filter?

Feature Request: Make bokeh script tags load last

As discussed in #8 bokeh scripts can slow down the rendering of a report when the amount of data is relatively large (a few mb). This can be reproduced with the following report:

Pre-Figure

```{.bokeh format=HTML}

import numpy as np
from bokeh.plotting import figure
from bokeh.models import ColumnDataSource

p = figure(width=200, height=200)
p.circle(x=np.random.random(300000), y=np.random.random(300000))
```

Post-Figure

Opening the rendered HTML will show the text Pre-Figure immediately, but nothing else. Then after some time the plot and the text Post-Figure will appear. When there are many figures and these figures appear "early" this can give the appearance that the file is broken.

Apparently bokeh starts rendering (and blocking) before the browser rendered the rest of the DOM. The DOM is probably already loaded, as the bokeh rendering is only exectued when the DomContent is loaded:

        if (document.readyState != "loading") fn();
        else document.addEventListener("DOMContentLoaded", fn);

In #8 LaurentRDC suggest using script defer. Unfortunately for inline scripts this is not availabe (source).

My suggestion is to move the script tag that provides bokeh with all the data to the bottom of the document.

This is somewhat of a niche usecase: Data has to be "relatively big" to have a measurable effect (more then >100.000 primitives are required on my computer). There are some bokeh specific workarounds: (1) opengl figures render faster (2) when rendering HTML which gets delivered via a webserver an AjaxDataSource can be used. (2) is not available for standalone HTML files (as to my knowledge local HTML files can't access adjacent json files).

How to run this filter directly from YAML.

I have installed pandoc-plot from AUR. How to exec this filter from YAML. It works great when I use --filter=pandoc-plot, but when I use YAML:

filters:
 - pandoc-plot

It does not work. What is the right way to call it from YAML?

Feature Request: Caching dependent on external files

To speed up calculations pandoc-plot caches figures, identified by the hash of the source code. If the figure code depends on external code or data, these external dependencies can change, but pandoc-plot will not regenerate the figure.

Example report:

```{.bokeh format=HTML}
import json
from bokeh.plotting import figure 
with open("data.json", mode="rb") as f:
    data = json.load(f)

p = figure(width=300, height=300)
p.line(x=data["x"], y=data["y"])
```

I suggest adding an attribute depends that designates files that the figure depends on. If any of these files changes, the figure should be regenerated. Example:

```{.bokeh format=HTML depends=data.json}

In #8 LaurentRDC mentioned alternatively a cache parameter, that would disable caching for one figure:

```{.bokeh format=HTML cache=False}

In this case if caching is critical, it could be implemented in the figure generating code by the user.

Image path

Great project. I am using pandoc to generate HTML from markdown for a Jekyll site and trying to use pandoc-plot with it.

To do this I need the output location of the generated image and the HTML path to the image to vary.

For example, config like this would be helpful:

format: SVG
source: false
output_directory: /site/_site/assets/images/
path_to_images: /blog/assets/images/

where output_directory is where the image is saved to output_directory, but in the actual HTML it refers to the image using path_to_images. Is this possible?

Here is an example to explain. It is possible I am using it wrong, but it seems like other static site generators would need this feature as well to use pandoc-plot.

Here is what the file structure of a Jekyll site using pandoc might look like:

/_posts/example.md <- has pandoc plot in it
/_site/ -> mapped in a webserver to somedomain.com/blog/
/_site/example/index.html <- example.md generated to here by jekyll/pandoc becoming somedomain.com/blog/example/
/_site/assets/images/ <- need images to show up here 

I can get the images in the correct place like this:

directory: /_site/assets/images/

But in the document, the path to the image will be incorrect:

<img src="/_site/assets/images/796308625745987558.svg" />

If there were a second config, path_to_images = /blog/assets/images the image could go to the same place, but the correct (for this use case) path coudl be in the document:

<img src="/blog/assets/images/796308625745987558.svg" />

Support PDF output

I was unable to produce a PDF with replaced graphs in the current version. However the html output worked really good.

It would be nice to have the option to export PDFs with the filter

using packages other than Plots.jl may induce an error

If I try to use a CSV.jl package to read a csv datum from a file, it complains an error while executing the pandoc. The code inside my block starts like using CSV, Plots. But, using CSV may be a reason of the error. I wonder how to work it around. Thanks in advance for advice.

Now also available on Homebrew

Thanks a lot for a great tool!

I was disappointed not to find pandoc-plot on Homebrew, so I packaged it so that it can now be found there as "pandoc-plot". Maybe you can add the link to the readme to let others know it is there too.

yaml-0.11.7.0 has a change which breaks newest version (1.3.0) on Hackage

Using pandoc-plot-1.3.0, in the newest version (0.11.7.0) of yaml on Hackage, (.:?) now takes an Aeson.Key.Key instead of Text as second argument. This caused building a Hakyll site to fail for me; I am new to both Hakyll and your package, so there might be some other workaround I don't know of, but I had to add a dependency like yaml == 0.11.6.* to get it to compile.

It took me several hours to figure out what the problem was (at first I thought I needed a GHC pragma), so I hope it can be fixed to avoid that headache for others. Since you're using yaml >= 0.8 && < 1 right now, I think changing it to yaml >= 0.8 && < 0.11.7 should make your package keep working until someone finds a way to support the new way (like using some class Key and Text are both in, or something).

Anyway, thank you for your work on this project. Until I thought to search for something like this, I was building a simple site using a separate image compilation step with a Makefile and LaTeX, but I think this way will be much more convenient.

Feature Request: Sage support

Thanks for creating this filter, it's really convenient! I would be interested in using Sage with this filter (e.g. https://doc.sagemath.org/html/en/reference/plotting/sage/plot/plot_field.html) if it could be added.

I might be able to work on it, but I don't have haskell/pandoc experience. I saw that one can pass a path to executable to the pandoc CLI. I tried pandoc --filter ./executable/Main.hs unsuccessfully, and wanted to try building the pandoc-plot binary and using that but I couldn't find it after running cabal build.

Path Problems

Laurent,
First, thank you for creating this pandoc filter plugin.
I used your prebuilt plugin for windows but it can not work.

D:\Tplot
$ pandoc --filter pandoc-plot input.md --output output.html
python: can't open file 'C:\Users\Zhang': [Errno 2] No such file or directory
pandoc-plot: The script failed with exit code 2

I had checked the running environment, it seems ok

D:\Tplot
$ pandoc --version
pandoc 2.9.2
Compiled with pandoc-types 1.20, texmath 0.12.0.1, skylighting 0.8.3.2
Default user data directory: C:\Users\Zhang Ke\AppData\Roaming\pandoc
Copyright (C) 2006-2019 John MacFarlane
D:\Tplot
$ pandoc-plot --toolkits

AVAILABLE TOOLKITS

Toolkit: Python/Matplotlib
    Code block trigger: matplotlib
    Supported save formats: PNG, PDF, SVG, JPG, EPS, GIF, TIF

Toolkit: MATLAB
    Code block trigger: matlabplot
    Supported save formats: PNG, PDF, SVG, JPG, EPS, GIF, TIF

Toolkit: Python/Plotly
    Code block trigger: plotly_python
    Supported save formats: PNG, JPG, WEBP, PDF, SVG, EPS

Toolkit: GNU Octave
    Code block trigger: octaveplot
    Supported save formats: PNG, PDF, SVG, JPG, EPS, GIF, TIF

Toolkit: ggplot2
    Code block trigger: ggplot2
    Supported save formats: PNG, PDF, SVG, JPG, EPS, TIF

Toolkit: gnuplot
    Code block trigger: gnuplot
    Supported save formats: PNG, SVG, EPS, GIF, JPG, PDF


UNAVAILABLE TOOLKITS

Toolkit: Mathematica

I notice that there is a space in my computer user name zhang ke, so I think maybe the pandoc-plot can not recognize the correct route path due to this space.
Cloud you help me to fix this problem ?
Thank You,

pandoc-plot-0.9.2.0 release tarball seems to be incomplete

I've tried to build pandoc-plot using the release tarball from Hackage, but my attempt failed during the test suite run apparently because some expected data files are missing:

  Configuration tests
    empty configuration is correctly parsed to default values:  FAIL
      Exception: LoadSettingsException "tests/fixtures/.empty-config.yml" (InvalidYaml (Just (YamlException "Yaml file not found: /build/pandoc-plot-0.9.2.0/tests/fixtures/.empty-config.yml")))
    example configuration is correctly parsed:                  OK
    Configuration path stored in metadata is correctly parsed:  FAIL
      Exception: LoadSettingsException "tests/fixtures/.config-meta.yml" (InvalidYaml (Just (YamlException "Yaml file not found: /build/pandoc-plot-0.9.2.0/tests/fixtures/.config-meta.yml")))

Is it possible that those files are missing from the tarball?

Changed SHA256 sum of Release 1.2.3

Hey Laurent,

when version 1.2.3 released the file pandoc-plot-Linux-x86_64.zip had the SHA256 sum 2c84a366724496bc60f22d4c0ec945d891f19390f8c5ec6311a5e5614aef1e05 while now it has the SHA256 sum 108b70fb4120126fbf2f836d2a9d980deeff364c5f95f61873a5fa10e877712d . I think commit a6de079 triggered the CI to upload a new file under the release 1.2.3. In the past files of a release never changed as far as I noticed.

Now with the CI in place is pandoc-plot going to change the "release strategy" and updating releases with the newest current commit? Or was this more of a one-time occurence due to the change in packaging?

I'm asking because I'm maintaining the pandoc-plot-bin PKGBUILD in the archlinux user repository . Currently I'm semiautomatically downloading pandoc-plot on every release, verify that it's working and update the PKGBUILD with the sha256 sum. If the file of a version changes the sha256 sum doesn't match anymore and a user has to manually override it. If a file of a release changing will occur more often in the feature I'll either skip the sha256 verification or will have to automate calculating the sha256 whenever that happens.

Thanks!
Syntonym

Better error processing & message when output format string is in lower caps.

The output format string in a pandoc-plot configuration YAML file must be in upper case, because that is what Haskell seems to expect. Unfortunately, making a mistake here leads to hard-to-debug errors:

In a typical tool-chain, pandoc-plot will only fail to load the settings - but this is hard to spot in some cases, e.g. if it silently uses another path for the binaries of the toolkits.

It was only when I tried this command

$ pandoc-plot toolkits --config ~/myproject/pandoc-plot-settings.yaml

that I found the true cause:


pandoc-plot: Could not convert to expected type: Error in $.format: parsing 
Text.Pandoc.Filter.Plot.Monad.Types.SaveFormat 
failed, expected one of the tags ["PNG","PDF","SVG","JPG","EPS","GIF","TIF",
"WEBP","HTML","LaTeX"], but found tag "png"
CallStack (from HasCallStack):
  error, called at src/Data/Yaml/Config.hs:215:19 in yml-0.11.7.0-3e357bc6:Data.Yaml.Config

So bottom-line:

  1. The output format identifiers in pandoc-plot configuration files must be all upper-case! Like so
# Default format in which to save the figures. This can be specified
# individually as well.
format: PNG
  1. It would be really good to make the code more tolerant and/or throw more informative error messages, ideally both :-).

Support running executables in Docker containers (namely PlantUML)

Hi,
the diversity of the executables used make it quite a security concern, as they all have full-disk and network access.

Now, an effective mitigation would be to put the executables like PlantUML into a Docker container with limited privileges and run it from there, as e.g. PlantUML should not need anything beyond read access to the working directory and write access to the plots output directory.

I will be happy to work on a Dockerfile for PlantUML, but before doing so, I need to know how pandoc-plot invokes PlantUML and others.

  • Do you send the figure source code via stdout/stdin to the process?
  • Or do you create a temporary file of the image source?
  • If so, do you pass an absolute path, or a relative path for the source and the target locations?

This will be important to configure the Dockerfile so that it has access to the input and sufficient rights to create the output.

If you could post an example of how the entire command line for a figure would look like for PlantUML, that would be good.

Like so:

java -jar plantuml.jar plot323232.source -o plots/plot323232.png

Relative paths would be best. stdout/stdin would be ideal.

Then, an updated version of this Dockerfile would do the trick:

`--safe` option that only allows safe plotting options

While diversity is great feature, sometimes one would like to call pandoc-plot on Markdown of unknown origin.
In such case it would be nice to have --safe option that disables all those plotting libraries that can execute arbitrary code.

This would probably allow only gnuplot, graphviz, plantuml, and ggplot2 without shell escape.

To disable shell escape in gnuplot it may be sufficient to run with environment variable set: SHELL=/bin/false gnuplot.
As far as I know graphviz and plantuml allow no shell escape.

Add PlantUML Security Profile by Default

It would be very good to run PlantUML in a mode with minimal access privileges, in order to contain any potential security issues.

Luckily, PlantUML provides readily available means for this, as documented here.

Basically, it should be sufficient for PlantUML to have access to the input file to convert and the path to which it should write the result of the conversion.

This can be achieved as follows:

java -DPLANTUML_SECURITY_PROFILE=ALLOWLIST  -D<list_of_paths>  ...

The list of paths need to be combined using the ; (on Windows) or : (Linux/OSX) separator, like so

java -DPLANTUML_SECURITY_PROFILE=ALLOWLIST  -Dplantuml.allowlist.path=/usr/common/:/usr/plantuml/  ...

I assume that the JRE needs access to only

  • the folder than contains the PlantUML .jar file,
  • the folder with the current document or a temporary file from the figure source code,
  • the target folder (plots/).

I have not yet tested this on my machine, and the details may need to be fine-tuned, but my main point is that it should be made the default behavior. Otherwise, any vulnerability in PlantUML puts the entire Pandoc machine at risk.

Feature Request: `override: true/false` parameter for default output format in configuration YAML

It would be really cool if you could add an additional keyword parameter like

override: true

to the configuration file parser. This would then make it possible to override the output format for all plots in a document by loading a different configuration file (e.g. SVG for Web/HTML, PDF for LaTeX, PNG as a fallback etc.).

Like so (source):

# Default format in which to save the figures. This can be specified 
# individually as well.
format: PNG
# You can instruct pandoc-plot to override the settings in the individual plot attributes with this:
override: true

It may make sense to apply it to DPI etc. as well.

Likely easy to implement, yet very useful!

Compatibility with Pandoc 2.17.x?

When installing from conda-forge, I get complicated dependency conflicts, which seem to stem from the fact that pandoc-plot requires a Pandoc version <2.17:

Package pandoc conflicts for:
pandoc=2.17.0.1
pandoc-plot=1.4.0 -> pandoc[version='>=2.11,<2.17']

Is that a true dependency or something than can be fixed by simply updating the meta-data?

Feature Request: Support SVG output for bokeh

Currently pandoc-plot with the bokeh toolkit only supports png and html output. Bokeh itself can output svgs source. This is especially relevant for bokeh as the bokeh png export unfortunately ignores DPI and always generates pngs with 92 DPI (ticket) which makes it less useful for pdf files.

SVG Output from GraphViz cropped / Image Size Problem with pandoc-plot and Graphviz in SVG

Hi all,
in case others are facing a similar issue: When using SVG as the output format for GraphViz figures, the output in both HTML and LaTeX is often cropped. One can avoid this by either using PDF or PNG as target formats, but both are sub-optimal of course. Another alternative is using a wider margin parameter in the DOT code (but also not nice),

Now, I think I found the core issue for this:

The resulting SVG markup contains viewBox values that are smaller than the actual width and height attributes, like so

<svg width="360pt" height="137pt"
 viewBox="0.00 0.00 353.38 125.30" xmlns="http://www.w3.org/2000/svg" 
xmlns:xlink="http://www.w3.org/1999/xlink">

If you manually set them equal to the width and height, like so

viewBox="0.00 0.00 360.00 137.00"

the issue is resolved, but that will of course be lost the next time you update the graph source-code.

The core issue seems to be a rounding bug in GraphViz, as described here:

https://gitlab.com/graphviz/graphviz/-/issues/1855

Updating GraphViz to version 3.0.0 (available since about a week) might fix this; at least that is what they say in the referenced issue. I could not yet try this due to version conflicts.

But the issue seems to be unrelated to pandoc-plot.

So no action needed; just in case others are facing the same issue and can benefit from my analysis.

Matplotlib 'tight_bbox' ignored

Hi again,

I've discovered that the tight_bbox option for matplotlib is being ignored. What seems to be happening is that in Matplotlib.hs the attribute being read is named just tight, while the attribute being parsed in Renderer.hs (and the one documented) is tight_bbox.

This can be reproduced by rendering a matplotlib figure with and without tight_bbox=true in the figure attributes; inspecting the generated code shows no difference in the plt.savefig call.

Thanks for all your work on this great pandoc filter!

Same Problem as Issue2

Hi,
I update to the latest pandoc-plot, but get an error when plotting.

ERROR | ended with exit code ExitFailure 1
ERROR | *******
ERROR | 'C:\Program' is not recognized as an internal or external command,
ERROR | operable program or batch file.
ERROR |
ERROR | *******
ERROR | Running command

The gnuplot was installed in my C:\ Program Files\ xxxxx. it seems the space between Program and Files cause the error. This Problem may be the same as issue#2
.

Please fix it, thank you

[conda package] Please push to Pandoc 2.18

It would be nice to update pandoc-plot so that it can be used with Pandoc 2.18. Currently, only Pandoc 2.17.x is allowed as per the dependencies, tried with conda/mamba:

Encountered problems while solving:
  - package pandoc-plot-1.5.3-h694c41f_0 requires pandoc >=2.11,<2.18, but none of the providers can be installed

[feature] Ability to remove the caption from pdf output

Is there a way to remove the caption from pdf output? When I try something similar to the following:

```{.graphviz}
digraph G {
    a [label = "A"]
    b [label = "B"]
    a->b
}
```

It renders the figure as follows:
Screenshot_20220228_152705

This is compiled with:

pandoc --filter pandoc-plot -t beamer presentation.md -o presentation.pdf

I don't see a good way of being able to suppress the figure caption

Plotly module

hi, Thanks for develop this nice filter, somehow the matplotlib module works but plotly module doesn't.
Could you help me findout why? Thank you!

I am in Conda environment, and versions are

#conda install -y -c plotly plotly-orca psutil requests
#conda install -y pandoc=2.11.3.2 pandoc-plot
pandoc                    2.11.3.2             h7f98852_0    conda-forge
pandoc-plot               1.0.2.1              ha770c72_0    conda-forge
pandocfilters             1.4.3            py38h06a4308_1    defaults
plotly                    4.14.3             pyhd3eb1b0_0    defaults
plotly-orca               1.3.1                         1    plotly
python                    3.8.5                h7579374_1    defaults

Error

pandoc --filter pandoc-plot input.md --output output.html --verbose
[INFO] Running filter pandoc-plot
[pandoc-plot] ERROR | Running command
[pandoc-plot] ERROR |     python3 -c "import plotly.graph_objects"
[pandoc-plot] ERROR | ended with exit code ExitFailure 1
[pandoc-plot] ERROR | *******
[pandoc-plot] ERROR | Traceback (most recent call last):
[pandoc-plot] ERROR |   File "<string>", line 1, in <module>
[pandoc-plot] ERROR | ModuleNotFoundError: No module named 'plotly'
[pandoc-plot] ERROR |
[pandoc-plot] ERROR | *******
[pandoc-plot] WARN  | Renderer for Python/Plotly needed but is not installed
[INFO] Completed filter pandoc-plot in 59 ms

import in command line seems works.

python -c "import matplotlib" && echo $?
0
python -c "import plotly.graph_objects" && echo $?
0

minimal file input.md

# My document

This is a paragraph.

```{.matplotlib}
import matplotlib.pyplot as plt

plt.figure()
plt.plot([0,1,2,3,4], [1,2,3,4,5])
plt.title('This is an example figure')
```
```{.plotly_python caption="This is a caption"}
import plotly.graph_objects as go
fig = go.Figure(data=go.Bar(y=[2, 3, 1]))
```

Full path to executable being ignored

I'm trying to configure the python executable for running matplotlib to one in a virtual environment. I need to do this to have access to all the installed packages there. Unfortunately pandoc-plot seems to ignore the setting and still runs the system version of python.

Specifically I have a source script that looks something like the following, where some_package is a package installed only in the venv:

```{.matplotlib executable="/full/path/to/venv/bin/python"}
import sys
print(f"sys.executable={sys.executable}", file=sys.stderr)
import some_package
```

With this in a markdown file and converting it with pandoc + pandoc-plot, pandoc-plot would give something like following:

[pandoc-plot] ERROR | Running command
[pandoc-plot] ERROR |     python  "/tmp/pandocplot4826263390304275862.py"
[pandoc-plot] ERROR | ended with exit code ExitFailure 1
[pandoc-plot] ERROR | *******
[pandoc-plot] ERROR | sys.executable=/usr/bin/python
[pandoc-plot] ERROR | Traceback (most recent call last):
[pandoc-plot] ERROR |   File "/tmp/pandocplot4826263390304275862.py", line 9, in <module>
[pandoc-plot] ERROR |     import some_package
[pandoc-plot] ERROR | ModuleNotFoundError: No module named 'some_package'
[pandoc-plot] ERROR | 
[pandoc-plot] ERROR | *******
[pandoc-plot] ERROR | Command "python  "/tmp/pandocplot4826263390304275862.py"" failed with exit code 1. The script source was: 
[pandoc-plot] ERROR | 
[pandoc-plot] ERROR |  1 > import sys
[pandoc-plot] ERROR |  2 > print(f"sys.executable={sys.executable}", file=sys.stderr)
[pandoc-plot] ERROR |  3 > import some_package
[pandoc-plot] ERROR | 

Note that running the venv version of python directly and executing the above script works as expected: it prints the path to the virtual environment's executable (i.e '/full/path/to/venv/bin/python') and correctly imports the package.

I experimented with not setting the executable at all, and noticed that pandoc-plot seems to default to using the python3 executable. That means that my configuration is being read, but the absolute path is being lost somewhere.

pandoc-plot does not work properly with new pandoc citeproc version

Modern pandoc has relapsed pandoc-crossref filter by-C pandoc parameter. So if I use command pandoc -t ms --highlight-style=kate -F pandoc-plot -s -C -o "$base".pdf "$file" and try to refer plot:

```{#plot .matplotlib caption='Some awesome caption' }
import numpy as np
import matplotlib.pyplot as plt

np.random.seed(23)

# Compute areas and colors
N = 150
r = 2 * np.random.rand(N)
theta = 2 * np.pi * np.random.rand(N)
area = 200 * r**2
colors = theta

fig = plt.figure()
ax = fig.add_subplot(111, projection='polar')
c = ax.scatter(theta, r, c=colors, s=area, cmap='hsv', alpha=0.75)

And refer to [@plot], I got error citeproc: citation plot not found

But -C is after -F pandoc-plot

How to fix it?

Concurrency issue when generating identical graphs

Hi,

I found out that if a document contains multiple identical plots ( exactly same source code to generate the plot), pandoc-plot fail ( in a random way) over a race condition on the created files.

pandoc-plot: plots/pandocplot12097460476205103681.src.html: withFile: resource busy (file is locked).

I work around this by adding some random comment different in each plot code source.

I run across that issue when dealing with automated generated document (Test report) with identical plots ( basic pie chart showing the test results (PASS/FAIL).

I think this could be solve by adding a random part in the calculation of the plot ID. And probably conditioned by an option to not impact performance when not needed.

Unable to install with conda on linux

There seems to be no linux package on conda-forge. Would it be possible to add those?

On an unrelated issue: The tarball with the linux binary on the realeases page has filename that end with .xz but it is actually compressed with gzip (.gz). This is because in this line the -z option is used (-J would be xz). This is not a big issue, but software that only looks at the file extension can fail because of this.

By the way this is a very nice tool. I like it a lot.

Allow specifying language on code blocks for tools like VSCode

Currently, at least as far as i know for the codeblocks to be picked up by pandoc-plot they have to exactly match the format specified in the readme, which disables syntax highlighting and language features in editors like VSCode. I'td be nice if you could use something liike this:

```py{.matplotlib}
import matplotlib.pyplot as plt
```

for the marker, it looks like this is still recognized by vscode as a python code block, so you just need to allow this in pandoc-plot

deferred loading breaks plotly in HTML output

Dear @LaurentRDC ,

first of all, thank you for this filter, it allowed me to include some nice plotly figures in my Master's thesis.
I really like having things that can be rebuilt on demand when some data changes etc., so this filter was really helpful.

Now I'm in the middle of preparing my defense and I thought I might try an HTML/JS slideshow instead of PDF, which would grant the ability to navigate the plots during the presentation.

My first attempts showed no errors, but no plots either, but I've been able to narrow the issue down.
Apparently, the plotly JS doesn't like being loaded deferred, at least I can only see plotly figures when I remove all defer statements from the html file pandoc produces.

I've attached a minimal reproducer (which additionally shows that for bokeh, deferred loading of the library works flawless).
I'm not sure what would be the best option to solve this issue, perhaps making the deferred loading optional and disable it by default for plotly figures?

Here's the minimal reproducer:

---
title: pandoc-plot reproducer
...

# section foo

## plotly foo

```{.python .plotly_python format=html caption="sample plotly fig from https://plotly.com/python/line-and-scatter/"}
import plotly.graph_objects as go
import numpy as np

N = 1000
t = np.linspace(0, 10, 100)
y = np.sin(t)

fig = go.Figure(data=go.Scatter(x=t, y=y, mode='markers'))
```

## bokeh foo

```{.python .bokeh format=html caption="Move around in the plot by using your mouse. This gallery example was modified from [here](https://docs.bokeh.org/en/latest/docs/gallery/hex_tile.html)."}
import numpy as np

from bokeh.plotting import figure
from bokeh.transform import linear_cmap
from bokeh.util.hex import hexbin

np.random.seed(23)

n = 50000
x = np.random.standard_normal(n)
y = np.random.standard_normal(n)

bins = hexbin(x, y, 0.1)

p = figure(title="Interactive plotting with Bokeh", tools="wheel_zoom,pan,reset", match_aspect=True, background_fill_color='#440154', plot_width=550, plot_height=550)

p.grid.visible = False

p.hex_tile(q="q", r="r", size=0.1, line_color=None, source=bins,
           fill_color=linear_cmap('counts', 'Viridis256', 0, max(bins.counts)))

```

Version numbers involved:

[julian@fedora pandoc_plot_reproducer]$ pandoc -v
pandoc 2.14.0.3
Compiled with pandoc-types 1.22, texmath 0.12.3, skylighting 0.10.5.2,
citeproc 0.4.0.1, ipynb 0.1.0.1
User data directory: /home/julian/.local/share/pandoc
Copyright (C) 2006-2021 John MacFarlane. Web:  https://pandoc.org
This is free software; see the source for copying conditions. There is no
warranty, not even for merchantability or fitness for a particular purpose.
[julian@fedora pandoc_plot_reproducer]$ pandoc-plot -v
1.4.0
[julian@fedora pandoc_plot_reproducer]$ pip freeze | grep -i -e plotly -e bokeh
bokeh==2.4.2
plotly==5.6.0

How I built it:

pandoc --filter pandoc-plot test.md --standalone -o test.html

Testing:

firefox test.html
chromium-browser test.html

Attempted fix:

sed 's/defer></></g' test.html > test_no_defer.html
firefox test_no_defer.html
chromium-browser test_no_defer.html

Of course this sed call above is perhaps prone to failure, if anyone reads this in the future and thinks this is a good workaround.

This is what I get as the baseline:
image

Here's the "fixed" version without deferring:
image

Please let me know if there is anything else I can try or any way to help.
Despite having had to learn some Haskell in the second year of university, I could never wrap my head around it, so I fear that I can't solve this directly...

Best regards
Julian

Plot using MATLAB does not work

Hi,

I'm trying to use pandoc-plot as pandoc filter to generate plot figure with MATLAB, however ERROR is returned.

Command line:

pandoc --filter pandoc-plot -i .\input.md -o output.html
[pandoc-plot] ERROR | Running command
[pandoc-plot] ERROR |     matlab.exe  -batch "run('C:\Users\<username>\AppData\Local\Temp\pandocplot1453168419.m')"
[pandoc-plot] ERROR | ended with exit code ExitFailure 1
[pandoc-plot] ERROR | *******
[pandoc-plot] ERROR | Error using exportgraphics
[pandoc-plot] ERROR | The value of 'destination' is invalid. Cannot create output file 'plots\13451727385949682572.png'.
[pandoc-plot] ERROR |
[pandoc-plot] ERROR | Error in pandocplot1453168419 (line 11)
[pandoc-plot] ERROR |     exportgraphics(gcf, 'plots\13451727385949682572.png', 'Resolution', 80);
[pandoc-plot] ERROR |
[pandoc-plot] ERROR | Error in run (line 91)
[pandoc-plot] ERROR | evalin('caller', strcat(script, ';'));
[pandoc-plot] ERROR |
[pandoc-plot] ERROR | ERROR: MATLAB error Exit Status: 0x00000001
[pandoc-plot] ERROR |
[pandoc-plot] ERROR | *******
[pandoc-plot] ERROR | Command "matlab.exe  -batch "run('C:\Users\<username>\AppData\Local\Temp\pandocplot1453168419.m')"" failed with exit code 1. The script source was:
[pandoc-plot] ERROR |
[pandoc-plot] ERROR | 1 > % Generated by pandoc-plot 1.2.2
[pandoc-plot] ERROR | 2 >
[pandoc-plot] ERROR | 3 > x  = 0: .1 : 2*pi;
[pandoc-plot] ERROR | 4 > y1 = cos(x);
[pandoc-plot] ERROR | 5 > y2 = sin(x);
[pandoc-plot] ERROR | 6 >
[pandoc-plot] ERROR | 7 > figure
[pandoc-plot] ERROR | 8 > plot(x, y1, 'b', x, y2, 'r-.', 'LineWidth', 2)
[pandoc-plot] ERROR |

input.md:

```{.matlabplot caption="This is a simple figure with a **Markdown** caption"}
x  = 0: .1 : 2*pi;
y1 = cos(x);
y2 = sin(x);

figure
plot(x, y1, 'b', x, y2, 'r-.', 'LineWidth', 2)
```

Seems that when MATLAB running the line exportgraphics(gcf, 'plots\13451727385949682572.png', 'Resolution', 80);, it tries to put the png file under %TEMP% folder. Since there is no folder plots exists, MATLAB return a error. But pandoc-plot expect MATLAB output it with md file, so create plots folder alone with md.

Versions:

  • pandoc-plot 1.2.2
  • pandoc: 2.14.0.1
  • MATLAB: 9.10.0.1602886 (R2021a)

How can I parse a flowchart using pandoc plot?

My markdown file is as follows:

```plantuml
@startuml
participant John as "John Doe"
participant Jane as "Jane Doe"

John --> Jane: Hello world
@enduml
` ``


```flow
st=>start: Start
e=>end
op=>operation: My Operation
cond=>condition: Yes or No?

st->op->cond
cond(yes)->e
cond(no)->op

I downloaded plantuml Jar file and pandoc plot Exe in the same folder.
I run command
pandoc --filter pandoc-plot --filter pandoc-katex --css https://cdn.jsdelivr.net/npm/[email protected]/dist/katex.min.css --css style.css --highlight-style pygments  markdown.txt -s   -o output.html

I found that the flowchart was not parsed as a picture.

Pandoc plot does not recognize the Matplotlib I installed

My running environment is Windows 7

  1. Run the python command and the result is

python 3.8.5

  1. Run the PIP list command to display

matplotlib 3.5.1

  1. Run the pandoc plot toolkits command to display:

Toolkit: python/matplotlib

code block trigger: matplotlib

  1. Markdown file

import matplotlib. pyplot as plt



plt. figure()

plt. plot([0,1,2,3,4], [1,2,3,4,5])

plt. title('This is an example figure')

  1. Operation

pandoc --filter pandoc-plot --highlight-style pygments markdown. md -s -o output. html

  1. Error reporting

renderer for python/matplotlib needed but is not installed

Graphviz as a Toolkit

Could you please support Graphviz?

This is also a graph visualization software with which you can plot a graph via a language named DOT.

For Example if you have a file test.dot containing the following content:

digraph {
    A -> B [label="test"];
}

Then executing dot -Tpng -o test.png test.dot you will get test.png generated like this.

image

To have more complicated and amazing examples you can visit the official site.

I thought Graphviz was wildly used and would not be the least popular one among the already supported toolkits.

Ability to modify figure dimensions

I can't currently see a good way to modify the dimensions of the figure. The equivalent latex code would be:

\begin{center}
    \includegraphics[width=200px]{./plots/pandocplot9558264899394455289.png}
\end{center}

Is it possible to proxy parameters like this to the underlying images? I'm having an issue where the figure is overflowing the boundaries of the slide similar to the following:
Screenshot_20220228_153448

This is more than just image overflow though, the caption "Figure 1: XYZ" is also being pushed off the page. Is there currently a way to rectify this?

Incompatible with pandoc

> pandoc -o test.docx --filter pandoc-plot test.md
pandoc-plot: Error in $: Incompatible API versions: encoded with [1,17,5,4] but attempted to decode with [1,20].
CallStack (from HasCallStack):
  error, called at ./Text/Pandoc/JSON.hs:112:48 in pandoc-types-1.20-EEqezaSxv3uJXPNHvhPD1j:Text.Pandoc.JSON
Error running filter /Users/Roy/.pandoc/filters/pandoc-plot:
Filter returned error status 1

> pandoc -v
pandoc 2.7.2
Compiled with pandoc-types 1.17.5.4, texmath 0.11.2.2, skylighting 0.7.7
Default user data directory: /Users/Roy/.local/share/pandoc or /Users/Roy/.pandoc
Copyright (C) 2006-2019 John MacFarlane
Web:  http://pandoc.org
This is free software; see the source for copying conditions.
There is no warranty, not even for merchantability or fitness
for a particular purpose.

> pandoc-plot -v
0.5.0.0

The pandoc-plot is downloaded from this repo's release page.

Inline SVG

Great project. I am using pandoc to generate HTML from markdown for a Jekyll site and trying to use pandoc-plot with it.

It would be great if there was an option to inline SVG images into the HTML (or other formats that support it).

plantuml filename incorrect

with the attached markdown a.md

the plantuml.txt file generated to run plantuml on has a different stem filename to the one put into the html file. plantuml therefore generates a png or pdf file with the wrong filename.

Matplotlib code from file will not render if it contains plt.show in a Python comment

pandoc-plot checks if the code for a Matplotlib graph contains plt.show{.python}, because plt.savefig(...){.python} will fail otherwise. This is correct. Unfortunately, it fails to detect if the plt.show` line will be in a comment, but **only if the source code is imported from a local file.

Works:

import numpy as np
import matplotlib.pyplot as plt
labels = ['A', 'B', 'C']
values = [25, 25, 50]
plt.figure()
plt.pie(values, labels = labels)
# plt.show()

with pie-matplotlib1.py like so:

import numpy as np
import matplotlib.pyplot as plt
labels = ['A', 'B', 'C']
values = [25, 25, 50]
plt.figure()
plt.pie(values, labels = labels)

Does not work:

with pie-matplotlib2.py like so:

import numpy as np
import matplotlib.pyplot as plt
labels = ['A', 'B', 'C']
values = [25, 25, 50]
plt.figure()
plt.pie(values, labels = labels)
# plt.show()

Interactive HTML figures direct bokeh div directly

Hey LaurentRDC!

I'm trying out the new feature to include bokeh interactive plots and run into the problem that the dimensions of the embedded figure are hardcoded to 600x600 px. If I understand correctly the main reason is the usage of the embed tag to embed the entire HTML document created by bokeh. For bokeh this is not necessary, it can generate a script and a div tag that can be embedded in any html file (documentation). This way no embed tag needs to be used, no dimensions need to be hardcoded and the resulting html file is standalone (currently plots are stored in plots/).

I'm not sure how the situation looks for plotly. I couldn't find a solution with a quick google search, but I'm pretty sure it should be possible. Alternatively one could only use the embed tag for plotly but use the div tag for bokeh.

I would provide and example implementation, but unfortunately I'm not well versed in haskell.

Best,
syntonym

Data frames as Table

I believe having an option to output data frames say from pandas or R datagrams, as table could be a future feature addition.

Multiple figures in the same block

At the moment, only one figure is saved and placed per code block. In this example using matplotlib for example, only the figure titled "3" will be in the final document:

# My document
```{.matplotlib}
import matplotlib.pyplot as plt

plt.figure()
plt.plot([0,1,2,3,4], [1,2,3,4,5])
plt.title('1')

plt.figure()
plt.plot([0,1,2,3,4], [1,2,3,4,5])
plt.title('2')

plt.figure()
plt.plot([0,1,2,3,4], [1,2,3,4,5])
plt.title('3')
```

Would it be possible to support such a scenario and generate/place all figures in the generated document with reasonable effort?

Of course this would need some careful thought on at least the following matters:

  • How to place the figures: Will each figure have a individual node (for example each figure has it's own figure environment in latex and use \ContinuedFloat)?
  • IDs usually need to be unique. If there is an ID given which element will get it assigned or will there be a "-1" suffix e.g.?
  • How are links to the generating code handled?

Background: I'm generating a document which mainly consists of figures and pandoc-plot seems like a perfect fit to automate this. Creating the figures for one subsection is very simple in a for loop.* I can not use subplots because a combined figure would not fit on one page. (Actually, I don't even want to worry about how many pictures fit on one page when writing the input document (Markdown).) I am therefore looking for a solution which places the individual figures and let's the output processor (for example Latex) handle them floating around.

I'm aware that this is a corner case. If you think it adds too much complexity or you don't want to support multiple figures in pandoc-plot I understand and I'll look for a different solution.

Cheers,
Hannes

`* An example would be plotting measurement vs. model for each measured temperature. In reality its more complicated and and the sections are already generated programatically. Also, it's debatable if creating such a PDF document is reasonable, but these are unfortunately the current boundary conditions.

pandoc: 2.9.2.1
pandoc-plot: v0.5.0.0

Feature Request: Support for planuml toolkit

Hello,
I just found this extension and I love it :)

Would you be interested in adding the Plantuml toolkit?

It might be a bit trickyer than other toolkits because it needs to be runned using java, but I think it would be a great addition !

I use it to document firmware projects but also cloud architectures and more.
There are tons of examples on their website and it is also open-source.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.