fredrikekre / literate.jl Goto Github PK

View Code? Open in Web Editor NEW

516.0 8.0 61.0 5.39 MB

Simple package for literate programming in Julia

Home Page: https://fredrikekre.github.io/Literate.jl

License: Other

Julia 100.00%

julia literate-programming documentation-tool documentation-generator documentation

literate.jl's Introduction

Literate

Documentation	Build Status

Literate is a package for Literate Programming. The main purpose is to facilitate writing Julia examples/tutorials that can be included in your package documentation.

Literate can generate markdown pages (for e.g. Documenter.jl), and Jupyter notebooks, from the same source file. There is also an option to "clean" the source from all metadata, and produce a pure Julia script. Using a single source file for multiple purposes reduces maintenance, and makes sure your different output formats are synced with each other.

This README was generated directly from this source file running these commands from the package root of Literate.jl:

using Literate
Literate.markdown("examples/README.jl", "."; flavor=Literate.CommonMarkFlavor())

Related packages

Weave.jl

This page was generated using Literate.jl.

literate.jl's People

Contributors

Stargazers

Watchers

Forkers

iewaij sespaces adamslc mbauman tkf hhc2tech batmanabcdefg transgirlcodes saschamann ovidiuscicero gaybro8777 ggarza devmotion aerdely johnnychen94 jonas-schulze jrbuie liuyxpp okonsamuel kescobo tobi-lipede-oodle rfourquet alebonfanti migarstka prosoitos j-fu mortenpi khosravipasha berenz egidioln omar-elrefaei juliencalbert ranocha compound-eye roundnose virgile-baudrot jkrumbiegel st-- carlolucibello jvaverka manuelbb-upb artaxerces playfloor rafaqz jd-foster tpapp tlienart t-bltg leios svilupp wsshin tfiers beastyblacksmith dlfivefifty zcfrank1st kbarros vardleri jpfairbanks musoke

literate.jl's Issues

Bug executing Literate.notebook("examples/example.jl", "."; documenter=false)

Executing > Literate.notebook("examples/example.jl", "."; documenter=false)

throws the following error:

[ Info: generating notebook from C:\Users\Denis\.julia\dev\examples\example.jl
[ Info: executing notebook example.ipynb
┌ Error: error when executing notebook based on input file: C:\Users\Denis\.julia\dev\examples\example.jl
└ @ Literate C:\Users\Denis.julia\packages\Literate\hmb7i\src\Literate.jl:558
ERROR: LoadError: UndefVarError: MYVALUE not defined
in expression starting at string:1
when executing the following code block

MYVARIABLE = MYVALUE

Stacktrace:
[1] execute_notebook(::Dict{Any,Any}) at C:\Users\Denis.julia\packages\Literate\hmb7i\src\Literate.jl:599
[2] (::Literate.var"#20#22")() at C:\Users\Denis.julia\packages\Literate\hmb7i\src\Literate.jl:555
[3] cd(::Literate.var"#20#22", ::String) at .\file.jl:93
[4] #notebook#18(::Dict{Any,Any}, ::Base.Iterators.Pairs{Symbol,Bool,Tuple{Symbol},NamedTuple{(:documenter,),Tuple{Bool}}}, ::typeof(Literate.notebook), ::String,
::String) at C:\Users\Denis.julia\packages\Literate\hmb7i\src\Literate.jl:554
[5] (::Literate.var"#kw##notebook")(::NamedTuple{(:documenter,),Tuple{Bool}}, ::typeof(Literate.notebook), ::String, ::String) at .\none:0
[6] top-level scope at none:0

Literate.markdown on the same example is fine.

Nbviewer/Binder links should work in /previews/PRXX/?

I'm wondering if the nbviewer/binder badge links should work in previews or whether I'm doing something wrong. See, e.g., here:
https://fourierflows.github.io/GeophysicalFlows.jl/previews/PR70/generated/twodnavierstokes_decaying/

Print to stdout not printed in the rendered html.

I am trying to capture printed parts, as the documentation says at

https://fredrikekre.se/Literate.jl/latest/generated/example/#Output-Capturing-1

but even there, just the returned value of foo()is displayed and I noticed the same. Is that a problem of Literate.jl or a setting that changed in Documenter.jl?

[Feature request] Metadata for Jupyter notebooks

Based on the following code,

Literate.jl/src/Literate.jl

Lines 452 to 458 in d99e8ea

    
           if isa(chunk, MDChunk) 
        
               cell["cell_type"] = "markdown" 
        
               cell["metadata"] = Dict() 
        
               lines = String[x.second for x in chunk.lines] # skip indent 
        
               @views map!(x -> x * '\n', lines[1:end-1], lines[1:end-1]) 
        
               cell["source"] = lines 
        
               cell["outputs"] = []

and

Literate.jl/src/Literate.jl

Lines 459 to 466 in d99e8ea

    
           else # isa(chunk, CodeChunk) 
        
               cell["cell_type"] = "code" 
        
               cell["metadata"] = Dict() 
        
               @views map!(x -> x * '\n', chunk.lines[1:end-1], chunk.lines[1:end-1]) 
        
               cell["source"] = chunk.lines 
        
               cell["execution_count"] = nothing 
        
               cell["outputs"] = [] 
        
           end

it seems that it should be possible to define a new syntax to enter arbitrary notebook metadata, perhaps as JSON or a Julia Dict. One example of this syntax might look like:

#-
#nb-meta Dict("rise" => Dict("transition" => "zoom"))
<code here>

for which that metadata would be embedded within the cell.

This was motivated by a usecase provided by @briochemc.

From Slack:

Yes I was wondering if someone here had thought about adding some flags to the source .jl file for Literate to interpret and include in the notebook output in a way that Rise can know where the slides start and end. Right now I have to do the separation into individual slides by hand (it's not too bad but it'd be better if I could have the info in the source file)

Executed markdown: fenced output block no new line before it

I believe there should be one new line between the code block and the output block according to Markdown rules.
Current workaround: configure the codefence to "```julia" => "```\n". But it will add an extra new line (two new lines) after the output block.

Feature request: Filter lines ending by `#md`, `#jl` etc.

Is there a reason only #src is filtered at the end of a line?

I would like to have code only appear in the markdown version (or the the notebook), but I want it to be runnable in the source file.

Use case: I have a worked example for my package that I want to use for testing and illustration (with plots, but no tests). So, only the .jl file should contain tests, and only the .md file (that I weave to include in the documentation) should contain tests.

Btw: in the source it says that #jl is filtered at the end of the line, but this doesn't seem to be the case.

Literate.jl/src/Literate.jl

Line 24 in 8e11b4e

    
           # * Lines starting with, or ending with, `#jl` are filtered out unless creating a script file

Support for Pkg mode

Lines beginning with ] could be considered "Pkg mode" and evaluated as if in that repl mode; this follows what IJulia does when this is in a cell. Currently this just causes an error.

cc @dhairyagandhi96

Pkg.add fails on Julia 1.0.2

Here's the failure when attempting to add the package...

julia> Pkg.clone("https://github.com/fredrikekre/Literate.jl")
   Cloning git-repo `https://github.com/fredrikekre/Literate.jl`
  Updating git-repo `https://github.com/fredrikekre/Literate.jl`
[ Info: Path `/Users/robert/.julia/dev/Literate` exists and looks like the correct package, using existing path instead of cloning
 Resolving package versions...
ERROR: Unsatisfiable requirements detected for package StatsBase [2913bbd2]:
 StatsBase [2913bbd2] log:
 ├─possible versions are: [0.1.0, 0.2.0-0.2.1, 0.2.3-0.2.10, 0.3.0-0.3.13, 0.4.0-0.4.4, 0.5.0-0.5.3, 0.6.0-0.6.16, 0.7.0-0.7.4, 0.8.0-0.8.3, 0.9.0, 0.10.0, 0.11.0-0.11.1, 0.12.0, 0.13.0-0.13.1, 0.14.0-0.14.1, 0.15.0, 0.16.0-0.16.1, 0.17.0, 0.18.0, 0.19.0-0.19.5, 0.20.0-0.20.1, 0.22.0, 0.23.0-0.23.1, 0.24.0, 0.25.0, 0.26.0] or uninstalled
 ├─restricted to versions 0.14.0-* by Plots [91a5bcdd], leaving only versions [0.14.0-0.14.1, 0.15.0, 0.16.0-0.16.1, 0.17.0, 0.18.0, 0.19.0-0.19.5, 0.20.0-0.20.1, 0.22.0, 0.23.0-0.23.1, 0.24.0, 0.25.0, 0.26.0]
 │ └─Plots [91a5bcdd] log:
 │   ├─possible versions are: 0.22.0 or uninstalled
 │   └─Plots [91a5bcdd] is fixed to version 0.22.0+
 ├─restricted to versions 0.25.0 by an explicit requirement, leaving only versions 0.25.0
 └─restricted by julia compatibility requirements to versions: 0.26.0 or uninstalled — no versions left
Stacktrace:
 [1] #propagate_constraints!#61(::Bool, ::Function, ::Pkg.GraphType.Graph, ::Set{Int64}) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/Pkg/src/GraphType.jl:1005
 [2] propagate_constraints! at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/Pkg/src/GraphType.jl:946 [inlined]
 [3] #simplify_graph!#121(::Bool, ::Function, ::Pkg.GraphType.Graph, ::Set{Int64}) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/Pkg/src/GraphType.jl:1460
 [4] simplify_graph! at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/Pkg/src/GraphType.jl:1460 [inlined]
 [5] macro expansion at ./logging.jl:317 [inlined]
 [6] resolve_versions!(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}, ::Nothing) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/Pkg/src/Operations.jl:353
 [7] resolve_versions! at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/Pkg/src/Operations.jl:316 [inlined]
 [8] #add_or_develop#62(::Array{Base.UUID,1}, ::Symbol, ::Function, ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/Pkg/src/Operations.jl:1201
 [9] #add_or_develop at ./none:0 [inlined]
 [10] #add_or_develop#13(::Symbol, ::Bool, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:64
 [11] #add_or_develop at ./none:0 [inlined]
 [12] #develop#19 at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:70 [inlined]
 [13] develop at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:70 [inlined]
 [14] macro expansion at ./logging.jl:308 [inlined]
 [15] clone(::String, ::String) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:468 (repeats 2 times)
 [16] top-level scope at none:0

Give more control, markdown(inputfile, outputfile)

Can we have more control over the file names generated? I mean,

markdown("foo.jl", "foo.md")

Literate.markdown(inputfile, outputdir; config::Dict=Dict(), kwargs...)

I would take inputfile -> outputfile or input_dir -> output_dir, but not input_file -> output_dir.

Real-life test case: have test/test_fields.jl and you want to make it to be docs/src/fields.md.

Spurious #- in Markdown

Running markdown on a file with the content

# SIMD sparse matrix dense vector multiplication

1 + 1

gives

```@meta
EditURL = "@__REPO_ROOT_URL__/"
```

SIMD sparse matrix dense vector multiplication

```@example sparse_matvec
1 + 1#-
```

*This page was generated using [Literate.jl](https://github.com/fredrikekre/Literate.jl).*

The #- in the @example block seems weird.

error when calling `include`

If I run Literate on a .jl file with an include statement, it throws an error. E.g. for file include_error.jl with contents

include("foo.jl")

and file foo.jl with contents

f(x) = 3x+2

Literate on julia-0.7.0 errors as follows

julia> using Literate
                                                                                                                                                                              
julia> Literate.notebook("include-error.jl", "."; documenter=false)
[ Info: generating notebook from `~/gitworking/julia-examples/src/include-error.jl`                                                                                           
[ Info: not running on Travis, skipping links will not be correct.                                                                                                            
[ Info: executing notebook `include-error.ipynb`                                                                                                                              
┌ Error: error when executing notebook based on input file: `~/gitworking/julia-examples/src/include-error.jl`                                                                
└ @ Literate ~/.julia/packages/Literate/tArjO/src/Literate.jl:495                                                                                                             
ERROR: LoadError: UndefVarError: include not defined                                                                                                                          
in expression starting at string:1                                                                                                                                            
when executing the following code block                                                                                                                                       
                                                                                                                                                                              
  '''julia           # I manually replaced backticks here to avoid screwing up Markdown formatting                                                                                                                                 
  include("foo.jl")                                                                                                                                                             
  '''                                                                                                                                                                   
                                                                                                                                                                              
Stacktrace:                                                                                                                                                                   
 [1] error(::String) at ./error.jl:33                                                                                                                                         
 [2] execute_notebook(::Dict{Any,Any}) at /home/gibson/.julia/packages/Literate/tArjO/src/Literate.jl:530                                                                     
 [3] (::getfield(Literate, Symbol("##14#17")))() at /home/gibson/.julia/packages/Literate/tArjO/src/Literate.jl:492                                                           
 [4] cd(::getfield(Literate, Symbol("##14#17")), ::String) at ./file.jl:96
 [5] #notebook#11(::typeof(identity), ::typeof(identity), ::Bool, ::Bool, ::Bool, ::String, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::String, ::String) at /home/gibson/.julia/packages/Literate/tArjO/src/Literate.jl:470
 [6] (::getfield(Literate, Symbol("#kw##notebook")))(::NamedTuple{(:documenter,),Tuple{Bool}}, ::typeof(Literate.notebook), ::String, ::String) at ./none:0
 [7] top-level scope at none:0

Am I missing something simple? I need to use include to load some lengthy utility code that is a side-issue from the example I'm trying to illustrate with Literate.

execute markdown files

People have suggested that it should be possible to execute code and splice in the output in the markdown files. Example:

# # Hello

foo(a, b) = a + b
foo(2, 3)

would result in a markdown file with

# Hello

```julia
foo(a, b) = a + b
foo(2, 3)
```

```
5
```

See https://discourse.julialang.org/t/ann-literate-jl/10651

subtypes() not defined error

I've got another one! This one is weird...

In test.jl:

# some test
supertype(Number)
#-
subtypes(Number)

In the REPL:

julia> using Literate

julia> supertype(Number)
Any

julia> subtypes(Number)
2-element Array{Any,1}:
 Complex
 Real

julia> Literate.notebook("test.jl", "./")
[ Info: generating notebook from `~/computation/julia_playground/intro_julia/test.jl`
[ Info: not running on Travis, skipping links will not be correct.
[ Info: executing notebook `test.ipynb`
┌ Error: error when executing notebook based on input file: `~/computation/julia_playground/intro_julia/test.jl`
└ @ Literate ~/.julia/packages/Literate/OTyBa/src/Literate.jl:495
ERROR: LoadError: UndefVarError: subtypes not defined
in expression starting at string:1
when executing the following code block

```julia
subtypes(Number)
```

Stacktrace:
 [1] execute_notebook(::Dict{Any,Any}) at /Users/kev/.julia/packages/Literate/OTyBa/src/Literate.jl:536
 [2] (::getfield(Literate, Symbol("##14#17")))() at /Users/kev/.julia/packages/Literate/OTyBa/src/Literate.jl:492
 [3] cd(::getfield(Literate, Symbol("##14#17")), ::String) at ./file.jl:96
 [4] #notebook#11(::typeof(identity), ::typeof(identity), ::Bool, ::Bool, ::Bool, ::String, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::String, ::String) at /Users/kev/.julia/packages/Literate/OTyBa/src/Literate.jl:470
 [5] notebook(::String, ::String) at /Users/kev/.julia/packages/Literate/OTyBa/src/Literate.jl:420
 [6] top-level scope at none:0

plot figure is not shown

Hello, I recently started to use mkdocs but those plot from Julia backend do not show the figures as usual. Instead, I got a bunch of these:

Documenter.Documents.RawHTML("<?xml version=\"1.0\" encoding=\"utf-8\"?>\n\n")

And then I checked the generated markdown file, places where those figure located are filled with following:

Documenter.Documents.RawHTML("<?xml version=\"1.0\" encoding=\"utf-8\"?>\n<svg xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"600\" height=\"400\" viewBox=\"0 0 2400 1600\">\n<defs>\n  <clipPath id=\"clip4800\">\n    <rect x=\"0\" y=\"0\" width=\"2000\" height=\"2000\"/>\n  </clipPath>\n</defs>\n<defs>\n  <clipPath id=\"clip4801\">\n    <rect x=\"0\" y=\"0\" width=\"2400\" height=\"1600\"/>\n  </clipPath>\n</defs>\n<polygon clip-path=\"url(#clip4801)\" points=\"\n0,1600 2400,1600 2400,0 0,0 \n  \" fill=\"#ffffff\" fill-rule=\"evenodd\" fill-opacity=\"1\"/>\n<defs>\n  <clipPath id=\"clip4802\">\n    <rect x=\"480\" y=\"0\" width=\"1681\" height=\"1600\"/>\n  </clipPath>\n</defs>\n<polygon clip-path=\"url(#clip4801)\" points=\"\n212.353,1440.48 2321.26,1440.48 2321.26,47.2441 212.353,47.2441 \n  \" fill=\"#ffffff\" fill-rule=\"evenodd\" fill-opacity=\"1\"/>\n<defs>\n  <clipPath id=\"clip4803\">\n    <rect x=\"212\" y=\"47\" width=\"2110\" height=\"1394\"/>\n  </clipPath>\n</defs>\n<polyline clip-path=\"url(#clip4803)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n  212.353,1440.48 212.353,47.2441 \n  \"/>\n<polyline clip-path=\"url(#clip4803)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n  739.58,1440.48 739.58,47.2441 \n  \"/>\n<polyline clip-path=\"url(#clip4803)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n  1266.81,1440.48 1266.81,47.2441 \n  \"/>\n<polyline clip-path=\"url(#clip4803)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n  1794.03,1440.48 1794.03,47.2441 \n  \"/>\n<polyline clip-path=\"url(#clip4803)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n  2321.26,1440.48 2321.26,47.2441 \n  \"/>\n<polyline clip-path=\"url(#clip4803)\" style=\"stroke:#000000; stroke-width:2; stroke-opacity:0.1; fill:none\" points=\"\n  212.353,1263.28 2321.26,1263.28 \n 
...

I am not familiar with this thing but I guess there is enough information to form the figure I generate.

I basically follow the suggestion in Documenter.jl to specify the format to be DocumenterMarkdown.Markdown() as well as many other settings. And I also follow other packages (such as Optim.jl) to set up the document builder. I am not sure how it happened nor do I find other related issues. Thanks in advance.

Comments in code

In my scripts I have something like this, with the intention that you can easily enable GPU support if you have it set up.

using Flux
# using CuArrays

Unfortunately Literate puts the commented code into a plain text cell. I'm not sure what the best way around this is, but perhaps it's a reasonable heuristic to treat comments as code when there's no space like this?

empty deploy_folder for dev branch in travis

The link is set to be:
"@__NBVIEWER_ROOT_URL__/$(nbviewer_folder)/$(cardname).ipynb"

and the generated URL is

https://nbviewer.jupyter.org/github/johnnychen94/DemoCards.jl/blob/gh-pages//democards/simplest_demopage/dependencies/part2/simple_cameraman.ipynb

in Travis-CI (missing a dev here). Just swapped into Github Action, and it works okay (in both dev and stable branch).

Although it looks perfect to me, it might be related to this part of the codes (line 233)

Literate.jl/src/Literate.jl

Lines 230 to 242 in 00904a4

    
           if haskey(ENV, "HAS_JOSH_K_SEAL_OF_APPROVAL") # Travis CI 
        
               repo_slug = get(ENV, "TRAVIS_REPO_SLUG", "unknown-repository") 
        
               deploy_folder = if get(ENV, "TRAVIS_PULL_REQUEST", nothing) == "false" 
        
                   get(ENV, "TRAVIS_TAG", get(user_config, "devurl", "dev")) 
        
               else 
        
                   "previews/PR$(get(ENV, "TRAVIS_PULL_REQUEST", "##"))" 
        
               end 
        
               cfg["repo_root_url"] = "https://github.com/$(repo_slug)/blob/$(edit_commit)" 
        
               cfg["nbviewer_root_url"] = "https://nbviewer.jupyter.org/github/$(repo_slug)/blob/$(deploy_branch)/$(deploy_folder)" 
        
               cfg["binder_root_url"] = "https://mybinder.org/v2/gh/$(repo_slug)/$(deploy_branch)?filepath=$(deploy_folder)" 
        
               if (dir = get(ENV, "TRAVIS_BUILD_DIR", nothing)) !== nothing 
        
                   cfg["repo_root_path"] = dir 
        
               end

Not sure if it's related: I might incorrectly configure Travis-CI for Documenter, since I can only deploy a dev branch.

Julia code quoting for Jekyll

I didn’t work out how to format Julia code with the

{% highlight julia %}

{% endhighlight julia %}

that Jekyll wanted, rather than the

```julia
 
```

Is there a setting or is it a job for preprocess?

Feature request / question: automatic collection of doc pages entry

This is more of a request / question. I'd like to know if this added feature is something that could be incorporated into Literate, or if it's better for Documenter, or just create a separate package.

The gist of the feature (pages_array) is that the nested pages array of Pair's is automatically determined from a given directory tree with .jl literate tutorials. For example, assuming a literate_examples folder in the package, the generated markdown files are grabbed from

# After generating markdown files in `generated_dir` directory
tutorials_generated = [joinpath(root, f) for (root, dirs, files) in Base.Filesystem.walkdir(generated_dir) for f in files]
tutorials_md = filter(x -> endswith(x, ".md"), tutorials_generated)
tutorials_md = map(x -> String(last(split(x, joinpath("docs", "src")))), tutorials_md)

tutorials = pages_array(tutorials_md) # Main feature function

pages = Any[
    "Home" => "index.md",
    "Installation" => "Installation.md",
    "Tutorials" => tutorials,
    ...
]

How to include media (images)?

Given the following directory structure,

.git
docs
  build
  src
    index.md
    ..
  make.jl
src
  index.md
test
  runtests.jl
Project.toml
...

What could be the best solution for embed images in the src/file.jl?

A single comment character

I discovered that a single # on a line to itself triggered this error:

ERROR: BoundsError: attempt to access 0-element Array{Pair{String,String},1} at index [1]
Stacktrace:
 [1] getindex(::Array{Pair{String,String},1}, ::Int64) at ./array.jl:554
 [2] #parse#1(::Bool, ::Function, ::String) at /Users/me/.julia/v0.6/Literate/src/Literate.jl:77
 [3] (::Literate.#kw##parse)(::Array{Any,1}, ::Literate.#parse, ::String) at ./<missing>:0
 [4] #notebook#11(::Base.#identity, ::Base.#identity, ::Bool, ::Bool, ::Bool, ::String, ::Array{Any,1}, ::Function, ::String, ::String) at /Users/me/.julia/v0.6/Literate/src/Literate.jl:412
 [5] notebook(::String, ::String) at /Users/me/.julia/v0.6/Literate/src/Literate.jl:397
 [6] macro expansion at /Users/me/.julia/v0.6/Atom/src/repl.jl:118 [inlined]
 [7] anonymous at ./<missing>:?

using Literate.jl version 0.2.0 on Julia 0.6.2.

Suggestion: Turn admonitions into quotes for notebooks?

It seems admonitions like !!! note are not available in notebooks (jupyter/notebook#1292).
Maybe Literate.jl could translate those into quotes by default via Literate.notebook?
For example, take

# !!! note
#     This is a warning

as a source file.
It would become a Documenter admonition via Literate.markdown by simply stripping the # at the start, i.e., give:

!!! note
    This is a warning

But via Literate.notebook, it would look more like

> *Note*:
> This is a warning

which renders sort-of-like a Documenter admonition.

Note:
This is a warning

Change `#'` to simply `#` and require `##` for julia comments

As discussed in https://discourse.julialang.org/t/ann-literate-jl/10651/9, using #' could be annoying, and julia comments are probably not as frequent as markdown input.

FR: Macro that "shows" included code

It would be cool if there was a macro that expanded the code included from
include(file.jl) into showing the full source code for this file when building / translating the files to proper format.

The way I personally would use it is to have the source in a single file, everything there well defined and clear. This would make it even easier to use source files used for tests in the format that Literate.jl uses.

Initially I thought of posting this to Documenter.jl but I wasn't sure. So I am tagging @mortenpi as well!

Inert Julia code?

Is there a way to have some inert Julia code that looks like normal code but doesn’t run?

For example, if I wanted to say

# add this if you want to close the session
exit()

so that it looked like normal code but didn’t actually run?

Notebook does not display PyPlot figures

It seems that if I use PyPlot to generate figures, then the notebook generated from Literate.notebook does not display the PyPlot figures (but figures are displayed outside in a standalone GUI).
I have already let Jupyter trust the notebook. But, the figures are still missing.
I tried using Plots instead and it works.
Is PyPlot not supported by Literate or something needs to be done?
I am using Julia v1.0.1 with PyPlot v2.6.3.

using PyPlot
x = range(0, stop=6π, length=1000)
y1 = sin.(x)
plot(y1)

Literate.notebook & Plots: "_show is not defined for this backend. m=text/plain"

When generating a notebook which contains Plots.jl plot() commands, I get a warning _show is not defined for this backend. m=text/plain.

MWE: this file produces the error, attached as .txt to please github: test.jl.txt. Contents is:

# Save this as `test.jl` on the desktop, and then run
# ```
# using Literate
# Literate.notebook("/Users/me/Desktop/test.jl","/Users/me/Desktop/")
# ```
using Plots
gr()
plot(rand(20))

# That gives a warning, uncommenting these lines gives an error:

# unicodeplots()
# plot(rand(20))

Here is the warning produced:

julia> using Literate
julia> Literate.notebook("/Users/me/Desktop/test.jl","/Users/me/Desktop/")
Info: generating notebook from /Users/me/Desktop/test.jl
Info: not running on Travis, skipping links will not be correct.
Info: executing notebook test.ipynb
WARNING: _show is not defined for this backend. m=text/plain

If I uncomment the last two lines of the file, and run that again, then in addition I get the following errors:


┌────────────────────────────────────────────────────────────┐
0.9 │⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢀⡄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀│ y1
│⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢸⡇⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀│
│⠀⠀⠀⠀⠀⠀⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢸⡇⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀│
│⠀⠀⠀⠀⠀⢸⢣⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⡇⢱⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢸⡇⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀│
│⠀⠀⠀⠀⠀⢸⠀⢣⠀⠀⠀⠀⠀⠀⠀⠀⠀⡇⢸⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⡸⢇⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀│
│⠀⠀⠀⠀⠀⡜⠀⠀⢣⢀⡠⠔⡆⠀⠀⠀⢰⠁⢸⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢸⡇⠀⠀⠀⠀⡇⢸⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀│
│⠀⠀⠀⠀⠀⡇⠀⠀⠀⠁⠀⠀⡇⠀⠀⠀⢸⠀⠀⡇⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⡜⢣⠀⠀⠀⢰⠁⠘⡄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢰⢆⠀⠀⠀⠀⠀│
│⠀⠀⠀⠀⠀⡇⠀⠀⠀⠀⠀⠀⢸⠀⠀⠀⡸⠀⠀⡇⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢀⠇⠸⡀⠀⠀⡸⠀⠀⡇⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⡸⠀⠑⢄⡀⠀⠀│
│⠀⠀⠀⠀⢸⠀⠀⠀⠀⠀⠀⠀⠘⡄⠀⠀⡇⠀⠀⢇⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢸⠀⠀⡇⠀⠀⡇⠀⠀⢱⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⡇⠀⠀⠀⢇⠀⠀│
│⠀⠀⠀⠀⢸⠀⠀⠀⠀⠀⠀⠀⠀⡇⠀⠀⡇⠀⠀⢸⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⡎⠀⠀⢸⠀⢠⠃⠀⠀⢸⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢰⠁⠀⠀⠀⠘⡄⠀│
│⠀⠀⠀⠀⡜⠀⠀⠀⠀⠀⠀⠀⠀⢸⠀⢸⠀⠀⠀⢸⠀⠀⠀⠀⠀⠀⢸⢇⠀⠀⢠⠃⠀⠀⠈⡆⢸⠀⠀⠀⠀⡇⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢸⠀⠀⠀⠀⠀⢱⠀│
│⠀⠀⠀⠀⡇⠀⠀⠀⠀⠀⠀⠀⠀⠘⡄⢸⠀⠀⠀⠘⡄⠀⠀⠀⠀⠀⡜⠈⢆⠀⢸⠀⠀⠀⠀⢣⡇⠀⠀⠀⠀⢣⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⡇⠀⠀⠀⠀⠀⠀⢇│
│⠀⠀⠀⠀⡇⠀⠀⠀⠀⠀⠀⠀⠀⠀⡇⡎⠀⠀⠀⠀⡇⠀⠀⠀⠀⠀⡇⠀⠈⡆⡇⠀⠀⠀⠀⠸⠇⠀⠀⠀⠀⢸⠀⠀⠀⠀⠀⠀⠀⠀⠀⢠⠃⠀⠀⠀⠀⠀⠀⠘│
│⠀⠀⠀⢸⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢸⡇⠀⠀⠀⠀⡇⠀⠀⠀⠀⢰⠁⠀⠀⠘⠃⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠈⢇⠀⠀⠀⠀⠀⠀⠀⠀⢸⠀⠀⠀⠀⠀⠀⠀⠀│
│⠀⠀⠀⢸⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠘⠇⠀⠀⠀⠀⢱⠀⠀⠀⠀⡸⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠈⢆⠀⠀⠀⡰⠢⣀⠀⡇⠀⠀⠀⠀⠀⠀⠀⠀│
│⠀⠀⠀⡜⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢸⠀⠀⠀⠀⡇⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠈⢆⢀⠎⠀⠀⠀⠑⠇⠀⠀⠀⠀⠀⠀⠀⠀│
│⠀⠀⠀⡇⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢸⡀⠀⠀⢠⠃⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠈⠃⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀│
│⠀⠀⠀⡇⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠱⡀⠀⢸⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀│
│⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠱⡀⡇⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀│
0 │⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠱⠇⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀│
└────────────────────────────────────────────────────────────┘
0                                                           20
WARNING: _show is not defined for this backend. m=image/svg+xml
WARNING: _show is not defined for this backend. m=application/pdf
Error: error when executing notebook based on input file: /Users/me/Desktop/test.jl
ERROR: FileIO.File{FileIO.DataFormat{:UNKNOWN}}("/var/folders/1b/6flsxnb12rx3r3ktpr_zllyc0000gn/T/juliaiokN4H.pdf") couldn't be recognized by FileIO.

Stacktrace:
[1] #load#20(::Array{Any,1}, ::Function, ::FileIO.File{FileIO.DataFormat{:UNKNOWN}}) at /Users/me/.julia/v0.6/FileIO/src/loadsave.jl:88
[2] load(::FileIO.File{FileIO.DataFormat{:UNKNOWN}}) at /Users/me/.julia/v0.6/FileIO/src/loadsave.jl:86
[3] #load#13(::Array{Any,1}, ::Function, ::String) at /Users/me/.julia/v0.6/FileIO/src/loadsave.jl:52
[4] _show(::IOContext{Base64EncodePipe}, ::MIME{Symbol("image/png")}, ::Plots.Plot{Plots.UnicodePlotsBackend}) at /Users/me/.julia/v0.6/Plots/src/output.jl:230
[5] show(::IOContext{Base64EncodePipe}, ::MIME{Symbol("image/png")}, ::Plots.Plot{Plots.UnicodePlotsBackend}) at /Users/me/.julia/v0.6/Plots/src/output.jl:210
[6] limitstringmime(::MIME{Symbol("image/png")}, ::Plots.Plot{Plots.UnicodePlotsBackend}) at /Users/me/.julia/v0.6/Literate/src/IJulia.jl:68
[7] display_dict(::Plots.Plot{Plots.UnicodePlotsBackend}) at /Users/me/.julia/v0.6/Literate/src/IJulia.jl:28
[8] execute_notebook(::Dict{Any,Any}) at /Users/me/.julia/v0.6/Literate/src/Literate.jl:532
[9] #15 at /Users/me/.julia/v0.6/Literate/src/Literate.jl:466 [inlined]
[10] cd(::Literate.##15#19, ::String) at ./file.jl:70
[11] #notebook#11(::Base.#identity, ::Base.#identity, ::Bool, ::Bool, ::Bool, ::String, ::Array{Any,1}, ::Function, ::String, ::String) at /Users/me/.julia/v0.6/Literate/src/Literate.jl:465
[12] notebook(::String, ::String) at /Users/me/.julia/v0.6/Literate/src/Literate.jl:396

Multi-line comments

This is a great package! I wonder if multiline comments could be supported? In my experiments, a multiline comment (#= ... =#) turns into a code block prefaced with @example testing.

Run `Literate.notebook` in the same process

If I am generating a notebook with e.g. Plots.jl plots, it would be useful to somehow run each Literate.notebook() call in the same process so that the loading and compilation only occur once. Is this possible?

inline comments in jupyter notebook now fails

The following used to work in 1.1.0

# # Example
function foo()
    # a comment
    println("hello")
end

Now calling Literate.notebook results in an error:

[ Info: generating notebook from `~/misc/lit/lit.jl`
[ Info: executing notebook `lit.ipynb`
┌ Error: error when executing notebook based on input file: `~/misc/lit/lit.jl`
└ @ Literate ~/.julia/packages/Literate/1lTVT/src/Literate.jl:539
ERROR: LoadError: syntax: incomplete: premature end of input
in expression starting at string:1
when executing the following code block

```julia
function foo()

Stacktrace:
[1] error(::String) at ./error.jl:33
[2] execute_notebook(::Dict{Any,Any}) at /Users/simon/.julia/packages/Literate/1lTVT/src/Literate.jl:580
[3] (::getfield(Literate, Symbol("##15#17")))() at /Users/simon/.julia/packages/Literate/1lTVT/src/Literate.jl:536
[4] cd(::getfield(Literate, Symbol("##15#17")), ::String) at ./file.jl:96
[5] #notebook#13(::typeof(identity), ::typeof(identity), ::Bool, ::Bool, ::Bool, ::String, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::String, ::String) at /Users/simon/.julia/packages/Literate/1lTVT/src/Literate.jl:535
[6] notebook(::String, ::String) at /Users/simon/.julia/packages/Literate/1lTVT/src/Literate.jl:460
[7] top-level scope at none:0

Usage with package testing

I am trying to include example files that I create with Literate in my package tests, and need some advice.

Do you have a recommended way to perform a processing step that only shows Base.Test macros from my files if I ask for it?

Say I have a file Package/examples/example.jl

# # Using Literate
a = 1
@test a == 1 ## Should I use #src here or something?

And from Package/test/runtest.jl

using Base.Test

@testset "example.jl" begin
  include("../examples/example.jl")
end

BoundsError: attempt to access 7-element Array{Literate.Chunk,1} at index [0]

I'm getting the following error:

julia> Literate.notebook("juliacon_2019_interval_talk_25_july_2019.jl", "", execute=false)
[ Info: generating notebook from `~/Dropbox/talks/2019/juliacon_2019/juliacon_2019_intervals/juliacon_2019_interval_talk_25_july_2019.jl`
[ Info: not running on Travis, skipping links will not be correct.
ERROR: BoundsError: attempt to access 7-element Array{Literate.Chunk,1} at index [0]
Stacktrace:
 [1] getindex at ./array.jl:729 [inlined]
 [2] #parse#1(::Bool, ::Function, ::String) at /Users/dpsanders/.julia/packages/Literate/McVbf/src/Literate.jl:89
 [3] #parse at ./none:0 [inlined]
 [4] #notebook#11(::typeof(identity), ::typeof(identity), ::Bool, ::Bool, ::Bool, ::String, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::String, ::String) at /Users/dpsanders/.julia/packages/Literate/McVbf/src/Literate.jl:441
 [5] (::getfield(Literate, Symbol("#kw##notebook")))(::NamedTuple{(:execute,),Tuple{Bool}}, ::typeof(Literate.notebook), ::String, ::String) at ./none:0
 [6] top-level scope at none:0

The input file is

# # Constraints

# - **Constraint**: relation between

# - **Constraint satisfaction problem**
        > Find the **feasible set** of values satisfying all constraints

# - E.g.
# \begin{align}
#  x + y \le 4
#  x + 3y \le 9
#  -4 \le x, y \le 4
# \end{align}

using IntervalArithmetic, IntervalConstraintProgramming

C1 = @constraint x + y ≤ 4
C2 = @constraint x + 3y ≤ 9
C = C1 ∩ C2

X = IntervalBox(-4..4, 2)

paving = pave(C, X, 0.1)

using Plots
plot(paving.inner, aspect_ratio=1)
plot!(paving.boundary)

Julia 1.1, Literate 1.1.0

Behavior change with comments within functions/expressions?

Transforming an input like

function foo(x)
    # x times 2

    x * 2
end

with Literate.markdown("test.jl", ".", name = "test", documenter = false, credit = false) (using Literate v2.0.1) will split the function at the comment (# x times 2), resulting in an output like

------------------------------------------------------------------------------------

function foo(x)

x times 2

    x * 2
end

------------------------------------------------------------------------------------

so the Julia code becomes invalid. This used to be different in the past - though I'm not sure whether that was because the comment is indented or because it's within a function.

Would it be possible for Documenter to ignore comments within Julia functions/expressions/etc.?

Better error message?

on running a command like this
Literate.markdown("../example/safeFeet.jl", "src/example"; documenter=true), if the .jl file does not exist, an error like below is thrown. This is scary, and does not point to an obvious solution. At the very least, the file that failed should be shown.

ERROR: LoadError: SystemError: realpath: No such file or directory
Stacktrace:
 [1] #systemerror#39(::Nothing, ::Function, ::Symbol, ::Bool) at ./error.jl:106
 [2] systemerror at ./error.jl:106 [inlined]
 [3] realpath(::String) at ./path.jl:322
 [4] #markdown#10(::typeof(identity), ::typeof(identity), ::String, ::Bool, ::Bool, ::Pair{String,String}, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::String, ::String) at /Users/aviks/.julia/packages/Literate/5eSKp/src/Literate.jl:306
 [5] (::getfield(Literate, Symbol("#kw##markdown")))(::NamedTuple{(:documenter,),Tuple{Bool}}, ::typeof(Literate.markdown), ::String, ::String) at ./pair.jl:0

how do I use this with Documenter.jl

So, I understand that Literate.jl can be used to generate a Documenter.jl
compatible file. My question is rather trivial though -- how is this done?
What do I need to do so that makedocs invokes Literate.markdown to
make the temp file that is incorporated into my documentation? Further,
what makes Literate.jl attractive is that I could incorporate plots, does this
happen automagically or is there extra work needed? How could I take
the example provided and incorporate it into a whole made by Documenter?

Spurious @ inserted into notebook

Compiling this

# ### An example from Base
# `base/missing.jl` defines a bunch of operations when one or both operands are
# `missing`.  To avoid repetition this is done with eval-ing constructed expressions:
#
# ```julia
# # Unary operators/functions
# for f in (:(!), :(~), :(+), :(-), :(identity), :(zero), :(one), :(oneunit),
#           :(isfinite), :(isinf), :(isodd),
#           :(isinteger), :(isreal), :(isnan),
#           :(iszero), :(transpose), :(adjoint), :(float), :(conj),
#           :(abs), :(abs2), :(iseven), :(ispow2),
#           :(real), :(imag), :(sign))
#     @eval ($f)(::Missing) = missing
#     ## equivalent to writing
#     ## eval(:(($f)(::Missing) = missing))
# end
# ```

shows as (text block)

# Unary operators/functions
for f in (:(!), :(~), :(+), :(-), :(identity), :(zero), :(one), :(oneunit),
          :(isfinite), :(isinf), :(isodd),
          :(isinteger), :(isreal), :(isnan),
          :(iszero), :(transpose), :(adjoint), :(float), :(conj),
          :(abs), :(abs2), :(iseven), :(ispow2),
          :(real), :(imag), :(sign))
    @eval (@@0@@f)(::Missing) = missing))
end

Somehwo the $ gets transformed into @@0@@.

File size when embedding plots

I have some unreasonably large notebooks, because Literate.jl seems to be including several versions of every plot. Is there, or should there be, a way to disable this? I can't find anything in the documentation.

Adding Plots.default(fmt=:png) results in the displayed version being the PNG, but not only does this seem to be included twice, but the SVG version (which is huge because there are many plot points) still gets saved.

The generated notebook looks a bit like this:

{
 "cells": [
  {
   "outputs": [],
   "cell_type": "code",
   "source": [
    "using Plots\n",
    "Plots.default(fmt=:png)"
   ],
   "metadata": {},
   "execution_count": 1
  },
  {
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "Plot{Plots.GRBackend() n=1}",
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfQAAAEsCAIAAAC62dafAAAABmJLR0QA...",
      "text/html": [
       "<img src=\"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAfQAAAEsCAIAAAC62dafAAAABmJLR0QA...\">"
       ],
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"utf-8\"?>\n",
       "<svg xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" width=\"500\" height=\"300\" viewBox=\"0 0 2000 1200\">\n",
       "<defs>\n",
       "  <clipPath id=\"clip4400\">\n",
       "    <rect x=\"0\" y=\"0\" width=\"2000\" height=\"1200\"/>\n",
       "  </clipPath>\n",
       "</defs>\n",

Indent interferes with code chunk

It looks like an indent at the end of a code chunk interferes with the splitting of cells in the resulting notebook. MWE:

In "test.jl":

# Some text

a1 = [1 10 2 20; 3 30 0 0]
#- this works to split
a2 = [1 10 2 20;
    3 30 0 0]
#- this doesn't
a2 = [1 10 2 20;
    3 30 0 0]
a2
#- This does
println("What?")

julia> Literate.notebook("test.jl", "./")
[ Info: generating notebook from `~/computation/julia_playground/intro_julia/test.jl`
[ Info: not running on Travis, skipping links will not be correct.
[ Info: executing notebook `test.ipynb`
[ Info: writing result to `~/computation/julia_playground/intro_julia/test.ipynb`

Is there a way to figure out why a code block fails?

Executing the .md file (or a generated notebook) fails with:

Documenter: setting up build directory.
Documenter: expanding markdown templates.
 !! failed to run code block.

UndefVarError(:globe_toss_model) [src/clip_06_07t.md]
 !! failed to run code block.

# And several more ...

I expect it is the point where the Turing compiler does it's magic:

using StatisticalRethinking

# Define the data
k = 6; n = 9

# Define the model
@model globe_toss(n, k) = begin
  theta ~ Beta(1, 1) # prior
  k ~ Binomial(n, theta) # model
  return k, theta
end

# Create (compile) the model 
model = globe_toss(n, k)

The TuringTutorials use Weave.jl but it would be nice if Literate.jl could handle this as well. The above MWE is using StatisticalRethinking but that could also be using Turing. I did notice an issue with InteractiveUtils not being available, but I couldn't relate that to this issue.

Plots shown three times in nbviewer.js

When viewing notebooks made by Literate.jl using https://github.com/tuxu/nbviewer-app/ , plots are shown three times. This doesn't happen for notebooks saved directly by Jupyter.

The issue on that side is kokes/nbviewer.js#28 (starting with a now resolved problem, in which very little was shown). Here's the test document given there, all three versions are now the same size

# Test with a plot

sum(i for i=1:100)
# -
using Plots
plot(rand(20))

# The end.

Here's what fits on my screen:

Package compatibility caps

Ref: https://discourse.julialang.org/t/package-compatibility-caps/15301

Introduce negation filters

Would it be possible to have negative filters as well?
For example If I write some code, that I want to use in a Documenter.jl documentation, I could want to enhance keywords to be linked to the corresponding page, for example when speaking about my own Type MyType. In the code for Literate.jl. For both other cases (notebook and plain Julia) I don't want to clutter the notebook/example.

So for example I would write

#md # Now let's use [`MyType`](@ref)
#nb # Not let's use MyType
#jl # Now let's use MyType

but it would be shorter to write for example

#md # Now let's use [`MyType`](@ref)
#!md # Not let's use MyType

to say that the first line should be used in markdown and the second for all other types of renderings.

Of course for this example one could argue that the notebook might want to have just a marked up MyType so that we still need three versions, but then the example would just not fit my feature request ;)

Besides that: The idea of the package is quite nice, thanks for providing it :)

FR: Allow disabling the "generated with Literate" automatic text

As the files I generate with Literate become more and more this text added at the end becomes more and more disruptive.

Literate is awesome and deserves credit, but I'd prefer to give credit only once at the start of my documentation instead of at the end of each page. I have added to my front page:

using [Material+MkDocs](https://squidfunk.github.io/mkdocs-material/),  [Literate.jl](https://github.com/fredrikekre/Literate.jl) and [Documenter.jl](https://github.com/JuliaDocs/Documenter.jl/).

This is a Feature Request for allowing the extra Literate text to be disabled from the markdown function.

example does not work for notebook

I tried

using Plots
x = range(0, stop=6π, length=1000)
y1 = sin.(x)
y2 = cos.(x)
plot(x, [y1, y2])

and I got this in the notebook output

0 5 10 15 -1.0 -0.5 0.0 0.5 1.0 y1 y2

Is this still because #15?

Set up binder for deployed/generated notebook outputs

It will be great to have add a way to include a REQUIRE file in the gh-pages branch to use binder to interact with the generated outputs: https://github.com/binder-examples/demo-julia
Also, to have something similar to @__NBVIEWER_ROOT_URL__ to add the launch binder icon.

Issue with Reexport?

I am unsure but it seems that generating and executing a notebook prevents Reexport.jl from working properly.

MWE:

First, I create and save a sparse matrix using JLD2 via:

using JLD2, SparseArrays
M = sprand(10, 10, 0.5)
@save "a_sparse_matrix.jld2" M

Then I have this load_a_sparse_matrix.jl file to load the matrix M and do a simple operation on it (just doing -M here) using a module Foo. (Note the module Foo must reexport SparseArrays to be able to do operations on, well, sparse arrays like M.) I want to generate a notebook out of this file using Literate. Contents:

# Create a little module to load a sparse array using JLD2

module Foo # begin module
using JLD2
using Reexport
@reexport using SparseArrays
function load()
    @load "a_sparse_matrix.jld2" M
    return -M
end
end # module

# Use the new module

using .Foo

# Load the sparse array

A = Foo.load()

From there, starting a new Julia session and doing

include("load_a_sparse_matrix.jl")

works fine, but starting a new Julia session and doing

using Literate
Literate.notebook("load_a_sparse_matrix.jl", "./")

throws an error because SparseArrays was not reexported somehow and thus the -M operation is not possible for the reconstructed type:

ERROR: LoadError: MethodError: no method matching -(::getfield(JLD2.ReconstructedTypes, Symbol("##SparseArrays.SparseMatrixCSC{Float64,Int64}#364")))

What am I doing wrong? Should I not use Reexport.jl at all?

GR plots does not appear in notebook

My script contains a command that uses GR to plot. But when I set execute=true for notebook, it does not appear in the notebook (appears like the script is executed). Will this be possible to fix?

using GR
# ## Training target
function gaussian_pdf(n, μ, σ)
    x = collect(1:1<<n)
    pl = @. 1 / sqrt(2pi * σ^2) * exp(-(x - μ)^2 / (2 * σ^2))
    pl / sum(pl)
end

const n = 6
const maxiter = 20
pg = gaussian_pdf(n, 2^5-0.5, 2^4)
fig = plot(0:1<<n-1, pg)

Documentation is down

Thanks for the package!

Documentation seems to be down currently, I'm getting a 404.

Decouple from Travis (or document how to)

Certain aspects of generating Literate.jl-based documentation are tied known services like Travis, and it would be nice to generalize or at least be able to customize for other environments (e.g. GitLab CI). FWIW, Documenter.jl is also pretty Travis-specific, but only concerning deployment. Generating documentation on GitLab and pushing it manually to GitLab pages has been working great.

Issues I ran into:

Expansion of `@__REPO_ROOT_URL`

This is currently done as part of the default replacements, and throws a warning if the environment isn't known:

┌ Warning: expansion of `@__REPO_ROOT_URL__`, `@__REPO_ROOT_URL__` and  `@__REPO_ROOT_URL__` will only be correct if running in DocumentationGenerator.jl or Travis CI.
└ @ Literate ~/.julia/packages/Literate/wWXGE/src/Literate.jl:214

It's possible to customize this using a postprocess hook, e.g., for GitLab:

function set_repo_root(content)
    if haskey(ENV, "GITLAB_CI")
        commit = ENV["CI_COMMIT_REF_NAME"]
        repo_root_url = "https://github.com/.../blob/$(commit)"
        return replace(content, "@__REPO_ROOT_URL__" => repo_root_url)
    else
        return content
    end
end
Literate.markdown(...; postprocess=set_repo_root)

However, that runs into:

Empty value for `EditURL`

If not running on Travis, the EditURL template is empty:

Literate.jl/src/Literate.jl

Lines 364 to 377 in 123e4bf

    
           repo = get(ENV, "TRAVIS_REPO_SLUG", nothing) 
        
           if repo === nothing 
        
               path = "" 
        
           else 
        
               pkg = String(first(split(last(split(repo, '/')), '.'))) 
        
               pkgsrc = Base.find_package(pkg) 
        
               if pkgsrc === nothing 
        
                   path = "" 
        
               else 
        
                   repo_root = first(split(pkgsrc, joinpath("src", pkg * ".jl"))) 
        
                   path = relpath(inputfile, repo_root) 
        
                   path = replace(path, "\\" => "/") 
        
               end 
        
           end

This leads to the following invalid markdown headers:

```@meta
EditURL = "@__REPO_ROOT_URL__/"
```

As a workaround, I'm spoofing the environment variable now:

withenv("TRAVIS_REPO_SLUG" => "username/repository") do
    Literate.markdown(...; postprocess=set_repo_root)
end

Mention how to get rid of GR error messages in the docs?

I was annoyed by all the error messages that I got when creating Plots with Literate + GR (using Plots) on Travis. Maybe it could be mentioned in the documentation that one can get rid of these error messages by setting the environment variable GKSwstype to nul before loading Plots (or GR) (see https://discourse.julialang.org/t/deactivate-plot-display-to-avoid-need-for-x-server/19359/2), e.g., by adding it to the documentation stage for Travis or make.jl?

fail to suppress the output

Hello, I used to add ; at the end line of a block of code to suppress the output. But I couldn't do it now:

How may I suppress the output in current version? Thank you!

	if isa(chunk, MDChunk)
	cell["cell_type"] = "markdown"
	cell["metadata"] = Dict()
	lines = String[x.second for x in chunk.lines] # skip indent
	@views map!(x -> x * '\n', lines[1:end-1], lines[1:end-1])
	cell["source"] = lines
	cell["outputs"] = []

	else # isa(chunk, CodeChunk)
	cell["cell_type"] = "code"
	cell["metadata"] = Dict()
	@views map!(x -> x * '\n', chunk.lines[1:end-1], chunk.lines[1:end-1])
	cell["source"] = chunk.lines
	cell["execution_count"] = nothing
	cell["outputs"] = []
	end

	if haskey(ENV, "HAS_JOSH_K_SEAL_OF_APPROVAL") # Travis CI
	repo_slug = get(ENV, "TRAVIS_REPO_SLUG", "unknown-repository")
	deploy_folder = if get(ENV, "TRAVIS_PULL_REQUEST", nothing) == "false"
	get(ENV, "TRAVIS_TAG", get(user_config, "devurl", "dev"))
	else
	"previews/PR$(get(ENV, "TRAVIS_PULL_REQUEST", "##"))"
	end
	cfg["repo_root_url"] = "https://github.com/$(repo_slug)/blob/$(edit_commit)"
	cfg["nbviewer_root_url"] = "https://nbviewer.jupyter.org/github/$(repo_slug)/blob/$(deploy_branch)/$(deploy_folder)"
	cfg["binder_root_url"] = "https://mybinder.org/v2/gh/$(repo_slug)/$(deploy_branch)?filepath=$(deploy_folder)"
	if (dir = get(ENV, "TRAVIS_BUILD_DIR", nothing)) !== nothing
	cfg["repo_root_path"] = dir
	end

	repo = get(ENV, "TRAVIS_REPO_SLUG", nothing)
	if repo === nothing
	path = ""
	else
	pkg = String(first(split(last(split(repo, '/')), '.')))
	pkgsrc = Base.find_package(pkg)
	if pkgsrc === nothing
	path = ""
	else
	repo_root = first(split(pkgsrc, joinpath("src", pkg * ".jl")))
	path = relpath(inputfile, repo_root)
	path = replace(path, "\\" => "/")
	end
	end

fredrikekre / literate.jl Goto Github PK

literate.jl's Introduction

Literate

Related packages

literate.jl's People

Contributors

Stargazers

Watchers

Forkers

literate.jl's Issues

MWE:

Expansion of @__REPO_ROOT_URL

Empty value for EditURL

Recommend Projects

Recommend Topics

Recommend Org

Expansion of `@__REPO_ROOT_URL`

Empty value for `EditURL`