Jupyter notebook to emacs' org-mode format about jupytext HOT 13 OPEN

@srnnkls , if that can help, I have prepared a branch with a tentative implementation of the org format based on pandoc (back and forth), see the last three commits here: https://github.com/mwouts/jupytext/commits/org_pandoc:

• The first commit 086178b adds the org:pandoc format to Jupytext (and simply calls pandoc)
• The second commit 93f6478 activates the round trip tests on that format
• And the third commit b5432c7 adds the files generated by the round trip tests

Note that the round trip test does not work. Is it correct that pandoc's conversion only works in one direction (ipynb to org)? Any way, feel free to experiment with this, and replace either converter with your favorite one.

Two additional comments:

• Jupytext removes the outputs before calling pandoc (because they are preserved in the .ipynb file), so no headache with outputs...
• pandoc is used for the md:pandoc format, but that is not Jupytext's default markdown format (see the example files at https://github.com/mwouts/jupytext/tree/master/demo )

from jupytext.

Hello @srnnkls, thanks for reaching out!

Well, I am afraid that we've not made big progresses here... As you saw above, one can use ox-ipynb, by @jkitchin, to convert org-mode documents to ipynb, but I am not aware of a tool doing the opposite conversion.

It would help much if you could write two functions that convert a notebook object to its text representation, and vice versa. Maybe at first we should target a limited conversion, i.e.

• keep the content of code and markdown cells verbatim in the org-mode representation,
• and ignore the cell and notebook metadata.

Ideally that first version should be compatible (should use?) ox-ipynb.

These two functions should be called in jupytext.reads and jupytext.writes if the format name matches the name you choose for this format, e.g. "org". You should also add a description of the new format in formats.py.

NB: If you like, you can also provide these two functions in a separate Python package, and add that package as an optional dependency in Jupytext - like we did for the md:pandoc or the md:myst formats.

from jupytext.

You can use pandoc to convert ipynb to org right now. It is technically possible to do a round trip conversion, but you will lose things like cell metadata, and probably some formatting.

It would not be hard to use elisp to convert an ipynb to an org file containing markdown blocks and code blocks. Also not hard to do that in Python. It might be tricky either way to deal with the results.

from jupytext.

Just a note for the people who subscribed to this thread... I have opened an issue at pandoc regarding the round trip ipynb-org-ipynb: jgm/pandoc#6367.

The issue also raises the question of how the notebook cells should be represented in org mode. Personally I think that the representation should remain as simple as possible, because the users are going to type it 😃 But obviously, it is simpler for the programmers (and maybe also safer in the long run?) to use explicit cell markers. Anyway... if you have an opinion about this, please follow the pandoc thread as well!

from jupytext.

The second and third links provided by Doug at #61 provide examples of code blocks with cell metadata. The first and fourth link point to ORG to Notebook converters, writen in Emacs Lisp.

Interesting facts:

• ORG is not markdown! If we want to support styling, round trip conversion is going to be tough!
• ORG can include results. Currently Jupytext supports no format with outputs. Probably we don't want to implement the matching with Jupyter outputs.
• More generally, ORG has support for many features that are not in Jupyter notebooks (tables, etc...). Is it acceptable for the users to have them, say, as raw cells in Jupyter?

from jupytext.

Maybe org-mode and jupytext aren't going to be a good match afterall. But my friends are still on the quest for combining emacs + notebooks...

from jupytext.

No problem! @dsblank , we'll try to make your friends happy 😄

Could you ask them to write a sample org file with text, code, a header, and a few org specific sections, and tell us how they imagine the corresponding notebook? Or even, could they contribute a test similar to test_read_simple_julia.py, but for org mode?

from jupytext.

What‘s the current state with this? Do you already have something to work on? I’d like to help.

from jupytext.

Oh, that's interesting! What we could do, then, is to use ox-ipynb on one side, and pandoc on the other side, plug this into Jupytext's collection of test notebooks, and see how well this work 😃 (or not). If time permits I will give it a try!

from jupytext.

Hey, thank's for taking the time to discuss options. Another option for ipynb to org conversion is nbcorg. I'm using nbcorg and ox-ipynb at the moment. That works well but I'd like to have a emacs independet solution that works from jupyter and the command-line. That is what brings me here. As a nbconvert plugin nbcorg uses jinja templates.

I will have a look at jupytext-reads and jupytext-writes. @jkitchin, you are right. I'm not sure about how to deal with results too. nbcorg includes results as EXAMPLE blocks what I'm not a fan about.

from jupytext.

Okay, I just realised that for markdown you use pandoc under the hood as well. I will write a round trip conversion test first, then. I don't know when I find the time to work on this but probably sometime within the next week.

from jupytext.

Wow, thank you! I‘ll have a look. Removing results before converting to text and just adding them back to the notebook version is a nice approach; good to know about it.

from jupytext.

Recent developments on the Pandoc side of things: jgm/pandoc#6367 (comment)

from jupytext.