zimbatm / mdsh Goto Github PK

It should be possible to switch modes from > to $ or reversely, re-run mdsh and have the new output.

This means expanding the clean regexp to support both modes at the same time.

Nice tool

Here's how I built it

nix-shell -p cargo --command "cargo build"

Maybe add instructions along those lines?

By default the work_dir should be the folder containing the input file. That way, calling mdsh path/to/README.md will give the same output as cd path/to && mdsh.

If the input is - then the work_dir shouldn't change.

I tried mdsh on the following readme.md:

Using input=File("./README.md") output=File("./README.md") work_dir=Parent(".") clean=false frozen=false
`$ koji-tool --version`
`$ koji-tool --help`
`$ koji-tool builds --help`
`$ koji-tool tasks --help`
`$ koji-tool install --help`
`$ koji-tool find`
` cabal-rpm builddep && cabal install || stack install`
thread 'main' panicked at /var/home/petersen/.cargo/registry/src/index.crates.io-6f17d22bba15001f/mdsh-0.7.0/src/main.rs:387:25:
WTF, not supported
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Is there an easy way to ignore some backquotes?

the proposed syntax to set a variable would be:

`! foo=bar`

once the variable is set in the document, all further programs will have it in their environment variables.

It should also be possible to set a variable with the output of a program with:

`! foo=$(cat /etc/passwd)`

Take the following in.md file as an example:

`$ echo hi`

\```
bye
\```

After running mdsh --input in.md --output out.md, out.md looks like the following:

`$ echo hi`
\```
hi
\```

Notice how the "bye" code block is now missing.

This can be worked around by adding a dummy code block or HTML comment after command substitutions, e.g. <!-- --> or

\```
\```

README.md:

`$ echo foo`



## Heading

mdsh

`$ echo foo`


```
foo
```
## Heading

mdsh

`$ echo foo`

```
foo
```
## Heading

mdsh

`$ echo foo`
```
foo
```
## Heading

Then it reaches a fixpoint.

As far as I understand mdsh, it is currently used to (re-)generate markdown snippets. It would be nice if a command could update an existing snippet, so we don't have to generate the entire thing in code but rather modify the tiny bit we are interested in.

This could be implemented by injecting the target block as stdin:

& sed -e 's/foo/bar/'

loooooooooooooooooooooooong snippet foo

Run mdsh. foo is fed as stdin to sed. Result:

& sed -e 's/foo/bar/'

loooooooooooooooooooooooong snippet bar

This could for example be used to update things like versions in more complex markdown blocks.

Matklad spent some time a while back reviewing the code and I just found it on a file on my computer. Here is what the file says:

• use a struct for all the regexp
• use once_cell crate instead of lazy_static!
  https://crates.io/crates/once_cell
• split parser and runner

when running mdsh on a file, it might make sense to insert a comment header to the file that explains how to run mdsh on the file again. If the header would be to contain a version number it would also allow upgrade scenarios in case the format has changed between version.

Technically this is doable with > ./somecommand that writes the image and outputs the markdown link to it. Maybe there is a shorter way to do that...

Hi there! I stumbled across your project while searching for something related to this other mdsh project, and just thought I should mention the possible naming conflict. The mdsh I wrote is a tool for doing both data processing and execution of blocks within markdown, implemented as a shell script. It has third-party extensions such mdshi, which is actually kind of close to some of what your mdsh is for.

Anyway, I can't really lay claim to the name even if I used it first, but it might be nice if you named yours something more about its function (md-live? mdexec? mdgen?) and less about its shellness, since yours isn't actually written in shell and so the shell aspect of it is kind of incidental. Whereas mine is written in shell and compiles markdown into shell scripts, so the shellness is part of its essential nature. ;-)

Also, as an FYI, there's a tool you might be interested in called cram that does something very similar to your mdsh as well, but it's for testing anything that can be tested by comparing a file's content with the output of shell commands, . It processes indented blocks like this:

$ ls
foo
bar

...by running the commands that begin with $ (optionally continued with > ), and verifying the output matches the input. If you set its indentation width to 4, then it can process markdown indented code blocks, and it can also be used to generate the data in the first place, because if the output doesn't match the input file, you can tell it to update the input file with the new output.

I actually use cram for testing mdsh (my mdsh, that is) and other shell tools, as it's nice to be able to have the documentation examples tested, as well as being able to document the tests. :-)

So, these all may or may not be projects you might or might not want to mention in the "Related Projects" part of your readme. Hope you find some or any of this illuminating and/or useful.

because it's nice

Just found this interesting project, thanks.
It seems close to what I need.

I think it would be really nice if mdsh could also support a code block format like this:

```shellsession
$ echo hello; echo there
hello
there
```

Instead of having to write:
`$ echo hello` as shellsession
```shellsession
hello
```

Particularly since the as ... will be rendered in the README, right? which is really not desirable.

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• fix(deps): update rust crate lazy_static to v1.5.0

Detected dependencies

• Check this box to trigger a request for Renovate to run again on this repository

I just tried to use mdsh on a command that outputs some color and that currently has the undesired effect of yielding the escape codes verbatim in the output:

�[32m[please]:�[0m Running sanity checks:

- Nix installed :  �[32mOK�[0m
- contrail channel: �[32mOK�[0m
- contrail cache: �[32mOK�[0m
- kvm support: �[32mOK�[0m

All essential tests passed.

It would be great if there could be an option to filter those.

Thank you for your work and I apologize if this is an obvious mistake on my part.
I was expecting mdsh to leave unfenced code alone.

For example, in a file containing

`leave-me-alone`

`$ echo 1`

I would expect the output to be

`leave-me-alone`
`$ echo 1`

```
1
```

Instead, I get

Using input=File("input.md") output=StdHandle work_dir=Parent("/tmp") clean=false frozen=false
`leave-me-alone`
thread 'main' panicked at src/main.rs:414:25:
WTF, not supported
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

This panic!("WTF, not supported") is there becasue mdsh expects leave-me-alone to be a RE_MATCH_FENCE_COMMAND, a RE_MATCH_MD_COMMAND, or a RE_MATCH_VAR_COMMAND, which of course it isn't. Instead of panicking, shouldn't the code simply leave the original_line.to_string() in place undisturbed? After all, there was no fence there.

It would be nice if the outputted blocks could also include the highlighting format.

Where in the following block do we give the syntax?

`[$ code.rb](code.rb)`

So that it outputs:

```ruby
require "pp"

pp ({ foo: 3 })
```

I think stderr should be included in the output - or at least there should be an option to include it.

mdsh should run all commands, fail if one command returned non-success and then output all commands that failed (with their stdout and stderr), in a way that is nice to integrate with CI.

Reasoning: I want to be able to use mdsh as a kind of doctest for README.md files, to make sure the examples always work.

I will prepare a PR.

for long-running processes, it might be nice to record the output, and have a "player" to replay it, instead of having it inline in the document.

something like asciinema

This is another one for CI:

mdsh --frozen should check whether the new README is exactly the same as the old one, and fail if they are not the same.

We can achieve something similar with a trivial:

git diff --exit-code

after running mdsh, but that picks up all files that have been modified by the commands in the md file, which might or might not be what the user wants.

Alternatively, a wrapper script like

INPUT="$1"
tmp=$(mktemp)
cp "$INPUT" "$tmp"
mdsh "$INPUT"
diff "$tmp" "$INPUT"

might also be enough for most use cases.

I’m not sure whether --frozen is necessary in the end, but if it isn’t the best practices should be documented.

When generating documentation, the command itself is usually not very interesting and it may distract the reader.

This can replace for example terraform-docs-updater.

Proposed doc:

Invisible Includes

When generating documentation, the command itself is usually not very interesting and it may distract the reader.

Syntax regexp:
TBD

Invisible Includes work similarily to code blocks but with the comment syntax.

• $ loads the file and embeds it as a code block
• > loads the file and embeds it as markdown

Examples:

<!-- $ wc -l <sponsors.txt -->
```
42
```

<!-- > terraform-docs markdown ./my-module -->
<!-- BEGIN mdsh -->
# Inputs
[... actual generated docs ...]
<!-- END mdsh -->