Code Monkey home page Code Monkey logo

embedme's Introduction

EmbedMe

Simple utility to embed source files into markdown code blocks why tho?

npm version Build Status Commitizen friendly

Demo

Usage

With a README.md in your current working directory, add a code block for one of the supported file types and start the code block simply with a comment with the path to a file. For example

This is a *markdown* document with a code block:

```ts
// example.ts
```

Next, run the following

npx embedme README.md

Et voilà! Your README.md file will be updated with the content of your source file:

This is a *markdown* document with a code block:

```ts
// example.ts

export function helloWorld(name: string): string {
  return `Hello ${name}!, how are you today?`;
}

```

As the comment is preserved, you can happily re-run embedme and it will run again but there will be no changes.

Features

$ embedme --help
Usage: embedme [options] [...files]

Options:
  -V, --version              output the version number
  --verify                   Verify that running embedme would result in no
                             changes. Useful for CI
  --dry-run                  Run embedme as usual, but don't write
  --source-root [directory]  Directory your source files live in in order to
                             shorten the comment line in code fence
  --silent                   No console output
  --stdout                   Output resulting file to stdout (don't rewrite
                             original)
  --strip-embed-comment      Remove the comments from the code fence. *Must* be
                             run with --stdout flag
  -h, --help                 display help for command

Partial Snippets

Very often you only want to highlight a small part of a file, to do so simply suffix the filename with the GitHub line number syntax, e.g. path/to/my/file.ts#L20-L30.

Multi Language

embedme simply uses the file type hint in a code fence to choose a strategy for finding the commented filename in the first line of the code block. This is a relatively trivial regular expression, so many more languages can be supported in future

Here's a list of file types supported by this utility, if you have a need for another language please feel free to contribute, it is easy!

// src/embedme.lib.ts#L44-L82

enum SupportedFileType {
  PLAIN_TEXT = 'txt',
  TYPESCRIPT = 'ts',
  JAVASCRIPT = 'js',
  REASON = 're',
  SCSS = 'scss',
  RUST = 'rust',
  JAVA = 'java',
  CPP = 'cpp',
  C = 'c',
  HTML = 'html',
  XML = 'xml',
  MARKDOWN = 'md',
  YAML = 'yaml',
  JSON = 'json',
  JSON_5 = 'json5',
  PYTHON = 'py',
  BASH = 'bash',
  SHELL = 'sh',
  POWERSHELL = 'ps1',
  GOLANG = 'go',
  OBJECTIVE_C = 'objectivec',
  PHP = 'php',
  C_SHARP = 'cs',
  SWIFT = 'swift',
  RUBY = 'rb',
  KOTLIN = 'kotlin',
  SCALA = 'scala',
  CRYSTAL = 'cr',
  PLANT_UML = 'puml',
  MERMAID = 'mermaid',
  CMAKE = 'cmake',
  PROTOBUF = 'proto',
  SQL = 'sql',
  HASKELL = 'hs',
  ARDUINO = 'ino',
  JSX = 'jsx',
  TSX = 'tsx',
}

Alternate embedding syntax

It is recommended to use the syntax described above as it is a good hint for readers and maintainers where the source of this file is, however in some situations you may want to omit the comment in the code block, but still benefit from the embedding behaviour of embedme.

This can be achieved by preceding the code block with a markdown comment in the form of <!-- embedme path/to/your/file.txt -->

For example:

<!-- embedme example.ts -->
This is a *markdown* document with a code block:

```ts
```

Will result in the following output

<!-- embedme example.ts -->
This is a *markdown* document with a code block:

```ts
export function helloWorld(name: string): string {
  return `Hello ${name}!, how are you today?`;
}

```

Glob matching

If you want to run embedme over multiple files, you can use glob matching, i.e.

embedme "src/**/*.md"

Note embedme supports both quoted globbing and unquoted. Be careful using unquoted globbing as this can lead to behaviour that is not portable between different operating systems.

If you're using Windows, you must use forward slashes (/) to denote path separators.

You can also pass multiple separate glob patterns to match multiple sets of files

example:

embedme "src/**/*.md" "docs/**/*.markdown"

CI Checks

If you're using continuous integration, you can pass the flag --verify to embedme to check that there are no changes expected to your files. This is useful for repositories with multiple contributors who may not know about embedme, and also for yourself as a sanity check that you remembered to run it after updating sample code!

Output to stdout

Don't want to rewrite the file in-place? That's ok too - you can pass the flag --stdout to have the output pass to stdout - this will allow you to redirect the output to another file.

Additionally, in this mode a --strip-embed-comment flag is available, which allows embedme to exclude the matched comment from the output. This isn't generally recommended as the comment is generally unobtrusive, and will really help maintainers to know where they should go to update the file.

Example

# readme/output-to-std-out.sh

embedme --stdout README.template.md > README.md

Note that with --stdout flag the log output from embedme is redirected to stderr so you can still see the logs but the output can be redirected.

Ignoring files

By default embedme uses the local .gitignore file to exclude any files that match your input but are ignored. You can customise this ignore behavior by creating a .embedmeignore file, which uses the same syntax as .gitignore. This file will be used instead of .gitignore, not merged.

Why?

Why do I want this utility? Writing code in a markdown document is not difficult?

True, however it is difficult to know when your documentation files become out of date - if you're introducing a breaking change, having your example code actually using your library guarantees it will be correct.

How can just having my examples in the language give me guarantees?

For starters if you're using a typesafe language (e.g. Typescript) you will get compiler errors, and secondarily you really should be writing unit tests on your example code. As simple as it might be, how embarrassing is it if your example doesn't even work?!

embedme's People

Contributors

ayazhafiz avatar aziaziazi avatar blu3r4y avatar dependabot[bot] avatar elektronikworkshop avatar guitarrapc avatar idkjs avatar jci-aws avatar thewalla07 avatar zakhenry avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

embedme's Issues

Add optional `--directory-root` flag to allow for shorter comment paths

It would be useful rather than having to go

```ts
// src/readme/app.component.ts
```
npx embedme README.md

You could instead do

```ts
// app.component.ts
```
npx embedme --directory-root=src/readme README.md 

This would keep the comment nice and lean and look less unexpected for people who don't know about embedme

Return error code when file is renamed or moved

When using the --verify flag, if a file being referenced is renamed or moved, embedme won't return a non-zero error code. This would be helpful in CI, to ensure that the code being referenced actually exists.

index.ts

function add(x: number, y: number): number {
  return x + y
}

README.md

```ts
// ./index.ts

function add(x: number, y: number): number {
  return x + y
}
```
mv index.ts add.ts # rename index.ts to add.ts
npx embedme README.md --verify # run embedme with --verify
echo $? # get return code

Current Behavior

Verifying...
  Analysing README.md...
   README.md#L1-L7 Found filename ./index.ts in comment in first line, but file does not exist at /embedme-test/index.ts!
0

Desired Behavior

Verifying...
  Analysing README.md...
   README.md#L1-L7 Found filename ./index.ts in comment in first line, but file does not exist at /embedme-test/index.ts!
1

I'm free to open a PR if you're open to it.

Consider using node-glob instead of shell expansion

First, awesome lib! Using this for our internal developer's wiki and it's such a nice & simple solution.

Context

My package.json

{
 "scripts": {
    "build": "embedme **/*.md"
  },
  "devDependencies": {
    "embedme": "^1.15.0",
  }
}

Problem

Using shell expansion for globbing leads to inconsistencies across platforms.

Because npm is using /bin/sh, and I'm on OSX, I'm stuck with Bash 3.x, and globbing doesn't work the same as it does my usual shell (zsh). Results may vary among other devs on my team.

So given the context above, when I run npm run build a file like foo/bar/baz/doc.md will not be included. However, that file will be included for developer whose /binb/sh points to Bash 4.x, for example.

Proposed Solution

Many node libraries use node-glob to solve this. The downside is that a user must quote glob patterns in their scripts entries, however this pattern is so common that it's easy to find this solution when things don't work as expected.

Workarounds

Currently, I'm handling this (rather naively) with this:

// scripts/build.js

const glob = require('glob');
const { execSync } = require('child_process');

const PATTERN = '**/*.md';

function init(pattern) {
  glob(pattern, {}, function(error, files) {
    const cmd = ['embedme', ...files].join(' ');
    execSync(cmd, { stdio: 'inherit' });
  });
}

init(PATTERN);
{
  "scripts": {
    "build": "node scripts/build.js"
  },
  "devDependencies": {
    "embedme": "^1.15.0",
    "glob": "^7.1.4"
  }
}

I'd be more than happy to contribute with a PR, but I wanted to raise this for discussion first.

Question: How to embed Markdown files?

I have a script that creates some Markdown file (say, output.md) and I would like to include this file in another file (say, `README.md).

I tried the following in REAMDE.md:

This is the output of the build step:

<!-- embedme some/dir/output.md -->

but this seems to have no result.

If I add a code fence below, I get the warning

Output snippet for file some/dir/output.md contains a code fence. Refusing to embed as that would break the document

The warning is correct, as the file to be included indeed contains a code fence. But I would like to embed it as-is - not within a code fence.

Is this possible?

(Sorry for opening so many issues, but this tool is so cool, and it really helps us cleaning up the documentation of a large project!)

Allow regex parsing of files

Hi,

I got an idea for a new feature: Regex parsing which contents of a file should be extracted.
As files tend to change, it would be very helpful to enhance the parser to allow something like:
my-file.json#R:/"scripts".*\]/imgs
which should match the following contents of my-file.json:

... other content here...
"scripts": [
    "scripts/general-env-variables.sh",
    "scripts/ca-certificates.sh",
    "scripts/apt-install.sh",
    "scripts/timezone-fix.sh",
    "scripts/python-pip-repository.sh",
    "scripts/cloudfoundry-cli.sh",
]
... other content here...

See here for example: https://regex101.com/r/URfVWn/1

This way we can extract in a more flexible / reliable way and do not need to adjust start and end of line if source files are changing...

What do you think?

Best wishes,
koseduhemak

Add line numbering to output so when there is an error it is clear where

Currently output looks like

...
    Embedded code snippet (4 lines) from file sample.kt
    Embedded code snippet (6 lines) from file sample.scala
    Embedded code snippet (8 lines) from file sample.cs#L6-L13
    Unsupported file extension [binary], supported extensions are txt, ts, js, scss, rs, java, cpp, c, html, xml, md, yaml, py, bash, sh, go, objectivec, php, cs, swift, rb, kotlin, scala, skipping code block
...

it would be better if it was

...
    test/fixture.md#L242-L249 Embedded code snippet (4 lines) from file sample.kt
    test/fixture.md#L253-L262 Embedded code snippet (6 lines) from file sample.scala
    test/fixture.md#L266-L277 Embedded code snippet (8 lines) from file sample.cs#L6-L13
    test/fixture.md#L281-L283 Unsupported file extension [binary], supported extensions are txt, ts, js, scss, rs, java, cpp, c, html, xml, md, yaml, py, bash, sh, go, objectivec, php, cs, swift, rb, kotlin, scala, skipping code block
...

Maybe even a verbose mode could output the offending code fence.

Support Pandoc fenced code blocks

In Pandoc, code blocks may not just start with the extension like

```json
{"key": true}
```

but may also have more information in the line after the starting backticks (see https://pandoc.org/MANUAL.html#fenced-code-blocks):

```{.json caption="Some JSON file"}
{"key": true}
```

Unfortunately, code like this is currently rejected with a message like

Unsupported file extension [{.json], supported extensions are txt, ts, js, scss, rust, java, cpp, c, html, xml, md, yaml, json, json5, py, bash, sh, go, objectivec, php, cs, swift, rb, kotlin, scala, cr, puml, mermaid, skipping code block

I would really love to use embedme together with Pandoc!

Request: Support for .ino files (Arduino sketches)

Hi Zak,

thanks for this great tool - one I was looking for for a long time.

I'm currently documenting some Arduino libraries and I use your tool to embed the example code into the readme. It works perfectly but spits out a warning of course:

Embedding...
Skipped 0 files ignored in '.gitignore'
  Analysing README.md...
   README.md#L5-L48 Embedded 40 lines from file examples/lcdpong-butenc/lcdpong-butenc.ino#L20-L59
   README.md#L67-L83 Unsupported file extension [c++], supported extensions are txt, ts, js, scss, rust, java, cpp, c, html, xml, md, yaml, json, json5, py, bash, sh, go, objectivec, php, cs, swift, rb, kotlin, scala, cr, puml, mermaid, cmake, proto, sql, hs, skipping code block
   README.md#L122-L129 No code extension detected, skipping code block...
   README.md#L137-L139 No code extension detected, skipping code block...
   README.md#L140-L141 Code block is empty & no preceding embedme comment, skipping...
   README.md#L144-L150 No code extension detected, skipping code block...
  Writing README.md with embedded changes.

The embedme-code was

```cpp
// examples/lcdpong-butenc/lcdpong-butenc.ino#L20-L59
```

The .ino are Arduino sketches and are plain C++, perhaps this could be implemented as an alias or so?

ASK: How one would use that with a Hugo website for example?

I'm running a bunch of code excerpts in Markdown files and I'm running issues on updating npm libs, they eventually broke my code. I found your repo and this is quite interesting, but due to my lack of the JS landscape in general, how would one would use this in a pipeline/bundler or CI/CD?

Could you elaborate on an example for that?

sql is in the list - but it says it's not support

This is a genius package! 🚀

I tried a dry run of an sql snippet but it gave me:

README.md#L65-L67 Unsupported file extension [sql], supported extensions are txt, ts, js, scss, rust, java, cpp, c, html, xml, md, yaml, json, json5, py, bash, sh, go, objectivec, php, cs, swift, rb, kotlin, scala, cr, puml, mermaid, cmake, proto, skipping code block

Something Wrong a comment start with // in code block

if code block with comment start with //
nothing writing into markdown, after Embedded xx lines from file

TEST.md#L420-L427 No comment detected in first line for block with extension go
   TEST.md#L437-L446 No comment detected in first line for block with extension go
   TEST.md#L460-L474 Unsupported file extension [go	], supported extensions are txt, ts, js, re, scss, rust, java, cpp, c, html, xml, md, yaml, json, json5, py, bash, sh, go, objectivec, php, cs, swift, rb, kotlin, scala, cr, puml, mermaid, cmake, proto, sql, hs, ino, jsx, tsx, skipping code block
   TEST.md#L482-L496 No comment detected in first line for block with extension go
   TEST.md#L514-L527 No comment detected in first line for block with extension go
   TEST.md#L537-L582 Embedded 42 lines from file ../GoProject/src/go_code/chapter03/floatdemo08/main.go
   TEST.md#L588-L633 Embedded 42 lines from file /Users/jayli/Desktop/Golang/GolangPlayground/GoProject/src/go_code/chapter03/floatdemo08/main.go

allow single lines (#L1 instead of #L1-L1)

i think it would be great since sometimes i only want to display a single line, and the extra duplicate line is cluttering. the only change to the code would be here, right?

if you could implement it real fast that would be really fantastic, thanks. i will try to implement it and open a pull request if you are unable to do so.

Trim code when writing to file

If I've got a file test.ts matching the following:

interface A {
  hello: string;
}

(note the empty line at the end as it's usually the configuration of IDE, prettier etc)

Within a README if I've got

image

After running embedme it's:

image

While I'd expect

image

I think that should be the default behaviour but if you want to let more freedom maybe add a flag to manage that case?

Add -w —watch mode

It would be really useful to be able to start embedme in watch mode so that I can be editing the source code files and see the markdown document update automatically.

Things to consider

  • cross platform file watching
  • should only watch files that are embedded
  • deregister from watch when file is removed

Add contribution guidelines

This project is contributed to by many different developers often from languages outside of js, so it would be good to have a quick "this is how you add a language" tutorial. The testing harness is also non-obvious, so an explanation of how ava snapshotting is used would help

see also #54 where https://github.com/idkjs would have been able to more easily get reasonML included without testing issues

When the file trying to embed starts with 2 white spaces it won't embed the content

I had some sql files that were copied from a database tool that needed embedding. I did not know why it would not embed the content for some of these, I'd get this warning: "Changes are trailing whitespace only, ignoring" . As soon as I removed the 2 heading white spaces(or tab) from the beginning of the first line of the SQL file it embed generate my code.

[Request] add support for .tsx files

For example, I cannot set code block syntax to tsx, which is unfortunate:

README.MD:

```tsx
// source.tsx
```
$ npx embedme README.MD
README.MD#L1-3 Unsupported file extension [tsx], supported extensions are txt, ts, js, re, scss, rust, java, cpp, c, html, xml, md, yaml, json, json5, py, bash, sh, go, objectivec, php, cs, swift, rb, kotlin, scala, cr, puml, mermaid, cmake, proto, sql, hs, ino, skipping code block

Language extension with metadata isn't parsed correctly

One can add metadata after the first word in the info string of a code fence.

```js startline=3
// file.js

Current Behavior

This isn't parsed correctly at the moment and produces the following error.

file.md#L10-L15 Unsupported file extension [js startline=3], supported extensions are [...], skipping code block

Expected Behavior

The CommonMark specification says the following on that:

An info string can be provided after the opening code fence. Although this spec doesn’t mandate any particular treatment of the info string, the first word is typically used to specify the language of the code block."

They also provide some examples - see here - where only the first word (here: "ruby") would be extracted from a code fence like that.

```    ruby startline=3 $%@#$
def foo(x)

Some frameworks rely on this metadata, e.g., reveal.js uses it for highlighting certain lines, see here.

Resolution

Only parse the first word and ignore metadata when identifying the extension.

Bug introduced on #22: ALL changes result in "Changes are trailing whitespace only, ignoring"

#22 appears to have introduced a bug in my use case. I am not using prettier, just running out of the box.

This code:

  if (replacement.substr(-3).trimRight() === substr.substr(-3).trimRight()) {
    log({ returnSnippet: substr }, chalk.gray(`Changes are trailing whitespace only, ignoring`));
    return substr;
  }

ALWAYS returns true, when it compares the existing block:

```js
// gulpfile.js
``` 

With the replacement:

```js
// gulpfile.js
"use strict";

const abc = 123;
...
``` 

It's comparing the trailing fences in every case, thus any change is considered "trailing whitespace".

[Feature Request] Support start and end tags for embedding snippets of code

I have been using this tool and find it very useful to ensure our documentation is updated every time code changes. Most of the documentation requires only a small snippet from the source file and for which we are currently using line numbers.
The challenge with line numbers is if there are multiple snippets in the same source file, updating one snippet may cause the line numbers of all other snippets to change requiring a lot of effort to update the line numbers in markdown files.

One proposal to solve this is by having start and end tags for code sections that should be embedded. So, as new lines are added or removed, the tags will keep the code snippets in tact.

For e.g.

In source file (Java)

// embedme_start: tag-name
code snippet to be embedded
// embedme_end: tag-name

In markdown file:

<!-- embedme /path/to/source/file/Sample.java#tag-name -->

No comment detected in first line for block with extension

Hi,

Can you help me on this I do not get the problem actually !

No comment detected in first line for block with extension

npx embedme Code.md

Embedding...
  Analysing Code.md...
   Code.md#L3-L6 No comment detected in first line for block with extension sh
  No changes to write for Code.md

The file Code.md
Code.md
or

jkhaskjdhkjhkjasd

```sh
//Dowload_Youtube-dl.sh

```

and the embed code :

echo hello world

Thanks

Include code from a public github repo?

Thank you for this tool.

Could I write a script that will load the source code from a public github repo? This would be great.

For example something like this:

```php
// https://raw.githubusercontent.com/astridx/boilerplate/bc1862aa67b46ec23830617137483903fa968d72/src/components/com_foos/tmpl/foo/default.php
```.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.