Code Monkey home page Code Monkey logo

processmd's Introduction

processmd

Build Status

processmd uses globby and markdown-it to process directories of markdown and yaml files to a JSON files with html. It has many options to format the output as you like, you can convert a nested directory of yaml to json and optionally add a "summary" file with info about all the files. Additionally, with the convertMode: "source" option you can convert back from json to the input markdown and yaml files. This is mostly useful for blogs or static content for websites or other places where json is easier to use but the readability of yaml is useful.

Comparison to similar tools:

Package Processes folder to single file Processes folder to multiple files cli API Uses gulp includes metadata about file Markdown preview
markdown-to-json or markdown-to-json-with-content × × ×
gulp-markdown-to-json × ×
markdown-json × × ×
processmd ×

I'll note that processmd can also convert back from json to yaml and it has a suite of tests. If you know a tool that you think should be on this list please open a PR.

Getting Started

Install with npm

npm install --save-dev processmd

Usage

You can then use the cli

processmd "content/**/*.{yml,md}" --outputDir output

And watch files to automatically recompile them.

processmd "content/**/*.{yml,md}" --outputDir output --watch

With an input markdown file such as this:

---
test: frontmatter
draft: true
num: 1
---

# processmd

Process a directory of markdown *and* yaml files to JSON files

This would be the resulting JSON:

{
  "test":"frontmatter",
  "draft":true,
  "num":1,
  "bodyContent":"# processmd\r\n\r\nProcess a directory of markdown *and* yaml files to JSON files",
  "bodyHtml":"<h1 id=\"processmd\">processmd</h1>\n<p>Process a directory of markdown <em>and</em> yaml files to JSON files</p>\n",
  "title":"processmd",
  "dir":"test/data/output",
  "base":"frontmatter.json",
  "ext":".json",
  "sourceBase":"frontmatter.md",
  "sourceExt":".md"
}

And given the following input directory:

.
├── L1
│   ├── L2
│   │   └── test2.yml
│   └── test.yml
├── README.md
└── frontmatter.md

It would produce this directory output:

.
├── L1
│   ├── L2
│   │   └── test2.json
│   └── test.json
├── README.json
└── frontmatter.json

Advanced Usage

processmd will also output some summary data as a json object to stdout when used with the --stdout option. Then you can direct that to a file.

processmd \"content/**/*.{yml,md}\" --stdout --outputDir output > summary.json

summary.json will contain:

{
  "fileMap":{
    "test/data/output/frontmatter.json":"{...}",
    "test/data/output/L1/test.json":"{...}",
    ...
  }
  "sourceFileArray":[
    "test/data/input/frontmatter.md",
    "test/data/input/L1/test.yml",
    ...
  }

Options

module.exports = {
  // The directory output will be processed to.
  outputDir: './dist',
  // Output location for the summary file (relative path to json file that has content from all files).
  summaryOutput: null,
  // Watch mode, recompile on file changes.
  watch: false,
  // Debounce timeout for watching files.
  watchDebounce: 1000,
  // Prefix for output filenames, default is no prefix, just the original filename.
  filenamePrefix: '',
  // For markdown files, rendering mathematical equations.
  renderLatex: true,
  // For markdown files, highlight code block.
  highlightCode: true,
  // For markdown files, how many characters should be included in an add `preview` property. 0 for no preview.
  preview: 0,
  // Option to override the preview delimeter to handle internationalization and add greater flexibility (default: ' ').
  previewDelimiter: ' ',
  // Include body props in summary file.
  includeBodyProps: false,
  // Include the markdown document title as `title` on the resulting json objects.
  includeTitle: true,
  // Include the directory as `dir` on the resulting json objects.
  includeDir: true,
  // Include the filename (.json) as `base` on the resulting json objects.
  includeBase: true,
  // Include the extension (.json) as `ext` on the resulting json objects.
  includeExt: true,
  // Include the source filename (.md / .yml) as `sourceBase` on the resulting json objects.
  includeSourceBase: true,
  // Include the source extension (.md / .yml) as `sourceExt` on the resulting json objects.
  includeSourceExt: true,
  // Convert mode. Possible options for this are 'json' or 'source'.
  convertMode: 'json',
  // Whether to output to stdout or not.
  stdout: false,
  // Custom markdown renderer function, null to use the default: `markdown-it`.
  markdownRenderer: null,
  // Options to pass to the default markdown processor, markdown-it.
  // See here for options: https://github.com/markdown-it/markdown-it#init-with-presets-and-options
  markdownOptions: {},
  // Include generated ids on headings.
  headingIds: false
}

To turn off options you can prefix them with '--no-'

For example, if you want to disable code highlighting you would run:

$ processmd ReadMe.md --no-highlightCode

Development

To update tests, add new test files to test/data/input, and run npm run start && npm run back

Enjoy!

processmd's People

Contributors

anhngd avatar dependabot[bot] avatar ianmitchell avatar nick-penaranda-ck avatar tscanlin avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

processmd's Issues

Link Open to new tab

hello,
i have issue when content have a link, and i click the link but link not opened to new tab, only reload current tab
have solutions?

[Feature Request] Allow access to other markdown-it options

I'd like to tweak some of the markdown process options (namely setting html:true to render actual HTML instead of escaping it). It seems like the module is just using markdown-it's default set of options. Would a passthrough be something in scope of what you're trying to do here?

single file output

Hello, I'm enjoying your tool very much, and find its flexibility a real pleasure. I'm a bit confused about how to generate a single json file from a directory. So, for instance, if I have

|--Articles
    |--article1.md
    |--article2.md

I'd like to end up with just one json file that assembles all the individual objects into an array. Is that something that can be done directly by the tool or would I have to walk through the files listed in summary.json and manually create the array?

Add sorting capability to stdout JSON file

Hi,

Fantastic library! I'm using your library to generate a single JSON file out of multiple markdown files. I'm using the following command to do so.

.\node_modules\.bin\processmd "inputFolder/**/*.{md}" --stdout --outputDir outputFolder > output.json

Everything works fine, however, I noticed that the entries in the fileMap object of the output JSON are not sorted in the same order as the sourceFileArray. I'm wanting the contents in fileMap to be sorted somehow (either filename or date field?). I tried prefixing the markdown filenames with dates (201701, 201702...etc) and I also tried prefixing with alphabets (a-filename, b-filename...etc), however, neither sorts the results in fileMap in the right order.

Would be great to either get some direction on how should I go about sorting or if none available at the moment, then I'd really like to see a sort option become available in the near future.

Thanks!

Key of each entries in fileMap object

Thanks for making convenient tool!

In summary.json file, Is it better to use .md file name as key of each entries in fileMap object, rather than .json file name. That way, each entry in the fileMap can be retrieved by items in sourceFileArray directly while looping through the array.

{
  "fileMap": {
    "contents/news/json/2018-06-05-cmsからの投稿1.json": {
      "title": "CMSからの投稿1",
      "hilight_days": "4",
      "dir": "contents/news/json",
      "base": "2018-06-05-cmsからの投稿1.json",
      "ext": ".json",
      "sourceBase": "2018-06-05-cmsからの投稿1.md",
      "sourceExt": ".md"
    },
    "contents/news/json/2018-06-05-test2.json": {
      "title": "お知らせ2",
      "hilight_days": "5",
      "dir": "contents/news/json",
      "base": "2018-06-05-test2.json",
      "ext": ".json",
      "sourceBase": "2018-06-05-test2.md",
      "sourceExt": ".md"
    },
    "contents/news/json/2018-04-21-test.json": {
      "title": "お知らせ1",
      "hilight_days": "7",
      "dir": "contents/news/json",
      "base": "2018-04-21-test.json",
      "ext": ".json",
      "sourceBase": "2018-04-21-test.md",
      "sourceExt": ".md"
    }
  },
  "sourceFileArray": [
    "contents/news/2018-04-21-test.md",
    "contents/news/2018-06-05-cmsからの投稿1.md",
    "contents/news/2018-06-05-test2.md"
  ]
}

Thank you.

Top Level Directory Mirrored?

Given this structure:

content/
  posts/
    - a.md
    - b.md

Running

$ processmd content/**/*.md --outputDir content --preview 110 --summaryOutput content/summary.json

Doesn't return what I'd expect. The command will create the following files:

content/
  content/
    posts/
      - a.json
      - b.json
  posts/          (existing)
    - a.md
    - b.md
  - summary.json

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.