unified is an interface for processing text using syntax trees. It’s what powers remark, retext, and rehype, but it also allows for processing between multiple syntaxes.
npm:
npm install unified
var unified = require('unified');
var markdown = require('remark-parse');
var remark2rehype = require('remark-rehype');
var document = require('rehype-document');
var format = require('rehype-format');
var html = require('rehype-stringify');
var reporter = require('vfile-reporter');
unified()
.use(markdown)
.use(remark2rehype)
.use(document)
.use(format)
.use(html)
.process('# Hello world!', function (err, file) {
console.error(reporter(err || file));
console.log(String(file));
});
Yields:
no issues found
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
</head>
<body>
<h1>Hello world!</h1>
</body>
</html>
unified is an interface for processing text using syntax trees. Syntax trees are a representation understandable to programs. Those programs, called plug-ins, take these trees and modify them, amongst other things. To get to the syntax tree from input text, there’s a parser, and, to get from that back to text, there’s a compiler. This is the process of a processor.
┌──────────────┐
┌─ │ Transformers │ ─┐
▲ └──────────────┘ ▼
└────────┐ ┌────────┘
│ │
┌────────┐ │ │ ┌──────────┐
Input ──▶ │ Parser │ ──▶ Tree ──▶ │ Compiler │ ──▶ Output
└────────┘ └──────────┘
Every processor implements another processor. To create a new processor, invoke another processor. This creates a processor that is configured to function the same as its ancestor. But, when the descendant processor is configured in the future, that configuration does not change the ancestral processor.
Often, when processors are exposed from a library (for example, unified itself), they should not be configured directly, as that would change their behaviour for all users. Those processors are frozen, and new processors should be made from them before they are used, by invoking them.
The syntax trees used in unified are Unist nodes,
which are plain JavaScript objects with a type
property. The
semantics of those type
s are defined by other projects.
There are several utilities for working with these nodes.
The following projects process different syntax trees. They parse text to their respective syntax tree, and they compile their syntax trees back to text. These processors can be used as-is, or their parsers and compilers can be mixed and matched with other plug-ins to process between different syntaxes.
When processing documents, metadata is often gathered about that document. VFile is a virtual file format which stores data, and handles metadata and messages for unified and its plug-ins.
There are several utilities for working with these files.
To configure a processor, invoke its use
method, supply it a
plug-in, and optionally settings.
unified can integrate with the file-system through unified-engine. On top of that, CLI apps can be created with unified-args, Gulp plug-ins with unified-engine-gulp, and Atom Linters with unified-engine-atom.
The API gives access to processing metadata (such as lint messages), and supports multiple passed through files:
var unified = require('unified');
var markdown = require('remark-parse');
var lint = require('remark-lint');
var remark2retext = require('remark-retext');
var english = require('retext-english');
var equality = require('retext-equality');
var remark2rehype = require('remark-rehype');
var html = require('rehype-stringify');
var reporter = require('vfile-reporter');
unified()
.use(markdown)
.use(lint)
.use(remark2retext, unified().use(english).use(equality))
.use(remark2rehype)
.use(html)
.process('## Hey guys', function (err, file) {
console.err(reporter(err || file));
console.log(file.toString());
});
Which yields:
1:1-1:12 warning First heading level should be `1` first-heading-level
1:8-1:12 warning `guys` may be insensitive, use `people`, `persons`, `folks` instead gals-men
⚠ 3 warnings
<h2>Hey guys</h2>
The processors can be combined in two modes.
Bridge mode transforms the syntax tree from one flavour (the origin) to another (the destination). Then, transformations are applied on that tree. Finally, the origin processor continues transforming the original syntax tree.
Mutate mode also transforms the syntax tree from one flavour to another. But then the origin processor continues transforming the destination syntax tree.
In the previous example (“Programming interface”), remark-retext
is
used in bridge mode: the origin syntax tree is kept after retext is
finished; whereas remark-rehype
is used in mutate mode: it sets a
new syntax tree and discards the original.
Object describing how to process text.
Function
— A new unfrozen processor which is
configured to function the same as its ancestor. But, when the
descendant processor is configured in the future, that configuration
does not change the ancestral processor.
The following example shows how a new processor can be created (from the remark processor) and linked to stdin(4) and stdout(4).
var remark = require('remark');
var concat = require('concat-stream');
process.stdin.pipe(concat(function (buf) {
process.stdout.write(remark().processSync(buf))
}));
Configure the processor to use a plug-in, and configure that plug-in with optional options.
processor.use(plugin[, options])
;processor.use(preset)
;processor.use(list)
;
plugin
(Plugin
);options
(*
, optional) — Configuration forplugin
.preset
(Object
) — Object with an optionalplugins
(set tolist
), and/or an optionalsettings
object;list
(Array
) — plugins, presets, and arguments (a plugin and options in an array), in an array.
processor
— The processor on which use
is invoked.
use
cannot be called on frozen processors. Invoke the processor
first to create a new unfrozen processor.
There are many ways to pass plugins to .use()
. The below example
gives an overview.
var unified = require('unified');
unified()
// Plugin with options:
.use(plugin, {})
// Plugins:
.use([plugin, pluginB])
// Two plugins, the second with options:
.use([plugin, [pluginB, {}]])
// Preset with plugins and settings:
.use({plugins: [plugin, [pluginB, {}]], settings: {position: false}})
// Settings only:
.use({settings: {position: false}});
function plugin() {}
function pluginB() {}
Parse text to a syntax tree.
file
(VFile) — Or anything which can be given tovfile()
.
Node — Syntax tree representation of input.
parse
freezes the processor, if not already frozen.
Function handling the parsing of text to a syntax tree. Used in the
parse phase in the process and invoked with a string
and VFile representation of the document to parse.
If Parser
is a normal parser, it should return a Node
: the syntax
tree representation of the given file.
Parser
can also be a constructor function, in which case it’s invoked with
new
. In that case, instances should have a parse
method, which is invoked
(without arguments), and should return a Node
.
Compile a syntax tree to text.
string
— String representation of the syntax tree file.
stringify
freezes the processor, if not already frozen.
Function handling the compilation of syntax tree to a text. Used in the
stringify phase in the process and invoked with a
Node
and VFile representation of the document to
stringify.
If Compiler
is a normal stringifier, it should return a string
: the text
representation of the given syntax tree.
Compiler
can also be a constructor function, in which case it’s invoked with
new
. In that case, instances should have a compile
method, which is invoked
(without arguments), and should return a string
.
Transform a syntax tree by applying plug-ins to it.
node
(Node);file
(VFile, optional); — Or anything which can be given tovfile()
.done
(Function
, optional).
Promise, if done
is not given. Rejected with an error,
or resolved with the resulting syntax tree.
run
freezes the processor, if not already frozen.
Invoked when transformation is complete. Either invoked with an error, or a syntax tree and a file.
Transform a syntax tree by applying plug-ins to it.
If asynchronous plug-ins are configured, an error is thrown.
Node — The given syntax tree.
runSync
freezes the processor, if not already frozen.
Process the given representation of a file as configured on the
processor. The process invokes parse
, run
, and stringify
internally.
Promise, if done
is not given. Rejected with an error,
or resolved with the resulting file.
process
freezes the processor, if not already frozen.
Invoked when the process is complete. Invoked with a fatal error, if any, and the VFile.
err
(Error
, optional) — Fatal error;file
(VFile).
var unified = require('unified');
var markdown = require('remark-parse');
var remark2rehype = require('remark-rehype');
var document = require('rehype-document');
var format = require('rehype-format');
var html = require('rehype-stringify');
var reporter = require('vfile-reporter');
unified()
.use(markdown)
.use(remark2rehype)
.use(document)
.use(format)
.use(html)
.process('# Hello world!')
.then(function (file) {
console.log(String(file));
}, function (err) {
console.error(String(err));
})
Yields:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
</head>
<body>
<h1>Hello world!</h1>
</body>
</html>
Process the given representation of a file as configured on the
processor. The process invokes parse
, run
, and stringify
internally.
If asynchronous plug-ins are configured, an error is thrown.
file
(VFile);value
(string
) — String representation of a file;
VFile — Virtual file with modified contents
.
processSync
freezes the processor, if not already frozen.
var unified = require('unified');
var markdown = require('remark-parse');
var remark2rehype = require('remark-rehype');
var document = require('rehype-document');
var format = require('rehype-format');
var html = require('rehype-stringify');
var reporter = require('vfile-reporter');
var processor = unified()
.use(markdown)
.use(remark2rehype)
.use(document)
.use(format)
.use(html);
console.log(processor.processSync('# Hello world!').toString());
Yields:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
</head>
<body>
<h1>Hello world!</h1>
</body>
</html>
Get or set information in an in-memory key-value store accessible to all phases of the process. An example is a list of HTML elements which are self-closing (i.e., do not need a closing tag), which is needed when parsing, transforming, and compiling HTML.
key
(string
) — Identifier;value
(*
, optional) — Value to set. Omit if gettingkey
.
processor
— If setting, the processor on whichdata
is invoked;*
— If getting, the value atkey
.
Setting information with data
cannot occur on frozen processors.
Invoke the processor first to create a new unfrozen processor.
The following example show how to get and set information:
var unified = require('unified');
console.log(unified().data('alpha', 'bravo').data('alpha'))
Yields:
bravo
Freeze a processor. Frozen processors are meant to be extended, and not to be configured or processed directly.
Once a processor is frozen, it cannot be unfrozen. But, a new processor functioning just like it can be created by invoking the processor.
It’s possible to freeze processors explicitly, by calling .freeze()
, but
.parse()
, .run()
, .stringify()
, and
.process()
call .freeze()
to freeze a processor too.
Processor
— The processor on which freeze
is invoked.
The following example, index.js
, shows how rehype
prevents extensions to itself:
var unified = require('unified');
var parse = require('rehype-parse');
var stringify = require('rehype-stringify');
module.exports = unified().use(parse).use(stringify).freeze();
The below example, a.js
, shows how that processor can be used and
configured.
var rehype = require('rehype');
var format = require('rehype-format');
// ...
rehype()
.use(format)
// ...
The below example, b.js
, shows a similar looking example which
operates on the frozen rehype interface. If this
behaviour was allowed it would result in unexpected behaviour, so
an error is thrown. This is invalid:
var rehype = require('rehype');
var format = require('rehype-format');
// ...
rehype
.use(format)
// ...
Yields:
~/node_modules/unified/index.js:436
throw new Error(
^
Error: Cannot invoke `use` on a frozen processor.
Create a new processor first, by invoking it: use `processor()` instead of `processor`.
at assertUnfrozen (~/node_modules/unified/index.js:436:11)
at Function.use (~/node_modules/unified/index.js:170:5)
at Object.<anonymous> (~/b.js:6:4)
A unified plugin changes the way the applied-on processor works, in the following ways:
- It modifies the processor: such as changing the parser, the compiler, or linking the processor to other processors;
- It transforms the syntax tree representation of a file;
- It modifies metadata of a file.
Plug-in’s are a concept which materialise as attachers.
move.js
:
module.exports = move;
function move(options) {
var expected = (options || {}).extname;
if (!expected) {
throw new Error('Missing `extname` in options');
}
return transformer;
function transformer(tree, file) {
if (file.extname && file.extname !== expected) {
file.extname = expected;
}
}
}
index.js
:
var unified = require('unified');
var parse = require('remark-parse');
var remark2rehype = require('remark-rehype');
var stringify = require('rehype-stringify');
var vfile = require('to-vfile');
var reporter = require('vfile-reporter');
var move = require('./move');
rehype()
.use(parse)
.use(remark2rehype)
.use(move, {extname: '.html'})
.use(stringify)
.process(vfile.readSync('index.md'), function (err, file) {
console.error(reporter(err || file));
if (file) {
vfile.writeSync(file); // written to `index.html`
}
})
An attacher is the thing passed to use
. It configures the
processor and in turn can receive options.
Attachers can configure processors, such as by interacting with parsers and compilers, linking them to other processors, or by specifying how the syntax tree is handled.
The context object is set to the invoked on processor
.
options
(*
, optional) — Configuration.
transformer
— Optional.
Attachers are invoked when the processor is frozen: either when
.freeze()
is called explicitly, or when .parse()
, .run()
,
.stringify()
, or .process()
is called for the first
time.
Transformers modify the syntax tree or metadata of a file.
A transformer is a function which is invoked each time a file is
passed through the transform phase. If an error occurs (either
because it’s thrown, returned, rejected, or passed to next
),
the process stops.
The transformation process in unified is handled by trough
,
see it’s documentation for the exact semantics of transformers.
Error
— Can be returned to stop the process;- Node — Can be returned and results in further
transformations and
stringify
s to be performed on the new tree; Promise
— If a promise is returned, the function is asynchronous, and must be resolved (optionally with a Node) or rejected (optionally with anError
).
If the signature of a transformer includes next
(third argument),
the function may finish asynchronous, and must invoke next()
.
err
(Error
, optional) — Stop the process;node
(Node, optional) — New syntax tree;file
(VFile, optional) — New virtual file.
A unified preset provides a potentially sharable way to configure processors. They can contain multiple plugins and optionally settings as well.
preset.js
:
exports.settings = {bullet: '*', fences: true}
exports.plugins = [
[require('remark-preset-lint-recommended')]
[require('remark-comment-config')],
[require('remark-validate-links')],
[require('remark-lint'), {
blockquoteIndentation: 2,
checkboxCharacterStyle: {checked: 'x', unchecked: ' '},
// ...
}],
[require('remark-toc'), {maxDepth: 3, tight: true}]
[require('remark-github')]
]
index.js
:
var remark = require('remark');
var vfile = require('to-vfile');
var reporter = require('vfile-reporter');
var preset = require('./preset');
remark().use(preset).process(vfile.readSync('index.md'), function (err, file) {
console.error(reporter(err || file));
if (file) {
vfile.writeSync(file);
}
})