Code Monkey home page Code Monkey logo

md-graph's Introduction

md-graph

Graph operations on a collection of interlinked Markdown (or VimWiki, ZimWiki, Zettlr, Obsidian, or Neuron) notes.

md-graph seeks to enrich an existing library of interlinked notes but not change how notes are written for the sake of its use. This enables interoperation with other note systems as well as respecting note libraries that strive to remain evergreen by staying simple.

Usage

md-graph - A utility for graph operations on a collection of markdown files

Usage: md-graph [(-l|--library FILE|DIR)] [-d|--default-ext EXT] COMMAND

Available options:
  -h,--help                Show this help text
  -l,--library FILE|DIR    Files or directories to parse
  -d,--database FILE       Sqlite database file to use
  -x,--default-ext EXT     Default extension to use for files linked without
                           extension (default: "md")

Available commands:
  subgraph                 The subgraph of a node
  backlinks                The backlinks (reverse subgraph) of a node
  unreachable              Files that are not linked to
  orphans                  Files without any links
  nonexistant              Links that cannot be resolved
  static                   Links that can be resolved but are not notes
  populate                 Just populate the database
  • -l, --library Optional, defaults to the current working directory. The folders or files to include in the graph. Folders will be recursively searched.
  • -d, --database Optional, defaults to creating a temporary mdgraphXXXX file.
  • -d, --default-extension Default md. Extension to use for links that do not specify an extension (like WikiLinks and VimWiki markdown links)

Subgraph

A subgraph is the graph of a file built by following its links, its links' links, etc. If foo.md links to bar.md and baz.md, and bar.md links to qux.md, the subgraph of foo.md will be foo.md bar.md baz.md qux.md

Usage: md-graph subgraph NODES [--inc-nonex True|False] [--inc-static True|False]
                               [--tag-direction In|Out|Both] [--depth ARG]
  The subgraph of a node

Available options:
  NODES                    Nodes (files or #tags) to process
  --inc-nonex True|False   Include non-existent files in output (default: False)
  --inc-static True|False  Include static files in output (default: True)
  --tag-direction In|Out|Both
                           Change the direction of tags (default: Out)
  --depth ARG              How deep traversal should go (default: -1)
  -h,--help                Show this help text
  • NODES The target files or tags to act as the start of the subgraph.
  • --inc-static=True|False Default True. Includes static files in the output (files that were linked to and can be resolved)
  • --inc-nonex=True|False Default False. Includes non-existent files in the output (files that were linked to but can not be resolved - includes HTTP links)
  • --tag-direction=In|Out|Both Default Out. Controls how tags are added to the graph from the perspective of the tag.
    • In: The link is coming in to the tag from the file
    • Out: The link is from the tag out to the file
    • Both: The tag and the file both have links to each other.
  • --depth INT Default -1 (infinite depth). Configures how many links deep from the target NODES a subgraph should traverse.

Backlinks

Backlinks are the files that link to a given file. foo.md is a backlink of bar.md if foo.md links to bar.md. It can also be thought of as the local subgraph of a file with the arrows reversed.

Usage: md-graph backlinks NODES [--depth ARG]
  The backlinks (reverse subgraph) of a node

Available options:
  NODES                    Nodes (files or #tags) to process
  --depth ARG              How deep traversal should go (default: 1)
  -h,--help                Show this help text
  • NODES The target files or tags to act as the start of the subgraph.
  • --depth INT Default 1 (immediate links only). Configures how many links deep from the target NODES a subgraph should traverse.

Nodes: Files & Tags

Arguments that take a "NODE" (such as subgraph and backlinks) can be given either a file or a tag. Tags are given by including the hash just like their file-parsed counterparts. The following will attempt to find the backlinks of the tag "my-special-tag": md-graph backlinks #my-special-tag.

With option --tag-direction=Out, you can find the subgraph of a tag.

Everything else will be parsed as a file.

Installation

Follow the instructions here to install stack

git clone https://github.com/CharlesSchimmel/md-graph
cd md-graph
stack install

Supported Formats

Link Formats

Because this merely parses out links, any format that uses markdown or wikilink style links should work. Links that are escaped or encoded in a way that is different from their filesystem name will not be traversed correctly.

Tag Format

Tags that follow the form of a hash ('#') followed by alpha-nums, dash, and underscores will be parsed as nodes. For example, this tag is valid: #someSort-of_tag123.

Background

I have a directory of notes in markdown format that link to each other like a Wiki. Sometimes I want to publish my notes on a given topic or group of topics including all of the related notes that they link to. For example, I might want to publish only my kombucha notes from my collection of notes on cooking. It would be a pain to have to manually trace the links between them in order to extract just the pertinent ones. This will take a root markdown file and traverse its links, returning all of the files necessary to complete its subgraph.

md-graph's People

Contributors

charlesschimmel avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

md-graph's Issues

Support config file

Add support for a config file.

  • format will be dhall or yaml.
  • default location will be $HOME/.config/md-graph
  • new flag --config to specify location

Values:

  • library root
  • include static, include nonexistant,
  • valid extension
  • tag direction

As-is this will not enable much functionality, but will enable greater functionality for other new features, like #5

Parse tags from YAML metadata

Pandoc supports tags in the YAML metadata at the top of a file. I prefer to use the YAML metadata/frontmatter for titling, tags, and datestamping so it would be nice if those tags were parsed out as well.

Readme: Neuron links to Obsidian

The link for Neuron links to Obsidian in the Readme. I would create a pull request but I have never heard about Neuron as a wiki/markdown-tool, so have no idea what the correct link is.

Ignore identity links

Links from a note to itself should be ignored. I can't think of a reason why this information would be useful.

Add inotify daemon

Add a new run command that starts the app in daemon mode, watching the library for changes. Parse new/changed files, remove links for deleted files. Will enable greater functionality for #5

Tag Pages

It would be nice if...

Tags could be associated with a page/note/node. So the tag kombucha was
either implicitly or explicitly associated with the note kombucha.md or
#kombucha.md or something like that. This is basically backwards-linking:
a link source -> sink being created by the sink, instead of the source.

Link context

It would be nice if...

Link text could be used as context for linking and possibly gain new insights by analyzing the common contexts of many links.

My first thought was that tags could be placed in the link text, and then queried for later, like this: [glucanobacter is in the acetic acid bacteria family #taxonomy](./some research note). I think this approach could be useful and the most straightforward, but it transgresses the principle that md-graph should not change how notes are taken.

Enrinching existing link contexts could include fuzzy matching the link text, but that seems coarse and not as useful. The next level would be analyzing the shape of relation that many links form, like a search engine.

"Tag Trees" / "Implicit Tags" / "Tag Families"

It would be nice to give tags their own hierarchy/directed acyclic graph/graph.
Although it could probably be cyclic, it's the same as building the file graph,
just with tags. Follow the connections and build up a list of tags to grab.

For example, all files tagged kombucha are also implicitly tagged as
project, fermentation, and food. This could be set up as kombucha -> food or food -> kombucha, both work as long as it's consistent (though it
will probably be easier to represent as the former)

This would probably be done in the config file added in #7

Enable populating & querying from a database

Parsing is pretty quick, but given a large enough library it starts to lag. Why not populate a database and query from that?

Sqlite for the database, persistent for the connection layer.

New command options:

  • --db-file for database location
  • Just populate/update nothing else
  • Populate and run a command
  • Run a command without a database, just find existing files. (Existing behavior)
  • Run a command with the database (new default)

Ignore files when querying a subgraph

It would be nice if...

When I'm querying a subgraph, I could ignore certain files or entire subgraphs and not traverse them. I might be exporting a subgraph of my kombucha research to share, but one of the files links to my top-secret kombucha recipe and its related notes.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.