Code Monkey home page Code Monkey logo

contentdb's Introduction

ContentDB

contentdb is a document store database library that is similar to CouchDB and MongoDB.

This project follows the general "NoSQL" trend away from table-oriented databases, usually toward document-oriented or otherwise graph-oriented. It will also support outline-oriented databses, like OPML and YAML, which facilitate great data management interfaces in simple text editors and even more powerful UIs, like WorkFlowy and OmniOutliner or open source solutions like Drupal Paragraphs Drag & Drop Mode and Drupal Draggable Views with Sections (draggableviews).

What makes ContentDB significantly different from other databases is that it uses a file to store each "record", called a document, in a specified format. This is similar to RDF-style XML databases.

With this approach, you can easily manage your data storage, retreival, and other metadata storage and retrieval as well. You can easily do version control for history tracking and easy deployment. Relationships are easy to represent using reference fields. See and Pelican and Grav for other similar systems, more so Pelican because of its pelican reference plugin.

To massively improve performance and scalability, Tantivy integration can provide immediate indexing and querying. This is the same as materialized views and mongo collections.

Format

ContentDB combines one structured and one unstructured data format. Headers use SCL, and bodies use Markdown or Lua.

Header field values provide a mechanism to represent attributes of world entities in virtual document texts. Relationship fields work by using the target's model for its field. You can see this implemented in Python in the pelican plugin above and some more background info about using antisymmetric relationships.

Default ID is UUIDv4, but this will soon shift to NanoID. See #17

Short names are a missing feature. #12

Model documents started by following some json document schema or something, forgot which schema

Stores are folders with documents from a single source.

contentdb's People

Contributors

arnaz87 avatar naturallymitchell avatar sh-zam avatar

Stargazers

avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar

contentdb's Issues

add outline support

After #20
Should support folder based hierarchies
Need outline-oriented model format, so need to clarify current document-oriented model format first
This will also work nicely for logs, not just opml-like texts

web log model

kind: request / response
from/to: (can "server" and "client" be united?) - incoming: 1, or outgoing 1

function renames etc

reviews appreciated

  • base: models-setup, stores-setup
  • get_validator -> document-check-model
  • validate_document -> document-check-model
  • add_document_to_index -> index-document
  • documents -> store-documents
  • get_document_path -> document-get-path
  • query -> index-documents
  • read_document -> document-read
  • read_log -> logdocument-read
  • setup_index -> index-setup
  • setup_schema -> schema-setup
  • split-header -> document-parse
  • walk-documents -> documents-walk
  • write-file -> document-write

schema polymorphism / model inheritance / object oriented data types

1

https://github.com/jazzdotdev/lighttouch/blob/364b3e3cb1be8caac90ab212ce3db4593d7d4b78/loaders/models.lua#L22

Lighttouch loaders preprocess everything, so this would make sense to do that way too..

While loading the models, check for this type of syntax and setup the data interfaces to Classes instead of database objects

cleanup

  • remove luvent from write document
  • remove log traces for running

Reading files from missing stores

To read files, lighttouch walks through the files in the store's directories, but when given a missing store it breaks. Make it detect a missing store and just walk through an empty list of files.

fix upstream markdown ambiguities, inconsistencies, and ordered listing antipatterns

needs: 1) feature complete structured writing, automatic list item numbering 2) unambiguous syntax 3) round-tripping between markup and markdown

numbering # for list ordering
portioning % for text sectioning, paragraphs
bulletting * for unordered listing

~~
raw notes:
~~

bug: ordered list syntax uses ambiguous, cumbersome, hardcoded symbols

https://spec.commonmark.org/0.28/#ordered-list-marker
https://en.wikipedia.org/wiki/Hard_coding (antipattern)

using unique, hardcoded characters for each list item, and then an ambiguous ending character is breaking my content. I'd like to change this syntax to a single character syntax.

the number sign, #, would make the most sense to me, since we're numbering things. this would make for an easy fix with nesting ordered lists, because then we could use multiple markers for list depth.

related: # [ ]

~~

bug: heading syntax uses a number symbol, instead of a portion symbol

https://spec.commonmark.org/0.28/#atx-heading
https://en.wikipedia.org/wiki/Number_sign

this bug also breaks numbering things. see []#

percentage is a portion calculated in terms of the rest of the portions, but it's the closest general character I can find to indicate a portion

the only other one I can think of is + a plus sign, but that's part of the problem I'm trying to solve. I would like to better organize content across documents for a combinatorial value whose mechanisms work differently than simple addition.

~~

~~

~~

use data geometry for transparent viewstore lay-up

https://en.wikipedia.org/wiki/Data_compression

shapes and symmetries of groups

http://www.coloring-book.co/
https://news.ycombinator.com/item?id=15326480
https://hn.algolia.com/?query=group%20theory&sort=byPopularity&prefix&page=0&dateRange=all&type=story

file systems

geometries

preprocessed field type

so Lighttouch interfaces can:

when document viewed
  if document model has preprocessed field
    trigger model's / document's CRUD events (can be processed async, but before return)

model model

to read model meta-documents as documents

shortname

slug
might need to error if multiple documents with same shortname exist in same data store, and warn if same across stores

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.