sql-json-bags's Introduction

datasets-br

Describing the datasets-br directives and using this project as point of generic discussions.

Dataset-BR directives

To post qualified datasets in the Datahub.io;
To unify, by curatory process, a set of Wikidata fragments if items, or commom instances of an item;
To unify terminology to express CSV colunm names, table and column semantics (SchemaOrg conventions when possible)
Digital preservation (CSV files and data dumps from original soruces) of the curated datasets;
Monitoring/auditing Wikidata and OpenStreetMap changes, in the context of the curated datasets.

Use as an ecosystem of datasets

Example of use with 2 BR's datasets, state-codes and city-codes.

Operating with pure SQL or SQL-unifier will be easy to merge with other datasets... With PopstgreSQL you can offer datasets in an standard API with PostgreREST (or its descendents pREST and PostGraphile), or plug-and-play with SchemaOrg standards, FrictionlessData standards (and tools), etc.

Documentation

... under construction

Conventions for data provenance and prepare.

Contents and data of this project are dedicated to

sql-json-bags's People

Contributors

Stargazers

Watchers

sql-json-bags's Issues

Include filters for normalization

Some JSONb can be easylly transformated into valid bags:

a "bag" with nulls or zeros: must delete them.
an array of bags: must expand into sql-array
... string filters for canonic element expression, but is application-specific.

So simple functions bag.std(JSONb) and bag.std_array(JSONb) that converts impercfect bags.

Use SQL-Domain

See https://www.postgresql.org/docs/current/static/sql-createdomain.html

jbag datatype

CREATE DOMAIN jbag AS JSONB
  CHECK(  VALUE IS NULL OR  jsonb_typeof(VALUE) IN ('object','null')  )
;  -- empty bag is JSON-null, and SQL NULL is "undefined"

Better a simple domain control and a validation function... The definition "... elementes are strings and the mutiplicity an positive non-zero integer" need a loop over all elements.
Complex checking is not good for performance, so validate only when request.

Recommend Projects