Code Monkey home page Code Monkey logo

toolkit's Introduction

JSON-stat Javascript Toolkit

This package (jsonstat-toolkit) contains version 1 of the JSON-stat Javascript Toolkit. Version 0 of this Toolkit can be found in the jsonstat package. If you need to support very old browsers (like IE6), stick to version 0. The two versions have identical APIs, except for the implementation of XHR (an optional feature, anyway) and version number exposition. Both changes affect only the JSONstat function: no method is affected. Version 0 is frozen.

The JSON-stat format is a simple lightweight JSON format for data dissemination. It is based in a cube model that arises from the evidence that the most common form of data dissemination is the tabular form. In this cube model, datasets are organized in dimensions. Dimensions are organized in categories.

The JSON-stat Javascript Toolkit (JJT) is part of the JSON-stat Toolkit. JJT's goal is to help dealing with JSON-stat responses in JavaScript.

To learn by example, you can read the interactive notebook Introduction to the jsonstat-toolkit.

Resources

Design principles

JSON-stat is based on a data cube information structure. The JSON-stat Javascript Toolkit exposes the data cube as a tree.

The JSON-stat tree

Datasets

Datasets are organized in dimensions and data.

  • Dataset
    • Dimension
      • Category
    • Data

Collections

Collections are sets of items. Items can be collections, datasets and dimensions (currently not supported by JJT).

  • Collection
    • Item

Generally, items in a collection contain just basic content and a pointer that allow a client to retrieve the full information about the item. But a collection can also contain the full information (embedded items).

Bundles (JSON-stat<2.0)

Bundles were packages of unordered arbitrary datasets.

  • Bundle
    • Dataset
      • Dimension
        • Category
      • Data

Even though JSON-stat currently encourages the use of collections of embedded datasets instead of bundles, JJT supports both approaches.

To retrieve information about the first category of the first dimension of the first embedded dataset in a JSON-stat collection (or bundle) j, the JSON-stat Javascript Toolkit allows you to traverse the JSON-stat tree like this:

JSONstat( j ).Dataset( 0 ).Dimension( 0 ).Category( 0 )

The class of the response can be checked using the class property:

if(JSONstat( j ).class==="dataset"){
   var cat0=JSONstat( j ).Dimension( 0 ).Category( 0 );   
}

General properties

  • label: label of the selected element (string)
  • length: number of children of the selected element (number).
  • id: IDs of the children of the selected element (array).

Reading and traversing methods

These methods (except JSONstat, which is not actually a method) accept a selection argument. If it is not provided, an array is returned with the information for every child of the selected element.

JSONstat

It reads a JSON-stat response and returns an object.

JSONstat( { ... } ).length
//number of datasets in the object

JSONstat( "https://json-stat.org/samples/oecd-canada-col.json" ).then(
  function(J){
     console.log( J.length );
  }
)
//number of items in oecd-canada-col.json.

Dataset

It selects an embedded dataset in the JSON-stat collection (or bundle).

JSONstat( j ).Dataset( 0 ).id //IDs of the dimensions in the first dataset

Dimension

It selects a particular dimension in a dataset.

JSONstat( j ).Dataset( 0 ).Dimension( "time" ).label
//Label of the "time" dimension in the first dataset

JSONstat( j ).Dataset( 0 ).Dimension( "country" ).role
//Role of the "country" dimension in the first dataset

Category

It selects a particular category in a dimension in a dataset.

JSONstat( j ).Dataset( 0 ).Dimension( "time" ).Category( 0 ).label
//Label of the first category of the "time" dimension in the first dataset

Data

When an argument is passed, selects a single cell of the data cube in the JSON-stat response. If no argument is passed, returns all the cells.

The resulting object contains the property "value" (value of a cell) and "status" (its status).

JSONstat( j ).Dataset( 0 ).Data( 0 ).value
//Value of the first cell (usually a number, but values can be of any type).

JSONstat( j ).Dataset( 0 ).Data( [ 0, 0, 0 ] ).value
//Value of the first cell in a dataset with 3 dimensions (usually a number).

JSONstat( j ).Dataset( 0 ).Data( { "metric" : "UNR", "geo" : "GR", "time" : "2014" } ).value
//Unemployment rate in Greece in 2014 (usually a number).

JSONstat( j ).Dataset( 0 ).Data( { "metric" : "UNR", "geo" : "GR", "time" : "2014" } ).status
//Status of unemployment rate in Greece in 2014.

When the argument is neither an integer nor an array, single category dimensions (“constant dimensions”) can be skipped. If one and only one non-constant dimension is not specified, the result will be an array with as many elements as categories in the unspecified dimension.

Transformation methods

Transformation methods get information in the JSON-stat tree and export it to a different JSON structure for convenience.

toTable

This is a dataset method. It converts the information of a particular dataset into a JSON table. The conversion can be setup using an optional argument.

JSONstat( j ).Dataset( 0 ).toTable()
//Returns an array of arrays that exposes a tabular structure (rows and columns).
//Useful in many situations. For example, it can be a Google Visualization API input.

JSONstat( j ).Dataset( 0 ).toTable( { field : "id" } )
//Uses ids instead of labels as column names.

JSONstat( j ).Dataset( 0 ).toTable( { vlabel : "Valor", type : "object" } )
//Returns an object of arrays (of objects) that exposes a tabular structure (rows and columns)
//in the Google DataTable format (it's the native input format of Google
//Visualization API input). The "vlabel" property is instructing the method to use
//"Valor" as the label of the values column (instead of "Value").

JSONstat( j ).Dataset( 0 ).toTable( { status : true, slabel : "Metadata" } )
//The table will include a status column with label "Metadata".

JSONstat( j ).Dataset( 0 ).toTable( { type : "arrobj" } )
//Returns an array of objects where each dimension id is a property, plus a "value" property.

JSONstat( j ).Dataset( 0 ).toTable( { type: "arrobj", content: "id" } )
//same but category ids ("AU") are used instead of labels ("Australia") even for content.

JSONstat( j ).Dataset( 1 ).toTable(
   { type : "arrobj", content : "id" },
   function( d, i ){
      if ( d.sex === "F" && d.concept === "POP" ){
         return { age : d.age, population : d.value*1000 };
      }
   }
)
//Get only the female population by age of Canada
//and convert values from thousands to persons.

toolkit's People

Contributors

badosa avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

toolkit's Issues

Performance ot fromTable and toTable is very bad

Hello,

The performance of this library leaves a lot to be desired.
I have been optimizing some parts of if (fromTable/toTable) and I'd like to contribute that work.

I've written a toTable function that handles big payloads. The current one does not complete those and takes forever processing fairly-big payloads. (the test that ran here for 5881ms was reduced to 1ms by proper handling)

What is the best way to contribute here?

Regards,

  • Stefán

Zero (0.0) values not showing

Hi @badosa, thank you for your work.

I'm having an issue where zero (0.0) values in a dataset disappear.

Can Dice() or toTable() cause this behaviour?

TY

TypeError: Cannot read property 'method' of null

const JSONstat = require("jsonstat-toolkit");

var url="https://data.ssb.no/api/v0/dataset/1054.json?lang=en";
JSONstat(url).then(main);
function main(j){
var ds=j.Dataset(0);
console.log(ds)
}

This returns error:
let method = init.method || input.method || 'GET';
^

let method = init.method || input.method || 'GET';
^

TypeError: Cannot read property 'method' of null
at new Request (/app/node_modules/node-fetch/lib/index.js:1217:21)
at /app/node_modules/node-fetch/lib/index.js:1439:19
at new Promise ()
at fetch (/app/node_modules/node-fetch/lib/index.js:1437:9)
at module.exports (/app/node_modules/jsonstat-toolkit/main.cjs:2:15410)
at Object. (/app/index.js:2:1)
at Module._compile (node:internal/modules/cjs/loader:1092:14)
at Object.Module._extensions..js (node:internal/modules/cjs/loader:1121:10)
at Module.load (node:internal/modules/cjs/loader:972:32)
at Function.Module._load (node:internal/modules/cjs/loader:813:14)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.