Code Monkey home page Code Monkey logo

downsample's Introduction

downsample

Downsampling methods for time series visualisation.

Build Status NPM Version Dev Dependency Status Known Vulnerabilities License

Installation | Usage | API | Demo | Acknowledgement

downsample is useful when, not extremely surprisingly, you need to downsample a numeric time series before visualizing it without losing the visual characteristics of the data.

Installation

downsample is an NPM module. You can easily download it by typing something like the following in your project:

# for all the npm people out there
npm install downsample

# or if you are a fan of yarn
yarn add downsample

Usage

The package exports several methods for data downsampling:

You can read more about the details of these in the API section below.

API

ASAP ๐Ÿ’ฅ new in 1.2.0 ๐Ÿ’ฅ

Automatic Smoothing for Attention Prioritization (read more here) is a smoothing rather than downsampling method - it will remove the short-term noise and reveal the large-scale deviations.

ASAP accepts an array of data points (see DataPoint) or a TypedArray (see TypedArray support) and a target resolution (number of output data points) as arguments. It will always return the points in XYDataPoint format. See advanced API if you need to work with a custom data type.

function ASAP(data: DataPoint[], targetResolution: number): XYDataPoint[]
import { ASAP } from 'downsample';

// Or if your codebase does not supprot tree-shaking
import { ASAP } from 'downsample/methods/ASAP';

const chartWidth = 1000;
const smooth = ASAP([
  [0, 1000],
  [1, 1243],
  // ...
], chartWidth);

SMA ๐Ÿ’ฅ new in 1.2.0 ๐Ÿ’ฅ

Simple moving average with variable slide (read more here).

SMA accepts an array of data points (see DataPoint) or a TypedArray (see TypedArray support), size of a window over which to calculate average and a slide - an amount by which the window is shifted. It will always return the points in XYDataPoint format. See advanced API if you need to work with a custom data type.

function SMA(data: DataPoint[], windowSize: number, slide?: number = 1): XYDataPoint[]
import { SMA } from 'downsample';

// Or if your codebase does not supprot tree-shaking
import { SMA } from 'downsample/methods/SMA';

const chartWidth = 1000;
const smooth = SMA([
  [0, 1000],
  [1, 1243],
  // ...
], chartWidth);

LTTB

Largest triangle three buckets (read more here). If you are looking for the best performing downsampling method then look no more!

function LTTB(data: DataPoint[], targetResolution: number): DataPoint[]

LTTB accepts an array of data points (see DataPoint) or a TypedArray (see TypedArray support) and a target resolution (number of output data points) as arguments. See advanced API if you need to work with a custom data type.

The format of the data will be preserved, i.e. if passing an array of [number, number] data points as data, you will get an array of [number, number] on the output.

import { LTTB } from 'downsample';

// Or if your codebase does not supprot tree-shaking
import { LTTB } from 'downsample/methods/LTTB';

const chartWidth = 1000;
const downsampled = LTTB([
  [0, 1000],
  [1, 1243],
  // ...
], chartWidth);

LTOB

Largest triangle one bucket (read more here). Performs only slightly worse than LTTB.

function LTOB(data: DataPoint[], targetResolution: number): DataPoint[]

LTOB accepts an array of data points (see DataPoint) or a TypedArray (see TypedArray support) and a target resolution (number of output data points) as arguments. See advanced API if you need to work with a custom data type.

The format of the data will be preserved, i.e. if passing an array of [number, number] data points as data, you will get an array of [number, number] on the output.

import { LTOB } from 'downsample';

// Or if your codebase does not supprot tree-shaking
import { LTOB } from 'downsample/methods/LTOB';

const chartWidth = 1000;
const downsampled = LTOB([
  [0, 1000],
  [1, 1243],
  // ...
], chartWidth);

LTD

Largest triangle dynamic (read more here). The simplest downsampling method.

function LTD(data: DataPoint[], targetResolution: number): DataPoint[]

LTD accepts an array of data points (see DataPoint) or a TypedArray (see TypedArray support) and a target resolution (number of output data points) as arguments. See advanced API if you need to work with a custom data type.

The format of the data will be preserved, i.e. if passing an array of [number, number] data points as data, you will get an array of [number, number] on the output.

import { LTD } from 'downsample';

// Or if your codebase does not supprot tree-shaking
import { LTD } from 'downsample/methods/LTD';

const chartWidth = 1000;
const downsampled = LTD([
  [0, 1000],
  [1, 1243],
  // ...
], chartWidth);

DataPoint type

Represents a data point in the input data array. These formats are currently supported:

type DataPoint =
  [number, number] |
  [Date, number] |
  { x: number; y: number } |
  { x: Date; y: number } |

TypedArray support

It is now possible to pass TypedArray data to downsampling functions. The returned type will then match the input type, e.g. if Int16Array is passed in, the result will be a Int16Array:

const input: Int16Array = new Int16Array(...);
const result: Int16Array = LTD(input, 1000);

Advanced API

All the functions above work with DataPoint objects as a reasonable default. If however this does not fit your needs you can create your own version of a function using a downsampling function factory.

createASAP

Creates an ASAP smoothing function for a specific point data type P.

function createASAP({
  x: string | number | (point: P) => number,
  y: string | number | (point: P) => number,
  toPoint: (x: number, y: number) => P
}): ASAP;

createSMA

Creates a SMA smoothing function for a specific point data type P.

function createSMA({
  x: string | number | (point: P) => number,
  y: string | number | (point: P) => number,
  toPoint: (x: number, y: number) => P
}): SMA;

createLTD

Creates an LTD downsampling function for a specific point data type P.

function createLTD({
  x: string | number | (point: P) => number,
  y: string | number | (point: P) => number
}): LTD;

createLTOB

Creates an LTOB downsampling function for a specific point data type P.

function createLTOB({
  x: string | number | (point: P) => number,
  y: string | number | (point: P) => number
}): LTOB;

createLTTB

Creates an LTTB downsampling function for a specific point data type P.

function createLTTB({
  x: string | number | (point: P) => number,
  y: string | number | (point: P) => number
}): LTTB;

Demo

There is a very minimal interactive demo app available if you want to play around with the results of downsampling. Check it out here.

Acknowledgement

The implementation of LTD, LTOB and LTTB is based on Sveinn Steinarsson's 2013 paper Downsampling Time Series for Visual Representation that can be found here.

The implementation of ASAP is based on Kexin Rong's and Peter Bailis's 2017 paper. ASAP: Prioritizing Attention via Time Series Smoothing that can be found here. The original code can be found here

downsample's People

Contributors

dependabot[bot] avatar janjakubnanista avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

downsample's Issues

Importing with --experimental-modules

Thanks for the module! Is there a way to import a specific method in Node v8.5+ with the --experimental-modules flag? import { LTTB } from 'downsample' fails.

SyntaxError: The requested module 'downsample' does not provide an export named 'LTTB'

Feature request: support x array and y array input data

I'm using Plotly and have formatted my tooling to generate data formatted like this:

var trace1 = {
  x: [1, 2, 3, 4],
  y: [10, 15, 13, 17],
  mode: 'markers',
  type: 'scatter'
};

var trace2 = {
  x: [2, 3, 4, 5],
  y: [16, 5, 11, 9],
  mode: 'lines',
  type: 'scatter'
};

var trace3 = {
  x: [1, 2, 3, 4],
  y: [12, 9, 15, 12],
  mode: 'lines+markers',
  type: 'scatter'
};

var data = [trace1, trace2, trace3];

Plotly.newPlot('myDiv', data);

Since there's a lot of data (the reason I'm downsampling in the first place!) changing data shapes twice would be a big overhead unless there's an efficient way I'm missing.

Support for ArrayBuffers

Is it possible to support ArrayBuffer and similar structures like Uint8Array, Float32Array, etc?

Ideally it would be great if I can pass a Uint8Array as argument and it will give me the same type array back.

Right now I'd have to convert the ArrayBuffer to the DataPoint structure and that's very inefficient in my case.

For ASAP method, what is the difference between 1000 original data points downsampled to 1000 data points versus 900 original data points to 900

Hello, we were playing around with the periodic data demo and had a question regarding the ASAP downsampling method.

When I 'downsample' 1000 original points to 1000 downsampled points, I get a graph that looks like this:
image

I would say that the ASAP trendline matches the original data fairly accurately, applied a little bit of smoothing, but the original data is pretty much intact.

However, when I downsample 900 original points to 900 downsampled points, I get a graph that looks like this:
image

Although theoretically it seems to me that mapping 1000 -> 1000 or 900 -> 900 should've made no difference in the amount of detail that is preserved, in practice, it seems like there's quite a bit of difference, where the 900 -> 900 lost significantly more detail than 1000 -> 1000.

Was wondering if this is a characteristic somehow of the ASAP downsampling method, or if this was a potential bug. We would like the results of the 1000 -> 1000 sampling consistently, to preserve around that level of detail as a result of the downsampling.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.