Code Monkey home page Code Monkey logo

reductio's Introduction

Crossfilter

Join the chat at https://gitter.im/crossfilter/crossfilter Tests CDNJS

Crossfilter is a JavaScript library for exploring large multivariate datasets in the browser. Crossfilter supports extremely fast (<30ms) interaction with coordinated views, even with datasets containing a million or more records.

NOTE: We are seeking new maintainers for this repo. See #171 for discussion.

Since most interactions only involve a single dimension, and then only small adjustments are made to the filter values, incremental filtering and reducing is significantly faster than starting from scratch. Crossfilter uses sorted indexes (and a few bit-twiddling hacks) to make this possible, dramatically increasing the perfor­mance of live histograms and top-K lists. Crossfilter is available under the Apache License.

This is a community-maintained fork of the original square/crossfilter library.

Want to learn more? See the wiki.

Gallery of Community Examples

Installation

This package can be found under the name crossfilter2 in npm:

 npm install crossfilter2

Development

Install dependencies:

  • npm install

Test changes with npm test. Build with npm run build for use in the browser. You may also want to benchmark your changes with npm run benchmark.

reductio's People

Contributors

alemarre avatar christophe-g avatar esjewett avatar gitter-badger avatar hauenstein avatar hypercubed avatar jkleint avatar saboya avatar tannerlinsley avatar xaseracheron avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

reductio's Issues

Import of reductio getting failed ❗

import reductio from "reductio";

While importing the package, getting the following error

========================================
node_modules\reductio\src\reductio.js:1
import build from './build.js';
^^^^^^

SyntaxError: Cannot use import statement outside a module

How can I get this resolved?

Any interest in forking crossfilter and combining it with reductio?

@esjewett
I know you're aware of this thread about keeping crossfilter moving forward: square/crossfilter#151

Reductio follows the Node approach of small modules built on top of one another, which has it's benefits. But the functions you've built in reductio are so commonly needed that it's a good argument to include them in the crossfilter library itself. At the same time, Square isn't able to put more resources into Crossfilter, and it's been a slow process in opening it up to other contributors.

There was talk of crossfilter being part of dc.js, but I don't think that makes sense. I use it outside of dc.js.

One solution would be for you to just fork crossfilter and combine it with reductio. You could invite collaborators that are in that other thread. It could be moved into an official organization, but I don't know if that is necessary.

Getting something up and running and starting to address a couple of the biggest issues crossfilter has right now, such as support for more than 32 dimensions and web workers (square/crossfilter#53) would be awesome and I think move the decision-making process along quicker. If it's done well, it will become the "blessed fork" and take some heat off the Square folks.

I think you would be great to take this on because of what you've done so far with reductio.

I'm asking in this repo instead of the other thread because you would be driving it initially (but I'm making sure to reference that thread so this question shows up there). Not sure if you like this approach and could take it on. If you do, I bet the other folks would support you in doing it.

Just an idea...

Change exception aggregation API

May want to change to be

reductio.exception(value)
exception.sum(value)
exception.count(boolean)

reductio.exception should return the exception object rather than the reductio object. Breaks chaining, but is probably a bit more intuitive?

Old exceptionSum, exceptionCount methods would stick around for now but would be deprecated and would not appear in documentation.

Ratio/Rate Reduction

Hi is there way to perform a reduction of the type sum(field1) / sum(field2). Thanks in advance.

max

hi,
i don't use open soruce. i use windows and want to use just max values of some data. there are more than one values of data and i want to group by firstly department name and after daiyly and calculate max value day.
department group day count


computer software 27/2/2015 3
computer hardware 27/2/2015 2
electric left 25/2/2014 3
computer technical 30/2/2015 4
is there any simple solution for this?

solution is:
computer 27/2/2015 5
electric 25/2/2015 3

Count Distinct

Is there a way to count distinct values currently?

I have a creeping suspicion I will be needing to do some cardinality dimensions and having them as reducers would be awesome

var group = reductio()
  .countDistinct(d => d.order_number)
  (group);

would have

[{
  "key": "dimension1",
  "value": {
    "countDistinct": 4
  }
}, {
  "key": "dimension2",
  "value": {
    "countDistinct": 2
  }
}]

Do exceptions support multiple values

Apologies if this is the wrong place to raise what is probably a question or a suggestion rather than an issue - does reductio.exception support multiple values in the same way as reductio.value - i.e. can I code reductio.value.exceptionsum for multiple items under the same reductio.exception unique key?

From my attempts I don't think this can be done via reductio, however I have a flattened one-to-many dataset where I have 2 separate values to sum at the lowest detail level, and 2 other separate values to sum at the exception level, so I need to do this either via reductio or using custom reduceAdd etc functions.

Question: has the Key of the Group been considered passed into aggregations as part of function signature

I ran into a situation where when i am using a groupAll with a grouping function involved to gather up all of the values of a string but it would be useful for filtering out duplicates if i knew what key i was working on as part of the dimension during aggregation... something that i am paraphrasing as what follows

const dimensionIncoming = crossfilterData.dimension((d) => d.incoming)

const group = dimensionTrain.groupAll()
const reducer = reductio().groupAll((record) => {
    return [record.incoming, record.outgoing]
})

reducer
      .value('incorrect')
      .filter((d, key) => {
        return d.incoming !== d.outgoing && d.incoming === key
      })
      .dataList(true)

the values in incoming and outgoing overlap but i am trying to get the stats of a given incoming value and what ends up happening is that because incoming / outgoing can be the same value it ends up duplicating the row. i can kinda use exception but it still has the same problem that i cant correctly filter out the things that have the key value only in outgoing which should not be part of the statistics on the key.

I realize that this is mainly a problem because im using the key value as part of the groupAll but in my particular use case it ends up working out that this makes sense to do and that the missing piece is that key value that is the dimension value. I also tried using the crossfilter.accessor(d) but that is telling me what the dimension value of this data item is not what group im currently working on unfortunately.

my ultimate solution was to simply do the incoming != outgoing without the key and then post process the results to add the behavior of trimming the list for the correct key matching strategy. while this works i gather that it would benefit from the performance that crossfilter already provides.

any advice or thoughts?

Cap group and retaining grouping values

I am trying to .cap(10) a crossfilter dimension group to having the 10 highest values and then a group for whatever is rest. However, I am struggling to find out how to what the group Others is actually containing.

Take this code:

const group = reductio().count(true)(dimension.group());
const result = group
  .post()
  .sortBy("value.count", (a, b) =>
    b < a ? -1 : b > a ? 1 : b >= a ? 0 : NaN
  )
 .cap(10)();

This groups as I am expecting, but I need to know what the Others is actually grouping. Is there anyway I can get that in? I was hoping for a result like:

[
  {
    key: "Group1",
    value:  {
      count: 123
      keys: ["Group1"]
    },
     /* ... more groups like this */
    {
      key: "Others",
      value:  {
        count: 3453
        keys: ["Group10", "Group11", "Group12"]
    }
]

If using .dataList(true) on reductio, the dataList array prop on Others is empty. I would love to use crossfilter alone and not rely on reductio, if possible.

Top alias array errors when defining an alias (ex: 'group.top(1)[0].complete' is undefined)

Occurs when defining a reductio.alias(mapping) or reductio.aliasProp(mapping), Attempting to return the k records according to the natural order of the dimension by specifying the alias per the API reference.

On the group, we can then call the following function to retrieve the new count value.

group.top(1)[0].newCount();

`//Sample Data
var data = [
{"type":"exe","total":24,"handled":0,"nothandled":24},
{"type":"exe","total":1,"handled":0,"nothandled":1},
{"type":"exe","total":1,"handled":0,"nothandled":1},
{"type":"exe","total":1,"handled":0,"nothandled":1},
{"type":"exe","total":1,"handled":0,"nothandled":1}
]

// Set crossfilter
var ndx = crossfilter(dirData);

// Dimension
var dim = ndx.dimension(function(d) {return "" + d.type;});

// Group
var group = dim.group();

// reductio.alias(mapping)
var reducer = reductio().count(true)
.sum(function(d) { return +d.handled; })
.aliasProp({
newCount: function(g) { return g.count; },
complete: function(g) { return g.sum * 100 / g.count; }
});
reducer(group); //<-- WORKS

// Optimal Result:
//object Object = [
// {"key":"dir","value":{"count":5,"sum":28,"newCount":28,"complete":0}}
//]

//On the group, attempt to call the following to retrieve the complete value.
group.top(1)[0].complete(); //<-- ERROR
//group.top(1)[0]; //<-- works

// Suboptimal Result:
//TypeError: group.top(1)[0].complete is not a function. (In 'group.top(1)[0].complete()', 'group.top(1)[0].complete' is undefined)`

Allow built in aggregations to be pre-aliased

It looks like there are aliasing methods available like:
https://github.com/esjewett/reductio#reductioaliasmapping
But, it looks like none of these are actually meant to override the property name of the aggregation from the get go. To produce a clean value object, you would need to go back and remove the original aggregator property name.

My idea would is something along the lines of allowing a second parameter to override the property name of the aggregation:

reductio()
  .count(true, 'myCount')
  .sum(function(d) { return +d.num; }, 'theSum')

// eg.
[{
  key: 1,
  value: {
    myCount: 5,
    theSum: 25
  }
},{
  key: 2,
  value: {
    myCount: 3,
    theSum: 45
  }
}]

Setting a group's order method

I am trying to use post to cap the keys but I am not able to make sense of the following in the README.

It might be wise to set your groups order method before using this.

I usually follow this
barGroup=reductio().count(true).sum('foo')(barDim.group())

Could someone please elaborate on this? How should I go about this?

d3 format '.2f' sum precision

Hi,

I have an attribute amount which I'm converting into d3.format('.2f'). In the reductio reducer the sum accessor is
function(d) { return +d.amount; }
I am expecting a sum amount that is also in '.2f' format but in one of the cases I see more than 2 decimal places.
Object { =4, sum=396.54999999999995}

The summed amounts are as follows
346.5, 15.4, 19.25, 15.4

I was expecting a 396.55

What is the reason behind this behaviour?

Thanks

group.post.cap(length).() always does natural ordering

var skuDim = ndx.dimension(function(d){ return d.sku; });
var skuGroup = skuDim.group();
var skuReducer = reductio().count(true).sum('hits');
skuReducer(skuGroup);
skuGroup.order(function(p) { return p.sum; });
console.log(skuGroup.top(Infinity));
console.log(skuGroup.post().cap(3)());`

crossfilter top orders correctly. reductio post().cap(length)() doesn't.

Reducing multiple values

Hi Ethan. Have you thought about reducing multiple different (unrelated) values? This is a common requirement for dc.js, e.g. for assigning another aesthetic to another value, like color and radius for a scatter/bubble plot.

(Out of scope: nesting/pivot/series type things - not talking about multiple keys but multiple values.)

Allow count to function like sum

Many times you get data from a server which has been pre-reduced in some way (e.g. already has a count)
In those cases it would be exceedingly valuable to allow count to simply sum those counts together.

Proposed api:

reductio().count(true); // p.count++;
reductio().count('items') // p.count += v.items;

test fail to run on recent nodejs

I'll provide a fix in the coming days - along with an upgrade to ES6 modules (for reductio, universe and crossfliter), if everything runs smoothly.

npm run test

> [email protected] test /home/christophe/Programming/Polymer/litElement/reductio
> ./node_modules/karma/bin/karma start --browsers PhantomJS --single-run

/home/christophe/Programming/Polymer/litElement/reductio/node_modules/socket.io/lib/store.js:35
Store.prototype.__proto__ = EventEmitter.prototype;
                                         ^

TypeError: Cannot read property 'prototype' of undefined
    at Object.<anonymous> (/home/christophe/Programming/Polymer/litElement/reductio/node_modules/socket.io/lib/store.js:35:42)
    at Module._compile (internal/modules/cjs/loader.js:778:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)
    at Module.load (internal/modules/cjs/loader.js:653:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
    at Function.Module._load (internal/modules/cjs/loader.js:585:3)
    at Module.require (internal/modules/cjs/loader.js:692:17)
    at require (internal/modules/cjs/helpers.js:25:18)
    at Object.<anonymous> (/home/christophe/Programming/Polymer/litElement/reductio/node_modules/socket.io/lib/manager.js:16:13)
    at Module._compile (internal/modules/cjs/loader.js:778:30)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] test: `./node_modules/karma/bin/karma start --browsers PhantomJS --single-run`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the [email protected] test script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/christophe/.npm/_logs/2019-10-03T14_03_31_075Z-debug.log

The last alias function is clobbering all the previous alias functions

Hi,
Thanks for putting this library together. While using it, I think that I've found a bug in the way the alias functions are mapped.

Unfortunately, I'm pretty new to javascript and not a software engineer (though I am a scientist), so I hope I can describe this adequately.

Let's say I set up an alias:
reducer.alias({a : function() {return "a";}, //for discussion call this function a
{b : function() {return "b";}); //for discussion call this function b

Later when I use the reducer, I have some object:
Object
a : function() // unfortunately, this is function b
b : function() // function b as it is supposed to be

But the problem is the pipes got screwed up and now 'a' calls function b (instead of function a), and 'b' calls function b.

If I reorder my alias, then both 'a' and 'b' call function a.
e.g.
reducer.alias({b : function() {return "b";},
{a : function() {return "a";} //for discussion call this function a
); //for discussion call this function b

I (with the help of a coworker) managed to track the issue to the reductio_alias variable assignment (https://github.com/esjewett/reductio/blob/master/reductio.js#L339-L349). It is due to the scoping of the prop variable inside the for loop:
var reductio_alias = {
initial: function(prior, path, obj) {
return function (p) {
if(prior) p = prior(p);
for(var prop in obj) {
path(p)[prop] = function() { return objprop; }; // prop inside this function is
// not the same as outside the
// function at the time the
// function gets call
}
return p;
};
}
};

In case the comment above is too cryptic, 'path(p)[prop]=' is evaluated correctly, because prop is defined and changing within the scope of the for loop. This is why we get an object in the end with all of the proper keys. However, 'objprop' inside the function doesn't get evaluated until the function is called later on. At this point in all cases, because we are out of the scope of the for loop, prop is equal to the last value that it held when the for loop ended. Thus all keys will call the same function (or something like that...)

Anyway, I solved the issue by wrapping the anonymous function in a named function that takes the prop as an argument. It all seems to work now.

var reductio_alias = {
initial: function(prior, path, obj) {
return function (p) {
if(prior) p = prior(p);
function buildAliasFunction(key){
return function(){
return objkey;
};
}
for(var prop in obj) {
path(p)[prop] = buildAliasFunction(prop);
}
return p;
};
}
};

I hope this helps, even if you choose to solve this a different way.

I am also opening a pull request for your convenience in case you choose this solution.

Unable to find suitable version of crossfilter

When installing via bower I see this error message:

$ bower install --save reductio 
bower reductio#*                cached https://github.com/esjewett/reductio.git#0.6.3
bower reductio#*              validate 0.6.3 against https://github.com/esjewett/reductio.git#*
bower crossfilter2#1.4.0-alpha.06           cached https://github.com/crossfilter/crossfilter.git#1.4.0-alpha.06
bower crossfilter2#1.4.0-alpha.06         validate 1.4.0-alpha.06 against https://github.com/crossfilter/crossfilter.git#1.4.0-alpha.06
bower                                    ECONFLICT Unable to find suitable version for crossfilter2

groupAll Performance

From square/crossfilter#100

I added a column in the data via js with date.toDateSting() to test your theory. Here is the changed code.

var testDim = cs_data.dimension(function(d) {return d.DateString;});
var test_shipments = testDim.group().reduceSum(dc.pluck('DollarValue'));

reducer = reductio().groupAll(function(record) {
    var idx = mapped_date_array.indexOf(record.DateString);

    if (record.ShipDate < date_array[30]) {
        return [mapped_date_array[idx]];
    } else {
        var i = 0;
        var return_array = [];

        // we are finding the 30 day moving avg 
        for (i = 30; i >= 0; i--) {
              if (typeof(mapped_date_array[idx - i]) != "undefined") {
                   return_array.push(mapped_date_array[idx - i]);
              }
        }

        return return_array;
    }
}).count(true).sum(function(d) { return d.DollarValue; }).avg(true);

and ran

console.time("values");
shipments_moving_avg.value();
console.timeEnd("values");

to test.

This yielded "values: 9869.228ms"

Readme.md code formatting issues

It appears that the markdown auto-formatting of some of the code blocks is messed up due to missing empty lines to separate the HTML title code and the markdown code blocks. I sorted it out in the attached file (sorry, Github doesn't allow uploading .md files, and I was too lazy to go through putting in a PR :-) )

README.txt

Thanks for the library!

Correct `crossfilter2` dependency for AMD usage.

The current ~UMD build doesn't correctly specify the crossfilter2 dependency for AMD style dependencies.

In particular, this is on the first line of the ~compiled reductio.js:

if("function"==typeof define&&define.amd)define([],e)

And then it later tries to load crossfilter via:

var crossfilter = (typeof window !== "undefined" ? window['crossfilter'] : typeof global !== "undefined" ? global['crossfilter'] : null);

I believe the dependency should be properly expressed via something like this:

if("function"==typeof define&&define.amd)define(['crossfilter2'],e)

I'm sure this is all supposed to be handled by the build tools, but their current settings are causing me to have to export crossfilter2 as a global to use in an AMD environment.

For what it's worth, dc.js works well in an AMD environment. Their ~compiled form includes this definition:

if(typeof define === "function" && define.amd) {
        define(["d3", "crossfilter2"], _dc);
    }

I realize this may not be a high priority for you, but I can always hope that it's an easy fix. Or at least I can warn other users (that might still be using an AMD environment).

Thanks for the nice tool. It's working very nicely in a 2-level aggregation scenario with a dc.js solution.

crossfilter not found

I'm using reductio 0.5.3 in a CommonJS environment (webpack in my case).

I'm getting Module not found: Error: Cannot resolve module 'cross filter' in these reductio modules:

@ ./~/reductio/src/reductio.js 5:18-40
@ ./~/reductio/src/value-count.js 1:18-40
@ ./~/reductio/src/value-list.js 1:18-40
@ ./~/reductio/src/histogram.js 1:18-40
 @ ./~/reductio/src/nest.js 1:18-40

Reductio 0.5.3 has this dependency: "crossfilter2": "^2.0.0-alpha.03". I don't know if there is a problem in reductio or crossfilter2. Previously I actually tried to use crossfilter2 with dc.js but gave up because I couldn't get dc.js to require it properly. Maybe crossfilter2 isn't exporting it's modules properly? Maybe this belongs in crossfilter2's repo, but the error is originating from Reductio so I wanted to let you know about it.

I've tried quite a few webpack shenanigans to get it to work, including the ones that are used with modules that weren't originally built to work in a CommonJS environment. But none of them worked. I've got lots of other modules working in my environment, which leads me to think it has something to do with reductio or crossfilter2.

Thanks

Please support Standard Deviation

Have you considered supporting Standard Deviation?

Moved over from Gordon's Issue (#1) Reducing Multiple Values...

On 9/15/2014 8:06 AM, Ethan Jewett wrote:

@mr23 https://github.com/mr23 Standard deviation would definition
just fit as a new type of aggregation - it won't really affect any
changes to the API. If you want to create a separate issue, that would
be great. And if you can remind me how to calculate the std dev, that
would be great too :-)


Reply to this email directly or view it on GitHub
#1 (comment).

http://en.wikipedia.org/wiki/Standard_deviation

I drafted up some files for Sum of Squares and StD.
I'm not sure this is the best arrangement or reflects everything needed
test-wise.
As I said, just a draft.
I did not set up a build environment yet so I'm not sure it will compile
without an error.

But I figured the least I could do would be to offer up some example
code for the Std.

Next opportunity I'll make another pass over this, but could be a few
days or more. You never know what life brings next.

-Chris

ps. in my original email reply tonight to the Issue#1 thread, I attached the draft files. I figured they wouldn't show up here in the issue, but I don't know if you (Ethan) will received a copy of them in an email. If not, please contact me via email (use one in profile); I don't have time tonight to work up a fork, ..., pull request.

Better instruction on how to avoid `VALUELIST accessor is being overwritten by max aggregation`

I was curious about a good kitchen sink test of reductio on my data, and so I kept incrementally growing my calls on it and checking the data.

Everything behaves as expected, but I noticed these warnings pop up:

reductio.js:67 VALUELIST accessor is being overwritten by max aggregation
reductio.js:51 VALUELIST accessor is being overwritten by median aggregation

Here's a test case to demonstrate how easy it is the run into this:

var data = [
    {date: "2011-11-14T16:17:54Z", quantity: 2, total: 190, tip: 100, type: "tab"},
    {date: "2011-11-14T16:20:19Z", quantity: 2, total: 190, tip: 100, type: "tab"},
    {date: "2011-11-14T16:28:54Z", quantity: 1, total: 300, tip: 200, type: "visa"},
    {date: "2011-11-14T16:30:43Z", quantity: 2, total: 90, tip: 0, type: "tab"},
    {date: "2011-11-14T16:48:46Z", quantity: 2, total: 90, tip: 0, type: "tab"},
    {date: "2011-11-14T16:53:41Z", quantity: 2, total: 90, tip: 0, type: "tab"},
    {date: "2011-11-14T16:54:06Z", quantity: 1, total: 100, tip: 0, type: "cash"},
    {date: "2011-11-14T16:58:03Z", quantity: 2, total: 90, tip: 0, type: "tab"},
    {date: "2011-11-14T17:07:21Z", quantity: 2, total: 90, tip: 0, type: "tab"},
    {date: "2011-11-14T17:22:59Z", quantity: 2, total: 90, tip: 0, type: "tab"},
    {date: "2011-11-14T17:25:45Z", quantity: 2, total: 200, tip: 0, type: "cash"},
    {date: "2011-11-14T17:29:52Z", quantity: 1, total: 200, tip: 100, type: "visa"}
];

    var preparedData = $.map(data, function(datum, idx) {
        datum.date = moment(datum.date ).toDate();
        return datum;
    });

    // Prepare data, dimensions, and groups
    var sharedDx = crossfilter(preparedData);

    var dateDimForReductio = sharedDx.dimension(function (d) {
        var momentedDate = moment( d.date );
        return momentedDate.year() + '-' + momentedDate.month() + '-' + momentedDate.hour();
    });
    var groupForReductio = dateDimForReductio.group();

    var totalsAccessor = function() { return function(d) {
        return d.total;
    }; };

    var reducer = reductio();
    reducer
        .min(totalsAccessor())
        .median(totalsAccessor() )
    ;

    reducer(groupForReductio);

I think that the warnings are unavoidable with how the source is written. Is that intentional, or is there a better way of accomplishing this I am not seeing?

Here are the alternatives I have been able to imagine so far:

  • Using several different reducers or the same reducer with separate value function sets. This definitely works, but leaves me wondering about performance -- is that causing multiple iterations through the dataset when only one is necessary, with both steps being performaned each tick? I would like to be as performant as possible, within reason. I can figure this out, but have not spent the time debugging things yet, and am hoping someone can illuminate this without me having to go to that extent.
  • If valueList were an object, and each reducio function put its own valueLists under a key with that reducio function's name as the key, then they could co-exist in the same reductio-ized group. Then it would look like value.valueList.min next to value.valueList.max.
  • If the valueList was compared against the proposed one, so that if they are identical no warning is thrown. This would have its own performance costs, every call. I point this out because the valueList from both min and max and median seem to be identical.. so the warning is harmless.
  • A boolean flag could be added to disable the warning. This way when we have seen the warning and understand the complications, we can explicitly dismiss it from appearing.

I am hoping there's just something silly I'm missing that someone can point me to, or that some of this might help improve reductio. :) 👍 to this library!!

How to include constant string values to reduced values (Crossfilter and Reductio)

Given a data that looks like:

data = [
    {   
        "partner": "ABCD",
        "category": "CORTICOSTERIODS",
        "name": "NP Slow Meedrol Methylprednisolone Sodium Succinate Injection USP 500mg x 1 Vial",
        "price_sum": 22288.00,
        "qty_sum": 50.00000000000000000
    },
    {
        "partner": "ABCD",
        "category": "MEDICAL DISPOSABLE",
        "name": "4 INCH CAST FIBREGLASS",
        "price_sum": 135324.00,
        "qty_sum": 63.00000000000000000
    },
    {
        "partner": "ABCD",
        "category": "DERMATOLOGICALS",
        "name": "ABF CREAM 20G",
        "price_sum": 150169.29,
        "qty_sum": 150.00000000000000000
    },
    {   
        "partner": "CDEF",
        "category": "CORTICOSTERIODS",
        "name": "NP Slow Meedrol Methylprednisolone Sodium Succinate Injection USP 500mg x 1 Vial",
        "price_sum": 22288.00,
        "qty_sum": 50.00000000000000000
    },
    {
        "partner": "CDEF",
        "category": "MEDICAL DISPOSABLE",
        "name": "4 INCH CAST FIBREGLASS",
        "price_sum": 135324.00,
        "qty_sum": 63.00000000000000000
    }
]

I would like to add the category of the keys to the values when grouped by names alongside the sum of prices and qtys for further processing. With the help of crossfilter and reductio, I got what is consumable by other functions, just that I couldn't figure out how to add the category of each of the keys to it.

What I am currently using:

function categ_tab(data){
    // The needed data should be grouped by names summed up by qty and price
    // Thus crossfilter and reductio will be used to take the burden off server
    var group = crossfilter(data).dimension(function(d) { return d.name; }).group(),
        reducer = reductio();
    // sum up qty and price
    reducer.value("qty").sum(function(d) { return d.qty_sum; });
    reducer.value("price").sum(function(d) { return d.price_sum; });
    return reducer(group).all();

Current Result:

[
    {
        "key":"4 INCH CAST FIBREGLASS",
        "value":{
            "qty":{"sum":126},
            "price":{"sum":270648}
        }
    },
    {
        "key":"ABF CREAM 20G",
        "value":{
            "qty":{"sum":150},
            "price":{"sum":150169.29}
        }
    },
    {
        "key":"NP Slow Meedrol Methylprednisolone Sodium Succinate Injection USP 500mg x 1 Vial",
        "value":{
            "qty":{"sum":100},
            "price":{"sum":44576}
        }
    }
]

Desired Result:

[
    {
        "key":"4 INCH CAST FIBREGLASS",
        "value":{
            "qty":{"sum":126},
            "price":{"sum":270648},
            "category": {"iDontCare":"MEDICAL DISPOSABLE"}
        }
    },
    {
        "key":"ABF CREAM 20G",
        "value":{
            "qty":{"sum":150},
            "price":{"sum":150169.29},
            "category": {"iDontCare":"DERMATOLOGICALS"}
        }
    },
    {
        "key":"NP Slow Meedrol Methylprednisolone Sodium Succinate Injection USP 500mg x 1 Vial",
        "value":{
            "qty":{"sum":100},
            "price":{"sum":44576},
            "category": {"iDontCare":"CORTICOSTERIODS"}
        }
    }
]

Thank you.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.