Code Monkey home page Code Monkey logo

universe's Introduction

Crossfilter

Join the chat at https://gitter.im/crossfilter/crossfilter Tests CDNJS

Crossfilter is a JavaScript library for exploring large multivariate datasets in the browser. Crossfilter supports extremely fast (<30ms) interaction with coordinated views, even with datasets containing a million or more records.

NOTE: We are seeking new maintainers for this repo. See #171 for discussion.

Since most interactions only involve a single dimension, and then only small adjustments are made to the filter values, incremental filtering and reducing is significantly faster than starting from scratch. Crossfilter uses sorted indexes (and a few bit-twiddling hacks) to make this possible, dramatically increasing the perfor­mance of live histograms and top-K lists. Crossfilter is available under the Apache License.

This is a community-maintained fork of the original square/crossfilter library.

Want to learn more? See the wiki.

Gallery of Community Examples

Installation

This package can be found under the name crossfilter2 in npm:

 npm install crossfilter2

Development

Install dependencies:

  • npm install

Test changes with npm test. Build with npm run build for use in the browser. You may also want to benchmark your changes with npm run benchmark.

universe's People

Contributors

akngs avatar etimberg avatar hypercubed avatar markherhold avatar saboya avatar tannerlinsley avatar tonyper avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

universe's Issues

Cleanup methods for queries

Right now, there is no method to dispose of a query (much like disposing a group in crossfilter), other than clearing the column definition (which disposes the dimension and all of it's groupings).

There should be a .clear() function returned with the .query() promise that when called will clear the query/group from memory and deregister all listeners.

Bugs in filter method

I haven't had the time to isolate this one into a test but if I create a filter using a function:

const fn => () { /* some function /* }
universe.filter('search', fn)

Then later try to add another filter I get the error: f is not a function at filterIndexFunction.

I have determined this is because the filter function gets converted to a string here: https://github.com/crossfilter/universe/blob/master/src/filters.js#L38

My suggested solution is:

      var newFilters = _.clone(service.filters, true)
      _.map(newFilters, function(fil, i) {
        if (fil.type === 'function') {
          fil.function = service.filters[i].function
          fil.value = service.filters[i].value
        }
      })

crossfilter2 package.json error when compiling

I got this error after installing it. Using it with Webpack and Babel. I don't know if it helps but I was using Crossfilter.js without issues.

image

Let me know if you need more information.

some more examples would be welcomed ; )

Hi,
I really appreciate the effort of lowering the barrier of entry to crossfilter - in particular providing some kind of grammar for querying multivariate dataset. However, playing around the library is not extremely straightforward at the moment, and I found that its added value over defining your own dimensions and groups in crossfilter directly was not obvious from the provided code samples.

Maybe porting the Airline on-time performance example to universe could help a user like myself see/understand/delve into a more comprehensive usecase (e.g. filtering the dataset on multiple dimensions, see how we can add listeners to filtering actions, ... ).

I actually tried to port this example myself - without success (at least given the timeframe I allowed myself to test out the lib - and the available amount of brain cells...).

Thanks anyway,
C.

P.S. There are some helpful hints/example in the gitter channel though - saw them a bit too late!
P.P.S. Oh - this should be highlighted ! http://hypercubed.github.io/Project-Chi/#/examples/universe

squash method not working like in the example

i'm working on a project that need capabilities that universe library have, but i had some trouble implementing the squash method.
This is my code, basically its the same with the example provided in the readme section

universe([
    {date: "2011-11-14T16:17:54Z", quantity: 2, total: 190, tip: 100, type: "tab", productIDs: ["001"]},
    {date: "2011-11-14T16:20:19Z", quantity: 2, total: 190, tip: 100, type: "tab", productIDs: ["001", "005"]},
    {date: "2011-11-14T16:28:54Z", quantity: 1, total: 300, tip: 200, type: "visa", productIDs: ["004", "005"]},
    {date: "2011-11-14T16:28:54Z", quantity: 3, total: 285, tip: 100, type: "tab", productIDs: ["001", "002", "005"]}
]).then(function(myUniverse){
    myUniverse.query({
        groupBy: 'type', // GroupBy the type key
        select: {
            $count: true, // Count the number of records
            quantity: { // Create a custom 'quantity' column
                $sum: 'quantity' // Sum the quantity column
            },
        },
        // Limit selection to rows where quantity is greater than 50
        filter: {
            tip: {
                $gt: 50
            }
        },
    }).then(function(result) {
          result.squash(null, null, {
            count: '$sum',
            quantity: {
                sum: '$sum'
            }
        },'Total').then(function(resultq) {
                   console.log(JSON.stringify(resultq.data));
            })
    })

universe_squash.txt

and this is the output
[{"key":"Total","value":{"count":4,"quantity":{"0":"sum"}}}]

what i was expecting from what i read in the example is something like the output below
[{"key":"Total","value":{"count":4,"quantity":{"sum":8}}}]

Am i calling the squash method the wrong way or is this a bug?
Sorry for my bad English.

Allow column aliasing

If an aggregation is the only nested item in an alias, then the resulting data should be flatten into that alias, eg.

.query({
  groupBy: 'type',
  select: {
    myColumn: {
      $sum: 'total'
    }
  }
})

should result in:

[{
  key: 'type1',
  value: {
    myColumn: 30
  }
}, {
  key: 'type2',
  value: {
    myColumn: 15
  }
}, {
  key: 'type3',
  value: {
    myColumn: 24
  }
}, {
  key: 'type4',
  value: {
    myColumn: 12
  }
}]

this should also work with the string syntax. eg.

.query({
  groupBy: 'type',
  select: {
    myColumn: '$sum(total)'
  }
})

Track and manage queries (groups)

Right now, dimensions are tracked and managed with easy references in universe.columns, but the queries made on those columns are not tracked anywhere.

universe.columns[n].queries would be a great place to put these query objects.

We could store there:

  • the original query object
  • crossfilter group
  • listeners/callbacks
  • post-aggregations (in the future)
  • etc.

query results object dimension is undefined

the result object in the console will not contain a crossfilter dimension as stated in the documentation.

universe([
  {date: "2011-11-14T16:17:54Z", quantity: 2, total: 190, tip: 100, type: "tab", productIDs:["001"]},
  {date: "2011-11-14T16:20:19Z", quantity: 2, total: 190, tip: 100, type: "tab", productIDs:["001", "005"]},
  {date: "2011-11-14T16:28:54Z", quantity: 1, total: 300, tip: 200, type: "visa", productIDs:["004" ,"005"]},
  {date: "2011-11-14T16:30:43Z", quantity: 2, total: 90, tip: 0, type: "tab", productIDs:["001", "002"]},
  {date: "2011-11-14T16:48:46Z", quantity: 2, total: 90, tip: 0, type: "tab", productIDs:["005"]},
  {date: "2011-11-14T16:53:41Z", quantity: 2, total: 90, tip: 0, type: "tab", productIDs:["001", "004" ,"005"]},
  {date: "2011-11-14T16:54:06Z", quantity: 1, total: 100, tip: 0, type: "cash", productIDs:["001", "002", "003", "004" ,"005"]},
  {date: "2011-11-14T16:58:03Z", quantity: 2, total: 90, tip: 0, type: "tab", productIDs:["001"]},
  {date: "2011-11-14T17:07:21Z", quantity: 2, total: 90, tip: 0, type: "tab", productIDs:["004" ,"005"]},
  {date: "2011-11-14T17:22:59Z", quantity: 2, total: 90, tip: 0, type: "tab", productIDs:["001", "002", "004" ,"005"]},
  {date: "2011-11-14T17:25:45Z", quantity: 2, total: 200, tip: 0, type: "cash", productIDs:["002"]},
  {date: "2011-11-14T17:29:52Z", quantity: 1, total: 200, tip: 100, type: "visa", productIDs:["004"]}
])
.then(function(myUniverse){
  return myUniverse.query({
    groupBy: 'productIDs',
    select: {
      $count: true,
    }
  })
  .then(function(res){
    console.info('res:',res)
   })
})

Add a linter?

I've noticed that the code contains mixed styles (ASI and semicolons). Would you consider a PR to add a linter? I prefer XO. Standard and semi-standard are both good as well. Personally, I prefer semicolons and spaces... but I will set up the linter however you prefer.

New tag version

Could we have a new tag version including the latest code from master ?

I'd personally need that to be able to integrate the latest fix into bower

universe filterAll method missing

The documentation shows usage of a filterAll method on a universe instance (https://github.com/crossfilter/universe#explore-your-data). However this method, while it exists in the ./src/filters.js file (https://github.com/crossfilter/universe/blob/master/src/filters.js#L48) ), is not exposed on the universe instance. Furthermore, the filterAll method as it exists is not sufficient.

Here is a gist of a mocha tests for filterAll: https://gist.github.com/Hypercubed/fb7eac133e8675f3e193de36cd3d3459

You will see three tests:

  1. filterAll method is not exposed
  2. filterAll method clears the universe.filters object but does not clear the individual filters.
  3. A promise chain that does clear each filter and clears the filter object.

Given this I think the proper filterAll method should be:

function filterAll() {
  return Promise.all(_.keys(service.filters).map(function(key) {  // clearing each filter
    return service.filter(key)
  }))
  .then(function() {  // clearing filters object
    return applyFilters({})
  })
}

or equivalent.

Root universe.js and universe.min.js

Currently, you are using gulp and browserify to build universe.js and universe.min.js files in the root. package.json uses src/universe.js as main. The tests use '../universe.js' (the browserified version). Can I suggest that the tests use src/universe.js and we .gitignore the built files? Are you planning to support bower?

Also, as an aside, I mostly switched from using gulp to using only npm scripts. Since this build is fairly simple we might consider the same here.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.