Code Monkey home page Code Monkey logo

mongo-views's Introduction

mongo-views

Build Status

Supports MongoDB 2.2 <= 3.0

This is a MongoDB skunkworks project to enable queryable views within the shell. Views are like virtual collections, that can be queried as regular collections.

They support:

  • Criteria
  • Projections
  • Joins
  • Nesting

Why might you want this? Well lets say you want to save a query for regular reuse. Say you have an employees collection:

db.employees.insert(
    [
        {name: "John", dob: new Date(1980, 1, 2)},
        {name: "Paul", manager: true, dob: new Date(1983, 7, 10), uid: 3},
        {name: "Mary", dob: new Date(1985, 5, 12), uid: 20},
        {name: "Aimee", manager: true, dob: new Date(1945, 2, 20), uid: 50}
    ]
)

and we want all managers from an employee collection. Then you could create a view via:

db.employees.createView("managers", { manager: true })

and query/sort/limit it as though it was a collection via

db._managers.find().sort({ name: -1 }).pretty()
/* yields =>
{
  "_id": ObjectId("54f9e58e1d8a2ac246213516"),
  "name": "Paul",
  "manager": true,
  "dob": ISODate("1983-08-10T04:00:00Z"),
  "uid": 3
}
{
  "_id": ObjectId("54f9e58e1d8a2ac246213518"),
  "name": "Aimee",
  "manager": true,
  "dob": ISODate("1945-03-20T04:00:00Z"),
  "uid": 50
}
*/

it's virtual, so if you add to the underlying collection(s)

db.employees.insert( {name: "Ian", manager: true, dob: new Date(1995, 1, 20), uid: 99 })

then the same view query yields:

db._managers.find().sort({ name: -1 }).pretty();
/* yields =>
{
  "_id": ObjectId("54f9e58e1d8a2ac246213516"),
  "name": "Paul",
  "manager": true,
  "dob": ISODate("1983-08-10T04:00:00Z"),
  "uid": 3
}
{
  "_id": ObjectId("54f9e5b41d8a2ac24621351a"),
  "name": "Ian",
  "manager": true,
  "dob": ISODate("1995-02-20T05:00:00Z"),
  "uid": 99
}
{
  "_id": ObjectId("54f9e58e1d8a2ac246213518"),
  "name": "Aimee",
  "manager": true,
  "dob": ISODate("1945-03-20T04:00:00Z"),
  "uid": 50
}
*/

you can of course add criteria to the find()

db._managers.find({ name: /Paul/ }).sort({ name: -1 }).pretty();
/* yields =>
{
  "_id": ObjectId("54f9e58e1d8a2ac246213516"),
  "name": "Paul",
  "manager": true,
  "dob": ISODate("1983-08-10T04:00:00Z"),
  "uid": 3
}
*/

you can then create nested views just as easily

db._managers.createView("senior_managers", { dob: {$lt: new Date(1990, 0 , 1) } })

db._senior_managers.find()
/* yields =>
{
  "_id": ObjectId("54f9d8b3f088c1c44badce68"),
  "name": "Paul",
  "manager": true,
  "dob": ISODate("1983-08-10T04:00:00Z")
}
{
  "_id": ObjectId("54f9d8b3f088c1c44badce6a"),
  "name": "Aimee",
  "manager": true,
  "dob": ISODate("1945-03-20T04:00:00Z")
}
*/

We can see all our views so far via

show views
/* yields =>
managers
senior_managers
*/

Maybe we don't want senior managers to show the _id field, then we use a projection

// remove view first
db._senior_managers.drop();

db._managers.createView("senior_managers", { dob: {$lt: new Date(1990, 0 , 1)} }, { _id: 0 })

db._senior_managers.find()
/* yields =>
{
  "name": "Paul",
  "manager": true,
  "dob": ISODate("1983-08-10T04:00:00Z"),
  "uid": 3
}
{
  "name": "Aimee",
  "manager": true,
  "dob": ISODate("1945-03-20T04:00:00Z"),
  "uid": 50
}
*/

we can even combine projections as in

db._senior_managers.find({}, {uid: 0, manager: 0})
/* yields =>
{
  "name": "Paul",
  "dob": ISODate("1983-08-10T04:00:00Z")
}
{
  "name": "Aimee",
  "dob": ISODate("1945-03-20T04:00:00Z")
}
*/

it's just a cursor, so we can sort and limit as expected:

db._senior_managers.find().sort({ dob: 1 }).limit(1)
/* yields =>
{
  "name": "Aimee",
  "manager": true,
  "dob": ISODate("1945-03-20T04:00:00Z")
}
*/

Now what about joins ? Easy. Join to the users.

// add users
db.users.insert([
  { id: 99, email: "[email protected]" },
  { id: 50, email: "[email protected]" },
  { id: 20, email: "[email protected]" },
  { id: 3, email: "[email protected]"}
])

db.employees.createView('employees_with_email', {}, {}, { target: db.users, from: "uid", to: "id"})

db._employees_with_email.find().sort({name: 1})

/* yields =>
{
  "_id": {
    "from": ObjectId("54f9ebb1257b0c8dc73be97a"),
    "to": ObjectId("54f9f1c4067bf2a2b99c53b1")
  },
  "name": "Aimee",
  "manager": true,
  "dob": ISODate("1945-03-20T04:00:00Z"),
  "uid": 50,
  "id": 50,
  "email": "[email protected]"
}
{
  "_id": {
    "from": ObjectId("54f9ec10461b20c42cabc3d4"),
    "to": ObjectId("54f9f1c4067bf2a2b99c53b0")
  },
  "name": "Ian",
  "manager": true,
  "dob": ISODate("1995-02-20T05:00:00Z"),
  "uid": 99,
  "id": 99,
  "email": "[email protected]"
}
{
  "_id": {
    "from": ObjectId("54f9ebb1257b0c8dc73be979"),
    "to": ObjectId("54f9f1c4067bf2a2b99c53b2")
  },
  "name": "Mary",
  "dob": ISODate("1985-06-12T04:00:00Z"),
  "uid": 20,
  "id": 20,
  "email": "[email protected]"
}
{
  "_id": {
    "from": ObjectId("54f9ebb1257b0c8dc73be978"),
    "to": ObjectId("54f9f1c4067bf2a2b99c53b3")
  },
  "name": "Paul",
  "manager": true,
  "dob": ISODate("1983-08-10T04:00:00Z"),
  "uid": 3,
  "id": 3,
  "email": "[email protected]"
}
*/

It's all a cursor, so guess what? You can even join a view to another view!

Want to see what's inside your view? Inspect it!

db._employees_with_email.inspect()
/* yields =>
{
  "name": "employees_with_email",
  "target": "employees",
  "query": {

  },
  "projection": {

  },
  "join": {
    "target": "users",
    "from": "uid",
    "to": "id"
  }
}
*/

Moreover, these views persist. Both when you switch DBs via use [db] or by restarting the shell.

Views are virtual, and only save the state of the query used to create them. This means that each time a query is performed on a view, the latest collection data is fetched.

Installation

  • In POSIX environments, run make

  • In WinX environments, please add mongorc.js to your MongoDB installation folder (if it doesn't exist), run npm run build to generate the browserify bundle, and finally copy the contents of dist/bundle.js into ``mongorc.js`.

Basic Usage

Create

db.[collection|view].createView(
  name:String,
  criteria:Object,
  projection:Object,
  join: {
    target: [collection|view]
    from: String,
    to: String
  })

See all views in DB

show views

Inspect a view

db._[view].inspect()

Query

db._[view].find(criteria:Object, projection:Object):DBQuery

Drop

db._[view].drop()

Criteria

  • Under the hood, views composed criteria using $and operators. So all criteria parameters in the view, along with any find criteria in the find call, will be condensed into a single criteria object.

ie. in the above example,

db.employees.createView("managers", { manager: true });
db._managers.find({ name: /Jane/ });

Will yield

db.employees.find({ $and: [{ manager: true }, { name: /Jane/ }] });

Projection

  • MongoDB allows for projections in the find function. Fields can be enabled or disabled, either as whitelists or blacklists see MongoDB docs.

  • In order to properly combine projections, we must combine the two sets in certain ways:

    1. For matched fields in both the view and the find projection, we bitwise AND them (meaning that unless they are both true, the field is off)
    2. For fields enabled in the base projection, only those enabled in the find projection will remain.
    3. For fields disabled in the base projection, all of those disabled in the find projection will be added.

Egs.

Case 1:

db.employees.createView("managers", { manager: true }, { name: 1, _id: 1 });
db._managers.find({ }, { _id: 0 });

// yields =>
db.employees.find({ ... }, { name: 1, _id: 0 }); // id set to 0 from 1~0

Case 2:

db.employees.createView("managers", { manager: true }, { name: 1, id: 1 });
db._managers.find({ }, { name: 1 });

// yields =>
db.employees.find({ ... }, { name: 1 }); // id removed as not in find() projection

Case 3:

db.employees.createView("managers", { manager: true }, { id: 0 });
db._managers.find({ }, { email: 0 });

// yields =>
db.employees.find({ ... }, { id: 0, email: 0 }); // id removed as not in find() projection

Join

Currently supports a single join to another collection or view.

Naming conflicts are solved by prefixing the fields with collection or view name and an underscore.

_id field is a compound key of from and to _ids

API:

join: {
    target: [collection|view],
    from: String, // foreign key in this collection or view
    to: String    // unique key in target collection or view
}

Guidelines

  • Views are scoped to the DB level

  • View names must be unique, and cannot match any given collection name in that DB

  • Views based on dropped collections or views will be removed automatically

  • Joins are performed in-memory, and may take a long time for large collections

Run Tests

First time grab deps: npm install

Then run npm test

mongo-views's People

Contributors

cswanson310 avatar geertbosch avatar jjgonecrypto avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar

Forkers

geertbosch cswanson310 stennie jell0720 linearregression isabella232

mongo-views's Issues

Persisting regexes does not work.

Including a regex in criteria, such as { name: /john/ }, will not load on startup. The reason is that JSON.stringify is not correctly serializing regular expressions.

Steps to reproduce:

  • create a view with an inline regex in the criteria
  • query the view, works as expected
  • restart mongo
  • query the view, note how the regex criteria is not hit

createView: empty criteria field should default to {} 

> db.employees.insert(
...     [
...         {name: "John", dob: new Date(1980, 1, 2)},
...         {name: "Paul", manager: true, dob: new Date(1983, 7, 10), uid: 3},
...         {name: "Mary", dob: new Date(1985, 5, 12), uid: 20},
...         {name: "Aimee", manager: true, dob: new Date(1945, 2, 20), uid: 50}
...     ]
... )

> db.employees.createView("managers")
WriteResult({ "nInserted" : 1 })

> db._managers.find()
Error: error: {
    "$err" : "Can't canonicalize query: BadValue $or/$and/$nor entries need to be full objects",
    "code" : 17287
}

But this works:

db.employees.createView("managers", {})

mongo refuses to start up in --nodb mode

$ ./mongo --nodb
MongoDB shell version: 3.1.0-pre-
mongo-views is initiating!
2015-03-11T12:20:29.187-0400 E QUERY ReferenceError: db is not defined
at getCurrentDb (/Users/charlie/github/my-mongo-views/lib/config.js:23:20)
at loadViewsInDB (/Users/charlie/github/my-mongo-views/lib/init.js:94:13)
at Object.init.js (/Users/charlie/github/my-mongo-views/lib/init.js:133:5)
at DBCollection (/Users/charlie/.mongo-views.js:29:34)
at /Users/charlie/.mongo-views.js:31:3
at /Users/charlie/.mongorc.js:2:1 at /Users/charlie/github/my-mongo-views/lib/config.js:23
2015-03-11T12:20:29.187-0400 E QUERY Error: error loading js file: /Users/charlie/.mongo-views.js
at /Users/charlie/.mongorc.js:2:1 at /Users/charlie/.mongorc.js:2
The ".mongorc.js" file located in your home folder could not be executed

Regex is not persisted correctly.

@cswanson310

Steps to repro:

db.createView('managers', { name: /john/})

restart shell

and nothing is returned from the query.

persistence for regex looks like

{
  "_id": ObjectId("54f8a114ad4fc3af583357f2"),
  "name": "managers",
  "target": "employees",
  "query": "{\"name\":{}}",
  "projection": "{}"
}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.