Code Monkey home page Code Monkey logo

gun-level's Introduction

gun-level

Travis npm Gitter

LevelDB is awesome. It's awesomer with gun.

indexedDB NOW SUPPORTED IN MAIN GUN REPO - THIS REPO NO LONGER MAINTAINED

The main GUN repo now has lib/rindexed that can be used like so, now built on the RAD storage engine for GUN.

If you still need NodeJS LevelDB bindings, consider writing a RAD adapter, or use this project - please contribute and maintain it for others, as it is now deprecated.

Overview

GunDB is a graph database engine with real-time sync and offline-editing. Although gun comes with storage and sync out of the box, it's design is pluggable, so you can still use your favorite storage backend or transport layer by using an adapter.

LevelDB operates on a similar paradigm. It ships with an interface called LevelUP that gives all the methods you'd expect from a storage engine (like get, put, batch, etc.) and forwards those to a lower-level database (it uses LevelDOWN by default).

Arguably the most valuable aspect of level is it's ecosystem. There are tons are plugins, backends, dashboards, and utilities made specifically for level. It's kinda like a database buffet.

Here, check out their list of modules!

If you're feeling stressed, don't worry. LevelDB comes out of the box as a quite capable database, and if you don't want to dabble with plugins and configs, there's no need.

So what happens when you combine Level with GunDB? You get the power and control of level right alongside the ease of gun.

That's what this library does.

Usage

To get started, you'll first need to install gun-level.

If you're unfamiliar with npm, you can get started here

$ npm install --save gun-level gun

Now require them from your node project.

// Import the `Gun` library
const Gun = require('gun')

// Imported for side effects, adds level adapters.
// MUST be required after Gun to work
require('gun-level')

Once they're imported you can create a new Gun database:

const gun = new Gun({
	// We'll put options here in a moment.
})

Sweet, you're set up! However, gun-level won't do anything unless you pass it a levelDB instance through the Gun constructor. For that, you'll need to download the Node levelup package.

There are some differences between levelup v1 and v2 so be sure to note which version you're using.

levelup v1

$ npm install --save levelup leveldown

If you get a build error at this step, replace all examples of leveldown with jsondown.

// Import the two libraries
const levelup = require('levelup')
const leveldown = require('leveldown')

// Create a new level instance which saves
// to the `data/` folder.
const levelDB = levelup('data', {
	db: leveldown,
})

levelup >v2

Note that Levelup v2 only supports Node >6.

$ npm install --save levelup leveldown encoding-down
// Import the required libraries
const levelup = require('levelup')
const encode = require('encoding-down')
const leveldown = require('leveldown')

// Create a new level instance which saves
// to the `data/` folder.
const levelDB = levelup(
	encode(
		leveldown('data'),
		{ valueEncoding: 'json' }
	)
)

Instantiating Gun with Level

Now we can pass our new levelDB instance to the Gun constructor.

const Gun = require('gun')
require('gun-level')

// ... instantiate LevelDB per above

// Pass LevelDB instance into Gun
const gun = new Gun({
	level: levelDB
})

Done! Now your gun database will store it's graph in your Level DB!

Let's try a few things...

const bob = gun.get('bob').put({ name: 'Bob' })
const dave = gun.get('dave').put({ name: 'Dave' })

// Write a fun circular reference.
bob.get('friend').put(dave)
dave.get('friend').put(bob)

// Print the data!
bob.get('friend').val(friend => {
	console.log(friend.name) // Dave
});

// Now with a circular reference
bob.get('friend').get('friend').val(friend => {
	console.log(friend.name) // Bob
});

That's pretty much all there is to the gun-level API. If you're unfamiliar with gun's API, here's a good reference.

Advanced Levelry

You've seen the basics, but it's not enough. You crave more power.

To exchange backends with level, like Riak, Mongo, IndexedDB, etc., you can find the official list of storage backends here. Usually it's just a matter of passing the module as the db option to levelup.

Here's an example with MongoDB using mongodown:

const levelup = require('levelup')
const mongoDown = require('mongodown')

// Initialize Level
const levelDB = levelup('localhost:27017/YOUR_COLLECTION_NAME', {
	db: mongoDown,
})

// Initialize Gun
const gun = new Gun({
	level: levelDB,
	file: false,
})

Here's another example with IndexedDB using level-js:

const levelup = require('levelup')
const leveldown = require('level-js')
const encode = require('encoding-down')

// Initialize Level
const levelDB = levelup(encode(leveldown('my-big-db'), { valueEncoding: 'json' }))

// Initialize Gun
const gun = Gun({
    level: levelDB,
    localStorage: false,
})

Even if you're content with the default levelDB setup, I really recommend you scan this list of plugins. It's inspiring what the community has built.

gun-level will try to read and write values as json. If you're having trouble getting a plugin to work, or keep seeing "[object Object]", make sure it's using the json value encoding.

Getting Support

If you're running into problems, feel free to either post an issue on GitHub or chat with us humans in the Gitter channel.

Installing from Source

Clone the repo and install the dependencies:

$ git clone https://github.com/PsychoLlama/gun-level.git
$ cd gun-level
$ npm install

Running Tests

# In directory `gun-level`
$ npm test

Building

$ npm run build
# Compiles to folder `dist/`

Contributors

  • Project owner @PsychoLlama
  • Code contributor @swhgoon
  • The friendly @greenkeeperio-bot

Sponsored by the fabulous people at GunDB.

gun-level's People

Contributors

509dave16 avatar amark avatar csterritt avatar dfreire avatar greenkeeper[bot] avatar psychollama avatar sjones6 avatar swhgoon avatar zrrrzzt avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

gun-level's Issues

gun-level fails to load in React Native

You'll get a nasty error when including gun-level in React Native.

screen shot 2016-10-08 at 3 50 06 pm

Here's a theory: React Native recursively builds all the projects you import through babel. However, since gun-level has the babel config in the package.json, babel sees it and tries to load the plugins it lists, and since those are only dev-dependencies, it can't find them and just dies.

Possible solution: move the config into a .babelrc file and list it in the .npmignore.

react-native-cli: 1.0.0
react-native: 0.34.1
[email protected]

I am trying to get gun-level to work with level-js in the browser

This is code I have tried in Runkit [https://runkit.com/58470cf725386e0014684507/5886876a7bb1cd001392cc50](Runkit gun-level with level-js)

I get this error

ReferenceError Stack Trace Viewer
ReferenceError: self is not defined
new IDBStore in idb-wrapper/idbstore.js
line 129, column 56
Level._open in level-js/index.js
line 37, column 14
AbstractLevelDOWN.open in abstract-leveldown/abstract-leveldown.js
line 28, column 17
LevelUP.open in levelup/lib/levelup.js
line 117, column 6
new LevelUP in levelup/lib/levelup.js
line 87, column 8
[object Object].LevelUP in levelup/lib/levelup.js
line 47, column 12
in this notebook
line 13, column 17
Code Node Startup Show Full Stack Trace...

But when trying to run it on my windows webpack setup I get a very different message

Error in .//levelup/package.json
Module parse failed: C:\Users\Jonathan\Documents\Freelance\Jehle\desconjugador\node_modules\levelup\package.json Unexpected token (2:9)
You may need an appropriate loader to handle this file type.
SyntaxError: Unexpected token (2:9)
@ ./
/levelup/lib/util.js 32:25-51

It seems to me it should just work but I am a bit new at this levelary stuff and not a ton of experience with node npm and wepack, I've just got the basics down.

Could you show me what I would need to do to get gun-level working with level-js. My goal is to have gundb backed by indexeddb in the browser.
Thanks

Test results are not consistent

I did a PR and discovered one of the tests failed on travis. I ran the tests on my local machine and it passed. By running the tests multiple times I found that the result was not predictable.

The problem seems to be this line: https://github.com/PsychoLlama/gun-level/blob/master/src/spec.js#L87

How to recreate

clone repo
npm install

Add a script to the root of the repo run-multiple.js

const { exec } = require('child_process')
let jobs = 100
let done = 0
let fail = 0
let success = 0

function printResult () {
  console.log(`Fail: ${fail}`)
  console.log(`Success: ${success}`)
  console.log('\n')
}

while (jobs > done) {
  done++
  console.log(`Running ${done} of ${jobs}`)
  exec('npm test', (error, stdout, stderr) => {
    if (error) {
      fail++
    } else {
      success++
    }
    printResult()
  })
}

Run the script node run-multiple.js

Look at the results.

gun-level not working (with gun 0.7.4)

I just tried out gun-level 0.5 and it does not work, but also does not give an error.

const gundb = Gun({ level: levelDB, file: false, });

leveldb itself works fine, I can put and get stuff into and out from it, but anything with gundb fails silently, e.g. after:

gundb.get('foo').put({bar:'hello'});

there is nothing in the actual database records representing that entry.

and when doing a query:

gundb.get('bar').val(function(data){ console.log("Hit:"); console.log(data); })

the callback does never gets called.

Failing test

  ․․․․․․․․․․!․․

  12 passing (504ms)
  1 failing

  1) Gun using level should merge with existing data:
     Uncaught Error: Expected { _: { '#': 'gkww0wuvad6', '>': { success: 1520904130506 } }, success: true } to include { success: true, data: true }
      at assert (node_modules/expect/lib/assert.js:29:9)

on src/spec.js:87
changing the makeGun() to the glogbal gun variable fixes the issue, but not sure if that's the right behavior or not? or this is an async issue?!

An in-range update of mocha is breaking the build 🚨

Version 3.4.0 of mocha just got published.

Branch Build failing 🚨
Dependency mocha
Current Version 3.3.0
Type devDependency

This version is covered by your current version range and after updating it in your project the build failed.

As mocha is “only” a devDependency of this project it might not break production or downstream projects, but “only” your build or test tools – preventing new deploys or publishes.

I recommend you give this issue a high priority. I’m sure you can resolve this 💪

Status Details
  • continuous-integration/travis-ci/push The Travis CI build failed Details

Release Notes v3.4.0

Mocha is now moving to a quicker release schedule: when non-breaking changes are merged, a release should happen that week.

This week's highlights:

  • allowUncaught added to commandline as --allow-uncaught (and bugfixed)
  • warning-related Node flags

🎉 Enhancements

🐛 Fixes

🔩 Other

Commits

The new version differs by 9 commits0.

  • 7554b31 Add Changelog for v3.4.0
  • 9f7f7ed Add --trace-warnings flag
  • 92561c8 Add --no-warnings flag
  • ceee976 lint test/integration/fixtures/simple-reporter.js
  • dcfc094 Revert "use semistandard directly"
  • 93392dd no special case for macOS running Karma locally
  • 4d1d91d --allow-uncaught cli option
  • fb1e083 fix allowUncaught in browser
  • 4ed3fc5 Add license report and scan status

false

See the full diff

Not sure how things should work exactly?

There is a collection of frequently asked questions and of course you may always ask my humans.


Your Greenkeeper Bot 🌴

level db files created, but gun using ./data.json as well

I am doing what was specified in readme, snippet:

 const levelDB = levelup(
      encode(
        leveldown(this.opts.file),
        { valueEncoding: 'json' }
      )
    );

    this.db = new Gun({level: levelDB, web: this.httpServer});

The leveldb folder is created and the data is in there but I still get the warning:
"WARNING! This file.js module for gun is intended for local development testing only!"

And the data is also in a data.json that gets put in my module root. This may be a gun issue or some configuration I am missing?

Sorry in advance if I am missing anything basic here.

file:false not respected by Gun

I have it all setup and it's working :)
BUT i named the data file 'level_data' in const levelDB =levelup('level_data',{db:jsondown})

It turns out that Gun is still creating the data.json file , so now 2 files are created.

Publish latest version to NPM

Hey!

The latest version of this package in NPM is still referencing gun 0.8.x
However, master is already aligned with 0.9.x
If there's no outstanding issue, increase gun-level version and publish to NPM.

Thanks

I am trying to get gun-level to work with fruitdown in the browser

I came across your example https://gist.github.com/PsychoLlama/5894445f48254f6f3f00

and added modified my create-elm-app to run parts of your example as I couldn't get it to run directly.

in index.js I have


var Gun = require('gun-level');

// so you can experiment in the browser
window.gun = new Gun({
    level: {
        // sets the "down" implementation
        down: require('fruitdown')
    }
});


const bob = gun.get('bob').put({ name: 'Bob' })
const dave = gun.get('dave').put({ name: 'Dave' })


// Write a fun circular reference. 
bob.path('friend').put(dave)
dave.path('friend').put(bob)


// Print the data! 
bob.path('friend.name').val()
bob.path('friend.friend.name').val()

and I added

new webpack.IgnorePlugin(/(gun/lib/wsp|fs)/)

to my pluggins

I am getting the following error in the console.

[HMR] Waiting for update signal from WDS...
index.js:118 Uncaught TypeError: level.get is not a function
    at Adapter.read (index.js:118)
    at Object.get (index.js:37)
    at wire (gun.js:732)
    at load (gun.js:709)
    at Object.on [as fn] (gun.js:738)
    at Gun.eval [as get] (gun.js:743)
    at eval (webpack:///./src/index.js?:39)
    at Object.<anonymous> (bundle.js:1336)
    at __webpack_require__ (bundle.js:556)
    at fn (bundle.js:87)
gun.js:1180 Warning! You have no peers to connect to!

I'm not sure what the problem could be.

fatal crash related to gun.off()

Not exactly sure what I did, but probably one of:

gun.get('testing').put({pointers: null})
gun.get(id).put(null)
gun.get(id).off()
gun.get(id).get(id2).off()
gun.get(id).map(v=> v.off())
gun.get(id).map().val(v=> v.off())
gun.get(id).map().val((v, k)=> gun.get(id).get(k).off())
const item = gun.get(id); item.map().val((v, k)=> item.get(k).off())

gun.get('name').put({key: 'value'});
gun.get('name').val(console.log)
gun.get('name').off()
gun.get('name').val(console.log)

...when trying to figure out how to remove data.

node_modules/levelup/lib/levelup.js:145
    throw new ReadError('get() requires a key argument')
    ^
ReadError: get() requires a key argument
    at LevelUP.get (/node_modules/levelup/lib/levelup.js:145:11)
    at Adapter.read (/node_modules/gun-level/dist/Adapter/index.js:114:20)
    at Object.next (/node_modules/gun-level/dist/index.js:32:14)
    at Object.onto [as on] (/node_modules/gun/gun.js:203:19)
    at Function.Gun.on.get (/node_modules/gun/gun.js:793:35)
    at Object.root [as next] (/node_modules/gun/gun.js:697:14)
    at Object.next (/node_modules/gun/nts.js:17:20)
    at Object.next (/node_modules/gun/lib/bye.js:11:45)
    at Object.onto [as on] (/node_modules/gun/gun.js:203:19)
    at receive (/node_modules/gun/lib/ws.js:84:7)
error Command failed with exit code 1.

I suspect context.get['#'] ("key") becomes null/undefined somehow which breaks it all, even after restart. Adding if (!key) return _this.afterRead(context, null) // possibly warn? or is this correct? at line 112 in /dist/Adapter/index.js should do.

...or just make it use the radix storage thing? :)

Update docs for Level v2

Level v2 is out! Woooohooooo!!

Thankfully, there's nothing which breaks the adapter functionality.

A few packages have been moved around though, so the readme will need to be updated. General gist for people finding this issue:

- var levelup = require('levelup')
- var level = levelup('name', {
+ var leveldb = require('level')
+ var level = leveldb('name', {
    valueEncoding: 'json',
-   db: require('leveldown'),
  })

Pull out leveldown and levelup and replace them with level. The rest is mostly the same.

For people using different level backends, it looks like the encoding-down package makes it work the same as it did in v1. Here's an example:

Note: this code hasn't been vetted. I'm pretty sure this is how it works though.

var levelup = require('levelup')
var down = require('some-other-backend-to-level')
var encoding = require('encoding-down')
var level = levelup('db-name', {
  valueEncoding: 'json',
  db: encoding(down),
})

Check out the level changelog for more details. They know more about the changes than I do 😉

gun-level not working with gun 8.8

Express server crashes with error message:

gun-level/dist/Adapter/index.js:179
        gun._.root.on('in', {
           ^

TypeError: Cannot read property '_' of undefined

The code below works perfectly with the file adapter, but not with gun-level

Server code:

const express = require('express');
const path = require('path');
const favicon = require('serve-favicon');
const logger = require('morgan');
const cookieParser = require('cookie-parser');
const bodyParser = require('body-parser');
const sassMiddleware = require('node-sass-middleware');
const levelup = require('levelup');
const leveldown = require('leveldown');
const Gun = require('gun/gun'); // this includes core, no defaults!
require('gun/lib/file');
require('gun/lib/path');
require('gun/lib/not');
require('gun/lib/ws');
require('gun/nts');
require('gun-level');

const app = express();
const publicDir = path.join(__dirname, 'public');

// uncomment after placing your favicon in /public
//app.use(favicon(path.join(publicDir, 'favicon.ico')));
app.use(logger('dev'));
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({extended: false}));
app.use(cookieParser());
app.use(sassMiddleware({
    src: publicDir,
    dest: publicDir,
    indentedSyntax: false, // true = .sass and false = .scss
    sourceMap: true
}));
//app.use(Gun.serve);
app.use(express.static(publicDir));

// catch 404 and forward to error handler
app.use(function (req, res, next) {
    const err = new Error('Not Found');
    err.status = 404;
    next(err);
});

// error handler
app.use(function (err, req, res, next) {
    // set locals, only providing error in development
    res.locals.message = err.message;
    res.locals.error = req.app.get('env') === 'development' ? err : {};

    // render the error page
    res.status(err.status || 500);
    res.render('error');
});

function normalizePort(val) {
    const port = parseInt(val, 10);

    if (isNaN(port)) {
        // named pipe
        return val;
    }

    if (port >= 0) {
        // port number
        return port;
    }

    return false;
}

const port = normalizePort(process.env.PORT || '3100');
app.set('port', port);

/**
 * Create HTTP server.
 */

const server = app.listen(port);

const levelDB = levelup('data', {
    db: leveldown
});

const gun = Gun({
    web: server,
    level: levelDB,
    file: false
});

As soon as the client (website, build with create-react-app setup) executes the following code, the server crashes with the error given above:

const Gun = require('gun/gun');
const peers = [
'http://localhost:3100/gun'
];
const gun = Gun(peers);
gun.get('docs').put({name: 'Test'});

With indexeddb, retrieving data from a node after a browser refresh returns undefined

I detected a problem when trying to use gun in the browser with indexeddb.
Apparently, I cannot get data from a node after a browser refresh.

This is the localStorage implementation, which works fine:

// setup with localStorage
const gun = Gun();

// run
gun.get('something').val() // undefined as expected
gun.get('something').put({ timestamp: Date.now() })
gun.get('something').val() // timestamp as expected

// refresh
gun.get('something').val() // timestamp as expected

This is the indexeddb implementation, which has the problem:
(same thing with level-js and fruitdown)

// setup with indexedDB
const Gun = require('gun-level');
const levelup = require('levelup');
const leveldown = require('level-js');
const db = leveldown('my-big-db');
db.status = 'unknown'; // see https://github.com/Level/level.js/issues/59
const gun = Gun({
  localStorage: false,
  level: levelup(db),
});

// run
gun.get('something').val() // undefined as expected
gun.get('something').put({ timestamp: Date.now() })
gun.get('something').val() // timestamp as expected

// refresh
gun.get('something').val() // undefined !!
// wait a lot of time
gun.get('something').val() // still undefined !!
// write again, then read
gun.get('something').put({ timestamp: Date.now() })
gun.get('something').val() // timestamp as expected

Example code not working with jsondown

Hey - thanks for doing this. I'm trying to run the demo:

// Import the gun library
var Gun = require('gun');
require('gun-level');

// Import the two libraries
const levelup = require('levelup')
const jsondown = require('jsondown')

// Create a new level instance which saves
// to the `data/` folder.
const levelDB = levelup('data', {
    db: jsondown,
})

// Create a new gun instance
var gun = Gun({
    level: levelDB,
});

// Read `greetings`, saving it to a variable.
var greetings = gun.get('greetings');

// Update the value on `greetings`.
greetings.put({
    hello: 'world',
})

And it's doing two surprising things. First, it still writes the 'data.json' file, and second, it writes nothing in the 'data' directory (although it creates it).

This is on macOS 10.12.2, using node v6.9.1, gun 0.3.9991, and gun-level 4.0.1.

What am I doing wrong? Thanks!

gun-level + level-js not working at all

const Gun = require('gun')
require('gun-level')
const levelup = require('levelup')
const leveljs = require('level-js')
const encode = require('encoding-down')
const db = levelup(encode(leveljs('bigdata'), {valueEncoding: 'json'}))
const a = Gun({
 level: db,
 localStorage: false
})
a.get('toto').put({toto: 'test'}, ack => {
 if (ack.err) console.error(ack)
})

This simple piece of code is not working at all on Chrome or Firefox.

No value in the indexedDb database. And the ack error is received.

PS 1: This code was not run direclty on the browser. This code was bundled using webpack, then run.
PS 2: This code works with leveldown on NodeJs instead of level-js

Using Chrome Version 68.0.3440.106 (Build officiel) (64 bits)
Using Firefox 61.02*

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.