get all is not releasing the file lock after it's used

Hi,

When using the plugin I get this console error: fs.existsSync is not a function. Also get this error while compiling code with Webpack: Error: Can't resolve 'fs' in '/Users/Gijs/Server/anvil/node_modules/electron-json-storage/lib.

Am I doing something wrong?

Two issues were found with this working as remote

First with promises:

See electron/electron#3740

An application using this as remote will hang. This can be resolved by removing the returns from the various exported methods, and using the callback.

Second, the app lookup in utils.js fails - the app needs to be retrieved from the electron remote instead of directly from electron.

After adding this dependency i'm seeing an error:

ERROR in ./~/cb2promise/index.js
Module not found: Error: Cannot resolve 'file' or 'directory' /Users/johnryan/Desktop/app/node_modules/cb2promise/lib/cb2promise in /Users/johnryan/Desktop/app/node_modules/cb2promise
 @ ./~/cb2promise/index.js 3:17-55

and

ERROR in ./~/coffee-script/bin/coffee
Module parse failed: /Users/johnryan/Desktop/app/node_modules/coffee-script/bin/coffee Unexpected character '#' (1:0)
You may need an appropriate loader to handle this file type.
SyntaxError: Unexpected character '#' (1:0)

Is there a recommended solution for this?

Due to the changes in v3.2.0, there are bugs when calling storage.set(key, json, [options], callback) without options

Uncaught Exception:
TypeError: Cannot read property 'dataPath' of undefined
    at Object.exports.set

It throws error when trying to access options.dataPath at https://github.com/electron-userland/electron-json-storage/blob/master/lib/storage.js#L256

I think we need to ensure options is set to {} at https://github.com/electron-userland/electron-json-storage/blob/master/lib/storage.js#L251 and make sure all other similar methods do the same thing, like .get and .has .remove .clear

Hi,
electron-json-storage seems to work fine while working on a mac app, though on windows it does not create a particular storage json file under app.getPath('userData') directory. Could you help ? What am I doing wrong ?
Sometimes also creating an empty json file.
Have checked with the object I am setting to the key

I have an electron app - in the renderer, I set a bunch of configuration variables, like:

Constants = namespace.Constants = {
    SHADOWS: false, //render shadows?
}

Then, as the system starts up, it reads the value of those variables and configures the application environment.

Now, in my main process, I build I tray menu:

    function createTray (){
      mainWindow.toggleDevTools();
      tray = new Tray(iconPath);
      contextMenu = Menu.buildFromTemplate([
        {label: 'Shadows', type: 'checkbox', checked: false, click: function(item){
          contextMenu.items[0].checked = contextMenu.items[0].checked;
          tray.setContextMenu(contextMenu);
          mainWindow.webContents.send('toggleShadows', contextMenu.items[0].checked);
        }},
      ])
      tray.setToolTip('Jordan')
      tray.setContextMenu(contextMenu)
    }

I then hooked up the ipc renderer to be able to detect that change in the context menu:

    const {ipcRenderer} = require('electron');

    ipcRenderer.on('toggleShadows', (event, message) => {
      Constants.SHADOWS = message;
    });

This works to change the value of that variable, the problem is, the variable is only taken into consideration when Constants is initialized, so toggling from the tray after initialization has no effect. So I decided to utilize electron json storage. Then I run:

npm install --save electron-json-storage

Then I require it in my main process:

const {storage} = require('electron-json-storage');

And then modified my contextMenu click listener:

contextMenu = Menu.buildFromTemplate([
    {label: 'Shadows', type: 'checkbox', checked: storage.get('shadows'), click: function(item){
      tray.setContextMenu(contextMenu);
      //mainWindow.webContents.send('toggleShadows', contextMenu.items[0].checked);
      storage.set('shadows', contextMenu.items[0].checked, function(error) {
          if (error) throw error;
      });
  mainWindow.webContents.reload();

}},

so that it tries to retrieve the currently set value from storage, then reloads the browser.

And in the renderer, I set it up to retrieve the set value as well:

Constants = namespace.Constants = {
    SHADOWS: storage.get('shadows'),
}

I can see the tray icon setting being toggled on and off, and the browser reloads, but no change is being registered to the shadows.

How do I make these changes from the tray icon persist?

I have occasionally run into an issue where my data will be corrupted in a way that appears to be due non-locked multi-process writes.

For example, the file will result like this:

{"foo": false,"bar": 42,"baz":12345}az":12345}
This usually happens when it seems like two or more separate electron render processes are writing different values to the same location.

Is there an way to guarantee mutex locking or a different access/write pattern that I should follow? Right now almost all of my reads/writes are using Electron's remote object API, so theoretically they're using the same object instance to read/write.

Thanks!

[edit] I found a few places in our code where we weren't using the same instance object and therefore I'm assuming that was responsible for the existing lock protection not correctly working. I'll close the issue when I can verify resolution, but would like to leave this issue up in case anyone else stumbles upon this error googling problems.

If I add a storage value with a key that contains URI encoded values (ex: "test:value"), if I do a 'storage.getAll', this value will not be extracted correctly.

Also, doing a storage.keys, the returned key will not be URI decoded, so I can not 'get' the stored data using that key without manually doing a decodeURIComponent on it.

getData(key) {
  let d;
  this.storage.get(key, (error, data) => {
    if (error) {
      err(`AutoComplete::getData(key) - Error for getting ${key}`);
    }
    d = data;
  });
  return d;
}

the content of storage file:

{}

But, an error occurred

Would you be open to a pull request that provides the ability to override the path where files are stored? I'm aware that you can use app.setPath('userData', 'newPath') to change the default userData path, but this can be inconvenient in certain cases (i.e. when you simply want to place the file in a subfolder).

I have a use case where I want to have essentially multiple instances of this library, because I want to store various json files in various locations. With the single instance I have to always set the data path immediately before running the command, but that's not perfect in case I have two async tasks running and I end up with a race condition error. I couldn't find anything in the documentation about creating new instances of this library, with each instance having its own data path.

Hi!

Whenever I attempt to use the storage.getDataPath/storage.setDataPath functions, I get a "TypeError: storage.setDataPath is not a function" error message.

I have the latest version of electron-json-storage and I can see the functions in my node_modules/electron-json-storage/storage.js file. Even when I copy the function out of the utils.js file and put it directly in storage.js I receive the same error.

I also tried installing the types/electron-json-storage, which seems to be out of date, and doesn't include these new functions.

Any advice?

Thanks!

This is a really useful library and I think it could be improved if there was support for promises. It would make the library easier to use, for example:

try {
    if (await storage.has(username)) {
      return await storage.get(username);
    }
    return false;
} catch(e) {
    return false;
}

https://github.com/ran-y/electron-storage is able to use inner folders when saving data - this way, you could have several electron apps in a computer, each using the same filenames (like 'settings.json') in each folder.

Both are great projects, though similar!

Right now JSON files are saved in plain text that anyone can open.

It would be great if there was a simple API of saving and retrieving obfuscated files.

callback is outdated, use promise instead.

bluebird, yaku, es6-promise, choose one.

Electron allows the change of 'userData'. I remember there used to be a bug in electron-json-storage where the 'userData' is read upon loading electron-json-storage lib which means any change to it made in the electron app will just get ignored. This bug was fixed then reappeared in 3.1.0. There might be ways to change 'userData' before electron-json-storage gets loaded but this change of behaviour could break existing code.

I keep getting an undefined from .get. JSON's are all there.

I just want to populate a element with the JSONs. Can't just filter out the file ext because it returns just the filename.

For some reason, Appveyor builds fail without any error, warning, nor output.

The tests pass on Windows PCs so we assume it's a problem with Appveyor, although I have other projects that run the exact test suite configuration with electron-mocha, and they run without any issues.

See https://ci.appveyor.com/project/jviotti/electron-json-storage/build/1.0.17 for an example. All we get is:

npm ERR! Test failed.  See above for more details.
Command exited with code 1

The mocha reporter doesn't even seem to print anything.

I'm running into a sporadic "Invalid data" error that I think is caused by multiple writes happening in quick succession. The stored json is invalid, with an extra object at the end of the file (after the end of an array).

My guess is that it's caused by fs.writeFile, because it's asynchronous and can cause issues like these: nodejs/node-v0.x-archive#4965

I don't have a good test case at the moment, but I'll try to put something together if I get some time later.

I'm getting some really strange behaviour here when I try to emit some data to the window once I've grabbed some data out of the storage.

I'm trying to send the stored data to the browser, but for brevity, here's a test scenario.

const storage = require('electron-json-storage');
mainWindow = new BrowserWindow({width: 500, height: 500});

storage.get('config', function(err, config){
   // Emitting inside here won't get to the main window
   mainWindow.webContents.send('test', {foo: 'bar'});
});

// But emitting here *will* go through to the main window
mainWindow.webContents.send('test', {foo: 'bar'});

The mainWindow.webContents.send() function returns true regardless of whether it's run from inside the storage closure or anywhere else, however it only gets to the browser when called outside the storage.get() closure.

This problem only seems to be occurring inside callbacks that use this package.

Hello, wouldn't it be nice to be able to merge a setting/object in?

Need to encode special characters such as ? : before using key in file name.

Digging into my Resources in the app, the dependencies are installed, but this app can't find them. I'm using it with remote, wondering if that has anything to do with it. It seems however to be an issue with bluebird.

I tried manually including all of the dependencies:

https://github.com/jviotti/electron-json-storage/blob/master/lib/storage.js#L227
This line is a potential disaster, it assumes only electron-json-storage can create json files in user data folder root. I suggest creating a subfolder.

In my main/index.js, storage.get works on 'selectedProfile', but in my upgrades.js used in index.html it returns [Object object]

Hi, great library!

Having app.getPath('userData') like this: .config/Electron Boilerplate (development)
results in ENOENT:

{ Error: ENOENT: no such file or directory, open '.config/Electron Boilerplate/data.json'
    at Error (native)
  errno: -2,
  code: 'ENOENT',
  syscall: 'open',
  path: '.config/Electron Boilerplate/data.json' }

Hi!
Is data written atomically to the disc?

I'm using electron-json-storage as a simple database and it's important that in case of some kind of crash data will be in one of the well known states.

Thanks!

I need to save user preferences when the checkbox is used. I found that after I press the checkbox 1 -4 times I get an error in console with each click "Uncaught Error: Invalid data"

I assume this has something to do with FS or async. If I stop clicking and wait 5 - 10 seconds, then I find that sometimes it will work correctly and sometimes it will not. Any suggestions for an improved implementation?

$('#KeySwitch').click(function() {
      storage.set('keySwitch', { keySwitch:this.checked })
      keySwitchB = this.checked
      console.log(keySwitchB);
      storage.get('keySwitch',function(error, data){
        if (error) throw error;
        console.log(data.keySwitch);
      })
    });

<input type="checkbox" id="KeySwitch" class="mdl-checkbox__input" checked />

I am new to this storage tool. It seems that this tool wants to create a new file for each object I add to the "db".

Database setup:

/**
 * Database setup
 */
const db = require('electron-json-storage');

const userDataPath = (electron.app || electron.remote.app).getPath('userData');
db.setDataPath(userDataPath+"/database");

console.log(db.getDataPath());

Logged:
/Users/Legend/Library/Application Support/budgeteer/database

Usage:

    var newID = 1;
    var newObject= {
        "id": newID,
        "category": "Giving",
        "lineItems": [
            {
                "name": "Tithe",
                "amount": 10,
                "date": "04/11/2017",
                "recurring": "weekly"
            }
        ]
    };

    var newID2 = 2;
    var newObject2 = {
        "id": newID2,
        "category": "Giving",
        "lineItems": [
            {
                "name": "Offerings",
                "amount": 10,
                "date": "04/15/2017",
                "recurring": "weekly"
            }
        ]
    };

    db.set(newID, newObject, function(){});
    db.set(newID2, newObject2, function(){});
    db.remove(newID, function(){});

    let listItems = db.getAll(function(error, data) {});

What I get (note that also the remove() apparently didn't happen - the file is still there and still has it's contents):

I am not sure what I am doing wrong here.

JSON.parse can take a second argument to revive dates, would it be possible to allow for that arg to be passed through?

Looks like my Spectron tests are failing due to this line of code on this line:

const app = electron.app || electron.remote.app;

I'm going to propose moving that line into the only place that it is used, with a conditional that it will use a default location if app does not exist. So getUserDataPath will look like this:

exports.getUserDataPath = function() {
  const app = electron.app || (electron.remote && electron.remote.app);
  if(app) {
    return path.join(app.getPath('userData'), 'storage');
  } else {
    return '/tmp/storage';
  }
};

This is not a breaking change.

@jviotti

Can you add to storage.get new features
array of keys (to get multiply jsons at once)
null(to get all storage)

Are there any store size limitations imposed by this library?

I want to store a list variable called notebookList {notebookOne,notebookTwo,notebookThree}

How do I modify this code:

const storage = require('electron-json-storage');

storage.set('foobar', { foo: 'bar' }, function(error) {
  if (error) throw error;
});

Also, do I insert it anywhere in the index.html file?

Please forgive me as I'm a total noob when it comes to javascript and node.

I'm using this module in an AngularJS (1.53) controller for an Electron (0.37.5) application and trying the following to apply the contents of an apikey.json file to a scope variable.

    $scope.apikey;
    storage.get('apikey', function(error, data) {
        console.log('get the api key : ' + data.key);
        $scope.apikey = data.key;
    });
   console.log('this will be undefined...' + $scope.apikey);

Is there some way to apply the data from the json file to a variable (or to the $scope in AngularJS)?

It works on the first console log "get the api key" but fails on the second one. Do I need to just attach to a global variable on the window object?

Javascript's scope issues always have been baffling to me. I'm primarily a PHP user.

I am getting this error when trying to import the library in electron main process. Same with importing or requiring. Not sure what else I can do to help specifying the issue.

... gutil.PluginError('webpack', buildError)

Error: electron/app/main.js from UglifyJs
Unexpected token: name (args) [./~/cb2promise/index.js:7,0]

I'm having trouble getting the library working in the renderer process. I'm importing in the main process similar to the examples and everything works fine, but if I import in the renderer process I get the following error:

Cannot find module 'electron' from '/.../node_modules/electron-json-storage/lib'

I've tried using import and require, both are giving the same error. I've also tried using different versions of electron-prebuilt, but that doesn't seem to be working either.

Not sure if I'm missing a dependency, or if I'm missing a key step. Any help would be appreciated.

Are you considering promises support?

Using getAll is causing fatal error:

Invalid data

Problem is I have 260 json objects an no idea which to delete or clean.

Also I see .DS_Store is being returned as a key, i wonder if this is causing problems.

I have an electron app, and for some reason, the storage is being deleted when I open the app again a few days later.

Is that common? I don't know how to provide more info about the problem. Let me know what I can send to explain more.

I'm using electron-json-storage in an Electron app. It works when I run the local dev build with npm run dev, but as soon as I try make a production build, UglifyJs gives me the following error message for each time I import electron-json-storage:

ERROR in renderer.js from UglifyJs
Unexpected token: name (args)

If the storage/ directory does not exist inside userData, then getAll() returns an error. The storage directory is completely private to electron-json-storage. If it doesn't exist, then please create it, or at least don't throw an error. Just return an empty object.

Our application sets storage values just before quitting. We need a way to make sure all the set operations are complete before we allow the app to proceed. The easiest way for us would be a synchronous call like waitForDone(). If there are no operations it just returns.

(Or, does electron-json-storage guarantee that state already exists before the app quits?)

Hi
What about setMany() method
For example
array['key1'] = json;
array['key2'] = json;
array['key3'] = json;
storage.setMany(array,function(result){});

is there a way to store real JSON format?

why are you using JSON.stringify anyway? I assumed when the package name is electron-json-storage that it will store real json files... even the file extension name is '.json' but the content is in string :/

Hi i have been using a hierarchy in key naming and want to get all keys matching a prefix. I feel like this could be useful. Should I pr?