Code Monkey home page Code Monkey logo

feathers-seeder's Introduction

feathers-seeder

npm build status

Straightforward data seeder for FeathersJS services.

About

FeathersJS is a dynamic application framework that makes it easy to prototype secure real-time Node applications.

It has uniform support for a wide variety of database options, both persistent and in-memory. However, it can often be a pain to bootstrap databases, especially for secure backends where creation permissions are locked down tightly.

feathers-seeder attempts to solve that problem by allowing you to fill your database (specifically feathers-memory) with similar of identical data every time your application is launched.

This can really be useful for projects using feathers-memory, or feathers-nedb to test applications.

Installation

These magic words will do the trick:

npm install --save feathers-seeder

Usage

  1. Configure the seeder.
  2. Call app.seed(). This will return a Promise.

Example:

const app = feathers().configure(hooks());
app
  .configure(seeder(app.get('seeder')))
  .seed()
  .then(() => {
    app.listen(3000);
  });

Configuration

feathers-seeder should be called as a function, with a single configuration object as a parameter.

To enable debug output logging, set the DEBUG env variable accordingly (e.g. DEBUG=*)

All data in the service will be wiped before seeding, unless delete is set to false. It is true by default.

To disable feathers-seeder (i.e. at production), you can simply set the value disabled: true on your configuration object. It is false by default.

You can pass service parameters to each service via a params object. This can be useful for projects that have services locked down via hooks.

This configuration object should contain an array called services. Each object in this array will have a key pointing to the path of a registered Feathers service, and its value will be a configuration detail as follows.

Example:

{
  delete: true,
  disabled: false,
  params: { provider: 'rest' },
  services: [
    {
      path: 'users',
      template: {
        text: "Hello, world!  I am {{name.firstName}}."
      }
    }
  ]
}

Configuration options:

  • count: Number - The number of times to generate objects. If you provide a template or template(s), then count objects adhering to the template(s) will be generated. Default = 1.

  • delete: Boolean - If set to true, then existing data for this service will be deleted before seeding. Overrides global delete setting.

  • disabled: Boolean - Disables seeding for this service.

  • params: Object - Additional parameters to pass to service methods. This is merged with (and supersedes) the global params configuration via Object.assign.

  • path: String - The path to the service to be seeded.

  • randomize: Boolean - (default true) - The seeder will pick a random template to generate the item for the service.

  • template: Object - A template object defining the structure of each object generated. For dynamic data, you can provide:

    • Template strings, as feathers-seeder uses @marak/Faker.js internally
    • Custom parameterless functions

    Example:

    {
      template: {
        username: "{{internet.userName}}",
        password: "{{internet.password}}"
        name: "{{name.firstName}} {{name.lastName}}",
        email: "{{internet.email}}",
        lastLogin: () => moment().subtract(7, 'days').format()
      }
    }
  • templates: Object[] - An array of templates. If the option randomize is true, each time an object is to be generated, a random template will be chosen. Otherwise, all templates will be generated for the service.

  • callback: Function(obj, cb) - You can register a callback each time a database record is created. This allows you to seed nested services. :) Callbacks MUST return a Promise.

{
  services: [
    {
      count: 100,
      path: 'users',
      template: {
        name: '{{name.firstName}} {{name.lastName}}'
      },

      callback(user, seed) {
        console.info(`Happy birthday to ${user.name}!`);

        // Call `seed` with regular service configuration.
        // This will return a Promise.

        return seed({
          path: 'users/:userId/posts',
          params: {
            userId: user._id
          },
          template: {
            text: "It's my birthday! :)"
          }
        });
      }
    }
  ]
}

Example

import feathers from 'feathers';
import hooks from 'feathers-hooks';
import memory from 'feathers-memory';
import seeder from 'feathers-seeder';

const options = {
  services: [
    {
      path: 'users',
      count: 10,
      template: {
        name: '{{name.firstName}} {{name.lastName}}'
      }
    }
  ]
};

const app = feathers()
              .use('/users', memory())
              .configure(seeder(options));

app.seed().then(() => {
  app.service('users').find().then(users => console.log(users));
});

Thanks

Thank you for using feathers-seeder. If you find any bugs, feel free to report an issue.

Follow me on Twitter: @thosakwe

Or check out my blog.

feathers-seeder's People

Contributors

codingfriend1 avatar erkkaha avatar jamesholcomb avatar josx avatar thosakwe avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

feathers-seeder's Issues

MIT License

Would you be willing to make the feathers-seeder library MIT license?

How to handle several dependent seeds

So, I have this scenario (using Postgres/sequelize):
1 insert predefined users
2 insert predefined roles
3 insert predefined users-roles
I'm stuck at the 3rd step, it requires users and roles to exist so that it can insert correctly.

if I use a callback after each template, the seeder tries to execute each following template the same number of times, but if I don't use a callback, then each service is executed asynchronously and it doesn't work because users and/or roles do not exist yet.

How can I do this?

Thanks!

configuration file:

module.exports = {
	services: [
		{
			delete: true,
			randomize: false,
			path: 'users',
			templates: [
				{
					user_id: '22d5362c-efdc-11e6-bc64-92361f002671',
					username: 'sss',
					email: '[email protected]',
					password: '1234'
				}
			],
			callback(user, seed) {
				return seed({
					delete: true,
					randomize: false,
					path: 'roles',
					templates: [
						{
							role_id: '5abec6e0-efda-11e6-bfc0-4979bfeeb47e',
							name: 'admin'
						},
						{
							role_id: '79e7c441-efda-11e6-a9dc-5364a8fb9f30',
							name: 'user'
						},
						{
							role_id: 'a18da632-efdb-11e6-a946-afdca6ae2a45',
							name: 'guest',
							status: 'inactive'
						}
					],
					callback(role, seed) {
						return seed({
							delete: true,
							randomize: false,
							path: 'user_roles',
							templates: [
								{
									user_id: '22d5362c-efdc-11e6-bc64-92361f002671',
									role_id: '5abec6e0-efda-11e6-bfc0-4979bfeeb47e'
								}
							]
						});
					}
				});
			}
		}
	]
};

Seeder works on "services" and not "models"

Seeders should work on models for a variety of reasons.

  • It is impossible to seed the users table. This is because that service is protected by feathers-authentication.... but there are no users yet. you have to comment out authentication for this to work.
  • I may want to populate tables that I don't want to expose via a service. For instance, users have clients. I do not intend to create a /clients route, but only access clients via /users/:id/clients. How do I create this seed? Do I have to have one seed for /users/1/clients and another for /users/2/clients?

Error on postinstall (Unknown plugin "add-module-exports")

Hello,

I am not able to install feathers-seed, here is the log:

> [email protected] postinstall /Users/.../backend/node_modules/feathers-seeder
> npm run compile

> [email protected] compile /Users/.../backend/node_modules/feathers-seeder
> rimraf lib/ && babel -d lib/ src/

ReferenceError: Unknown plugin "add-module-exports" specified in "/Users/.../backend/node_modules/feathers-seeder/.babelrc" at 0, attempted to resolve relative to "/Users/.../backend/node_modules/feathers-seeder"
    at /usr/local/lib/node_modules/babel-cli/node_modules/babel-core/lib/transformation/file/options/option-manager.js:177:17
    at Array.map (native)
    at Function.normalisePlugins (/usr/local/lib/node_modules/babel-cli/node_modules/babel-core/lib/transformation/file/options/option-manager.js:153:20)
    at OptionManager.mergeOptions (/usr/local/lib/node_modules/babel-cli/node_modules/babel-core/lib/transformation/file/options/option-manager.js:245:36)
    at OptionManager.init (/usr/local/lib/node_modules/babel-cli/node_modules/babel-core/lib/transformation/file/options/option-manager.js:383:12)
    at File.initOptions (/usr/local/lib/node_modules/babel-cli/node_modules/babel-core/lib/transformation/file/index.js:223:65)
    at new File (/usr/local/lib/node_modules/babel-cli/node_modules/babel-core/lib/transformation/file/index.js:140:24)
    at Pipeline.transform (/usr/local/lib/node_modules/babel-cli/node_modules/babel-core/lib/transformation/pipeline.js:46:16)
    at transform (/usr/local/lib/node_modules/babel-cli/lib/babel/util.js:52:22)
    at Object.compile (/usr/local/lib/node_modules/babel-cli/lib/babel/util.js:61:12)

Thanks in advance !

bug: number types in templates are empty

Template:

      const services = [{
        path: 'dummy',
        template: {
          name: '{{name.firstName}} {{name.lastName}}',
          age: 99
        }
      }];

produces

Seeding app...
Params seeding 'dummy': {}
Creating 1 instance(s)
Running 1 seeder(s)...
Deleted from 'dummy:' []
About to compile this template: { name: '{{name.firstName}} {{name.lastName}}', age: 99 }
Populating name from this value: {{name.firstName}} {{name.lastName}}
This is a string.
Populating age from this value: 99
This is a number
About to compile this template: 99
Compiled template: { name: 'Jaden Abshire', age: {} }
Created: { name: 'Jaden Abshire', age: {}, id: 0 }
Created 1 total items: [ [ { name: 'Jaden Abshire', age: {}, id: 0 } ] ]
1 dummy: { name: 'Jaden Abshire', age: {}, id: 0 }

support faker locale option

hi , I want to use custom locale with the faker but it is not pussible to do this . can you add option to change locale in service options level or seeder options level . and when there is no config for locale default back to english ?

thank you

Seeder not working with nuxt-js project

I have seeder setup with markets service:

{
      path: 'markets',
      count: 2,
      template: {
        name: '{{lorem.words(3)}}',
        slug: '{{lorem.words(1)}}'
      }
}

With result:

info: after: markets - Method: remove
info: after: markets - Method: create
info: after: markets - Method: create

But I see nothing show up in database (postgres). However, if I use REST api, I get it in the database.

info: after: markets - Method: create

App is otherwise working fine. Example:

Getting dates in ISO format

Is there any way to get a date such as {{date.future}} converted into ISO format? It seems when I use that I get a Date.toString() version of it such as Wed Jul 25 2018 18:42:57 GMT-0400 (Eastern Daylight Time) instead of ISO like 2018-01-27T11:59:18.201Z (these dates aren't equivalent, just examples). I've tried creating a new Date object from the string and using toISOString() but it's giving me errors. I'm probably just not doing it right.

The reason I'm asking is that I'm using Moment to display the dates and I'm getting a warning now about dates and I'd like to clear it up.

From Moment

Deprecation warning: value provided is not in a recognized RFC2822 or ISO format. moment construction falls back to js Date(), which is not reliable across all browsers and versions. Non RFC2822/ISO date formats are discouraged and will be removed in an upcoming major release.

Any help would be appreciated. Thanks.

question: creating users from predefined data

I have the need to load a set of static user data from a predefined JSON file (they are actual users from an external OAuth provider), rather than generating from a template.

The configuration allows an array of templates that are selected randomly, but that won't guarantee all users are loaded.

What do you recommend?

GeneralError: Cannot read property 'remove' of undefined

I am getting:

info: Feathers application started on http://localhost:8089
error: Unhandled Rejection at: Promise  Promise {
  <rejected> { GeneralError: Cannot read property 'remove' of undefined
    at new GeneralError (/home/usr/api_server/node_modules/feathers-errors/lib/index.js:177:17)
    at /home/usr/api_server/node_modules/feathers-seeder/lib/index.js:45:15
    at <anonymous>
    at runMicrotasksCallback (internal/process/next_tick.js:121:5)
    at _combinedTickCallback (internal/process/next_tick.js:131:7)
    at process._tickCallback (internal/process/next_tick.js:180:9)
    at Function.Module.runMain (module.js:684:11)
    at startup (bootstrap_node.js:191:16)
    at bootstrap_node.js:613:3
  type: 'FeathersError',
  name: 'GeneralError',
  message: 'Cannot read property \'remove\' of undefined',
  code: 500,
  className: 'general-error',
  data: undefined,
  errors: {} } } GeneralError: Cannot read property 'remove' of undefined
    at new GeneralError (/home/usr/api_server/node_modules/feathers-errors/lib/index.js:177:17)
    at /home/usr/api_server/node_modules/feathers-seeder/lib/index.js:45:15
    at <anonymous>
    at runMicrotasksCallback (internal/process/next_tick.js:121:5)
    at _combinedTickCallback (internal/process/next_tick.js:131:7)
    at process._tickCallback (internal/process/next_tick.js:180:9)
    at Function.Module.runMain (module.js:684:11)
    at startup (bootstrap_node.js:191:16)
    at bootstrap_node.js:613:3

my config:

const moment = require('moment');

module.exports = {
  count: 10,
  // delete: true,
  disabled: false,
  params: {seeder: 'true'},
  services: [
    {
      path: 'users.service.js',
      template: {
        name: '{{name.firstName}} {{name.lastName}}',
        password: '{{internet.password}}',
        username: '{{internet.userName}}',
        email: '{{internet.email}}',
        created_at: () => moment().subtract(7, 'days').format()
      }
    }
  ]
};

enabling https

Hi,
great package! Is there an easy way to enable all the URLs to use https instead?

Thanks!

1.0.8 not working anymore

Because of the latest change in the package.json (ffdaee9#diff-b9cfc7f2cdf78a7f4b91a753d10865a2L5) i get

Error: Cannot find module 'feathers-seeder'
    at Function.Module._resolveFilename (module.js:470:15)
    at Function.Module._load (module.js:418:25)
    at Module.require (module.js:498:17)
    at require (internal/module.js:20:19)

However when reverting the change(renaming back to .js) it works. I'm using latest nodeJS and intellij via:
"C:\Program Files\nodejs\node.exe" C:\Users\xxxx\AppData\Roaming\npm\node_modules\nodemon\bin\nodemon.js -x "node --harmony-async-await" --debug=3001 D:\yyyyyyy\src\index.js

I think it was just a typo, or there is something special about .jsnpm file format that i wasn't aware of :P

Low Severe Vulnerabilities Found

I get the following output when running npm audit:

 === npm audit security report ===


                                 Manual Review
             Some vulnerabilities require your attention to resolve

          Visit https://go.npm.me/audit-guide for additional guidance


  Low             Regular Expression Denial of Service

  Package         braces

  Patched in      >=2.3.1

  Dependency of   feathers-seeder

  Path            feathers-seeder > @feathersjs/cli > @feathersjs/tools >
                  @feathersjs/jscodeshift > micromatch > braces

  More info       https://nodesecurity.io/advisories/786


  Low             Regular Expression Denial of Service

  Package         braces

  Patched in      >=2.3.1

  Dependency of   feathers-seeder

  Path            feathers-seeder > @feathersjs/cli > generator-feathers >
                  @feathersjs/jscodeshift > micromatch > braces

  More info       https://nodesecurity.io/advisories/786


  Low             Regular Expression Denial of Service

  Package         braces

  Patched in      >=2.3.1

  Dependency of   feathers-seeder

  Path            feathers-seeder > @feathersjs/cli > generator-feathers >
                  @feathersjs/tools > @feathersjs/jscodeshift > micromatch >
                  braces

  More info       https://nodesecurity.io/advisories/786

found 3 low severity vulnerabilities in 877366 scanned packages
  3 vulnerabilities require manual review. See the full report for details.

Kindly update the affected package.

Reset autoincrement value for sequelize-adapter

The primary field auto increment value is not resetted on delete operation, thus when seeding the app the first time and creating 1 user he will have the id 1. After restarting node.js and seeding again, the database is clean but the id of the user is 2.

Currently with the callback one can store the ids of entities to dynamically set the correct relationships in later services(e.g. if i want to relate an item to the first user i created i need to know the userId). But to make life simpler, it would be great if the auto increment value is resetted, because one can then easier prototype the config, without having to make sure to not use hard-coded ids.

executing order of service array

Another question/possible improvement.

Currently it seems that the execution order of service array is not guaranteed to be like stated in the config file right?

e.g. i have the following pseudocode:

service: [
    {path: 'users',template:{...}},
    {path: 'panels',template:{...}},
    {path: 'studies',template:{...}}
]

It seems that he is first inserting the panels, then the studies and presumbly the users at last. Could it be alphabetical ordered? Anyway i first need users to be created, otherwise my relation constraints(sequelize-adapter) are not fullfiled.

Any idea why this is? Or how one can fix that? It would be totally fine if he is just respecting the order of the array.

ResourceRequest timed out

When I increase the number of 'users' I seed I get:

{ GeneralError: ResourceRequest timed out
    at new GeneralError (/home/usr/api_server/node_modules/feathers-errors/lib/index.js:177:17)
    at /home/usr/api_server/node_modules/feathers-seeder/lib/index.js:45:15
  type: 'FeathersError',
  name: 'GeneralError',
  message: 'ResourceRequest timed out',
  code: 500,
  className: 'general-error',
  data: undefined,
  errors: {} } +0ms

With just 10 users it gets through without error, but some more and the error will come.

Import data 1:1 without randomization

Hey, great project!

Given the following dataset:

export default
  [
      { 'name': '50l bag' },
      { 'name': 'holdall' },
      { 'name': 'bumbag (fannypack)' },
    ]

It would be great to be able to import this data 1:1 with the original dataset - no randomization or duplication. Any ideas how I could accomplish this?

So far I've tried

import data from './data/items'

let options = {
  path: 'items',
  count: data.length,
  template: [
    ...data
  ]
}

module.exports = options

But obvious problems are obvious xD

Cheers @thosakwe!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.