stellarfw / stellar Goto Github PK

To improve a little more the code splitting we could support folders on the actions folder. This will allows us to reduce the number of LoC in a single file. The follow snippet shows one example:

\actions
  - \repair
    - basicActions.js
    - getRepairsByState.js
    - getRepairsByClient.js
  - client.js
  - (...)

We are checking if the action returns an Promise, but the Waterline doesn't use the default Promise, so we need to search for a .then methods instead.

We must add support to run Stellar as a daemon, this is ideal for run it on servers. The command must be like:

stellar run --daemon

The follow validator fails when the field value is empty required_if:contactPreference,email|email.

Supposing that the contactPreference field is equals to sms and the field under validation is equals to "" the following validators must not be executed.

The modules can have satellites, this satellites can be used to replace or add new features to the Stellar instance. So it's necessary gather all the satellites before start loading them, in order to take advantage of the priority system to load the things at the correct time.

Steps to Reproduce Your Error

On a new empty module run a command to make a model with crud operations, like the following example:

stellar makeModel contact --module=contacts --crud

What is Expected?

The model should be created as well the action file.

What is Actually Happening?

The

Success: The "contact" model was created!

Error: ENOENT: no such file or directory, open '/Users/gil0mendes/Documents/projects/tecniphone/api/modules/contacts/actions/contact.js'
    at Object.fs.openSync (fs.js:557:18)
    at Object.fs.writeFileSync (fs.js:1222:33)
    at Function.createFile (/Users/gil0mendes/Documents/projects/stellar/core/bin/utils.js:111:49)
    at Function.generateFileFromTemplate (/Users/gil0mendes/Documents/projects/stellar/core/bin/utils.js:180:11)
    at MakeModel.execute (/Users/gil0mendes/Documents/projects/stellar/core/bin/commands/makeModel.js:80:13)
    at Commander.module.exports.args (/Users/gil0mendes/Documents/projects/stellar/core/bin/commands/makeModel.js:119:48)
    at Commander.parse (/Users/gil0mendes/Documents/projects/stellar/core/bin/commander.js:244:62)
    at Object.<anonymous> (/Users/gil0mendes/Documents/projects/stellar/core/bin/stellar:74:11)
    at Module._compile (module.js:571:32)
    at Object.Module._extensions..js (module.js:580:10)

When the configs changes the running instance breaks and does not reload. No exception or other error information is shown.

Now the run command only has the --clear option, when the user runs stellar with this options all the NPM modules will be removed and the installed again. But this isn't always necessary, sometimes we just want updated the existent packages or add a new one.

So, if the user runs stellar with the --update option the follows steps must occur:

• the package.json file must be removed, if present;
• a new package.json file must be generated;
• the npm update command must be executed.

The ideia is create a mechanism who allows developers manipulate meta data from the other modules. For example, if I enable the Identify module on my application the auth.register action is public by default, but I need that action be only called by admins. And there is no way to manipulate that without rewrite the action or patch the action manually.

This need more discussion and usage examples.

Add confirmation when generating actions that already exist in some module

Running System

• Stellar Version: 1.0.4
• Node.js Version: 7.1.0
• Operating System: macOS Sierra

Steps to Reproduce Your Error

Change a model when the Stellar is running.

What is Expected?

The model should reload and the Stellar instance must be restarted.

What is Actually Happening?

An error is thrown:

2016-12-22T17:01:41.489Z - alert: Stellar has been stopped
2016-12-22T17:01:41.490Z - notice: pid: 32149
2016-12-22T17:01:41.491Z - notice: server ID: 10.8.1.3
2016-12-22T17:01:41.491Z - notice: environment: development
(node:32149) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 2): Error: Cannot find module '/Users/gil0mendes/Documents/projects/tecniphone/api/node_modules/[object Object]'

Improve the identification of options and add an usage example in each available command!

Hmm... I need to think more about it!

• Node-LetsEncrypt

In the future we must add support to the new Fetch API who will replace the old XMLHttpRequest.

Some of the advantages is its supports all HTTP verbs and integrates with Service Workers, including Request/Response caching.

change exports.core.models ... to exports.core_models

The modules also can have configuration files to change the default behaviour of the Stellar, or to configure the features they implements.

The config satellite needs to be modified to load the configs folder of each enabled module, if exists. This action must be performed on the satellite load stage.

When the developer run the follow command stellar makeModel comment --module=exampleModule --crud, the Stellar will generate a model file and a set of actions. To make this more powerful we must add support to another options --rest to add REST like URLs on the actions declarations.

The follow text describe the format for the REST URLs:

// get all resources
[GET] /resource

// get one resource 
[GET] /resource/:id

// create a resource
[POST] /resource

// edit a resource
[PUT] / resource/:id

// delete a resource
[DELETE] /resource/:id

Reference a model when generating a action via command line to fill automatically the inputs array with the properties on the model.

After finish some pendant features, I will starting creating an website to expose the documentation, examples and a full guide of how to use the Stellar.

Some modules depends on other modules, this dependencies can be declared in the manifest.json like the follow:

{
  "dependencies": {
    "identify": "*"
  }
}

This dependencies property must be validated before the module loading, a message must be presented to the user if there is some problem.

This is a temporary feature, one time in the future we will be able to install dependencies automatically.

The listeners must be reloaded when they are changed.

The stellar command colorizes terminal output but it does not restore the terminal's default color. This means the color may persist even after the stellar command terminates. Depending on your terminal and settings, this may not be noticeable - for example, if you have your shell prompt configured to output colors, it will immediately overwrite whatever color stellar set the terminal to. But if your shell prompt does not output colors, the result will look like this:

For now the validator errors are a bit poor. The idea is the error messages should include an explanation about why the error occur.

For example, in case of the validator are alpha_num, the error message should be something like: The input field (${inputName}) must contains only letters or numbers

This feature allows apply the same configurations across multiple actions without rewrite the same code in all actions.

exports.default = {
  group,
  middleware: [ 'exampleMiddleware' ],

  actions: { ... }
}

The parameters must have a description property to be added to the automatic documentation.

{
  inputs: {
    name: {
      description: 'This is the real name of the user.',
      required: true
    }
  }
}

I really need help with this 😂

The English and Portuguese documentation mentions a stellar generateEvent command but the correct name must be makeListener, but event that command does not exists.

Running System

• Stellar Version: dev
• Node.js Version: 7.1.0
• Operating System: macOS Sierra

Problem

Some packages need to be upgraded to remove some deprecation warnings that we get in version 7.* of Node.js. I don't investigate yet, but the origin of this maybe come from the servers.

(node:60353) DeprecationWarning: os.tmpDir() is deprecated. Use os.tmpdir() instead.
(node:60353) DeprecationWarning: Using Buffer without `new` will soon stop working. Use `new Buffer()`, or preferably `Buffer.from()`, `Buffer.allocUnsafe()` or `Buffer.alloc()` instead.

Stellar must have an event system to allow subscribe and listen for events in the application. This is useful to manipulate data during the execution or to extend functionalities adding new behaviours to existing logic. The listeners must be stored in <moduleName>/listeners.

Generating Listener

Of course, manually creating the files for each listener is cumbersome. Instead, developer must be able to use a command to generate the listens automatically:

stellar generateEvent <eventName> --module=<moduleName>

Defining a Listener

The code bellow show a dummy listener implementation, in this example the listener will respond to the social.newComment event and will enqueu a task to send a new email for each command made in the application.

// File: social/listeners/comments.js

'use strict'

exports.default = [{
  event: 'social.newComment',
  run: (api, params, next) {
    // send a new email notifying the new command
    api.tasks.enqueue('sendNewCommentEmail', params)

    // create a new property named `emailSent` and set it to `true`
    params.emailSent = true

    // next(error <Error>)
    next()
  }
}]

Firing events

The follow code show how to fire an event. In this case the developer want fire the social.newComment event and give to the listeners a variable with the new comment's data.

api.events.fire('social.newComment', newComment, response => {
  // do something with the modified data...
})

Register Listeners Manually

To register listeners manually the developer can use the follow api:

api.events.listener('blog.newUser', (api, params, next) => {
  // do something...

  // finish the listener execution
  next()
})

The models system make use of the Mongoose, this makes my life easier 😉
But we need to implement some wrappers to improve the developer experience and reduce the number of code how is necessary to perform basic operations like: create, retrieve, update or delete.

// create
api.models.create('modelName', data, (err, model) => {
  next()
})

// get all 
api.models.findAll('modelName', (err, models) => {
  next()
})

// find
api.models.find('modelName', query, (err, models) => {
  next()
})

// find by ID
api.models.findById('modelName', id, (err, model) => {
  next()
})

// update
api.models.updateById('modelName', id, data, (err, model) => {
  next()
})

// delete
api.models.deleteById('modelName', id, err => {
  next()
})

A new command must be develop to generate a task structure.

Syntax

stellar makeTask <taskName> --module=<moduleName>

Generated Structure

'use strict'

exports.%taskName% = {
    name: '%taskName%',
    description: 'Change this to describe your task',

    queue: 'default',
    frequency: 1000,

    run: (api, param, next) => {
        // do stuff...

        next()
    }
}

Once the actions contain a good description of their behavior we can extract: the name, description, input declarations and restrictions to generate documentation 100% automatically without human interaction 😃

The output format must be HTML and the command for that should be stellar genDoc.

When a user initialize a folder with the Stellar project structure, Stellar must create a .gitignore file to keep out the temp/ folder and the package.json file out of the git track changes.

A new command named console must be created to allow developers connect with Stellar and run command using REPL.

Attention: this isn't to connect with a existing instance.

Command

stellar console

The ideia is list the active tasks on the auto generated documentation. Tasks have some interesting properties (like, name, frequency) to create useful information for the developers.

The generated files must use namespaces to reduce the probability of collision.

Implement a after validator like the before one. This is useful to apply restrictions to a date parameter.

Example

{
  // (...)
  inputs: {
    date: {
      validator: 'after:2016-11-22'
    }
  }
  // (...)
}

The Stellar should provide a series of methods and mechanisms to improve and accelerate the writing and execution of unit tests.

This need some planning and discussion!

For some reason, or when we are developing the API locally the Internet connection can go down and the Mongoose will throw an error. This errors must be caught and the connection must be reestablished.

/Users/gil0mendes/Documents/projects/stellar/core/node_modules/mongoose/lib/drivers/node-mongodb-native/connection.js:168
    var error = new Error(err && err.err || 'connection timeout');
                ^
Error: connection timeout
    at Db.<anonymous> (/Users/gil0mendes/Documents/projects/stellar/core/node_modules/mongoose/lib/drivers/node-mongodb-native/connection.js:168:17)
    at emitTwo (events.js:106:13)
    at Db.emit (events.js:191:7)
    at Server.listener (/Users/gil0mendes/Documents/projects/stellar/core/node_modules/mongodb/lib/db.js:1791:14)
    at emitOne (events.js:96:13)
    at Server.emit (events.js:188:7)
    at Server.<anonymous> (/Users/gil0mendes/Documents/projects/stellar/core/node_modules/mongodb/lib/server.js:270:14)
    at Server.g (events.js:291:16)
    at emitOne (events.js:96:13)
    at Server.emit (events.js:188:7)
    at Pool.<anonymous> (/Users/gil0mendes/Documents/projects/stellar/core/node_modules/mongodb-core/lib/topologies/server.js:322:12)
    at emitOne (events.js:96:13)
    at Pool.emit (events.js:188:7)
    at Connection.<anonymous> (/Users/gil0mendes/Documents/projects/stellar/core/node_modules/mongodb-core/lib/connection/pool.js:260:12)
    at Connection.g (events.js:291:16)
    at emitTwo (events.js:106:13)

With the implementation for Waterline as our default model system we add amazing features, for example: simplification on the model declaration, support for multiple types of databases, the creation of new a record is easier than before, and so on. But we also lose some features as the inner population or populate after a method or have a method to update just one record and get just one object as the returned value, and not an array.

So we need to make a fork of the current model system (Waterline) to address this issues and implement this required features that all of us need. This will be a long journey and we will need all the help you can give in order to move forward.

For now we will discuses and describe the features as better as possible, after that we will create a issue for each feature to better track the work we need to do and what we have done.

Please, comment and give some suggestion to append to the list below.

List of features to implement

• populate after an update
• whenever the first parameter of update method is an ID (string or number) the return must be the updated object and not an array

For Mongo

• a way to define a $text index (balderdashy/sails-mongo#357)

The utility functions must be moved to a satellite to be access from the API object

A new option to the init command must be created in order to allow generate a Dockerfile on the project creation. This will make possible run Stellar projects in a container. Who is cool 😉

Shouldn't /dist be added to .gitignore file ?

The developer should be able execute the stellar init command on an empty directory in order to generate the skeleton of a Stellar project.

Sometimes Stellar can't connect with the Redis server, I don't have no ideia why!!!

2016-05-27T13:28:39.209Z - debug:  > start: redis
2016-05-27T13:28:39.258Z - debug: connected to redis (subscriber)
2016-05-27T13:28:39.359Z - debug: connected to redis (client)

Now that I use Stellar in production products, I see that I need more complex models than simple ones, so I will make it a default:

Current models syntax

exports.default = {
  // --- example
  title: String,
  content: String
  // ---
}

Default syntax on the next release

exports.default = (api, mongoose) => {
  // get Mongoose Schema type
  const Schema = mongoose.Schema

  // create a new Schema for the repair resource
  const newSchema = new Schema({
  // --- example
    title: String,
    content: String,
    user: { type: Schema.Types.ObjectId, ref: 'user' },
  // ---
  }, { timestamps: true })

  // return the created schema
  return newSchema
}

The old way of creating schemas will continue available but by default the generated will use the new one.

We must be able to handle multiple events in the same listener. The example below shows both syntaxes (the current one and for multiple events):

For one event

{
  event: 'core.models.add.user'

  // (...)
}

For multiple events

{
  event: [
    'auth.checkSessionResponse',
    'auth.afterLogin'
  ]

  // (...)
}

The input field can have a property named convertTo, it will convert the input value to a given type or execute a function to format it.

So, after the explanation above I think we must rename this property to format. Please, give your thought.

Next Step

After this discussion a proper example will be added to the documentation.

{
  age: {
    format: Number
  },
  phone: {
    format (value, api) {
      return value.replace(/(\d{3})(\d{3})(\d{4})/, '$1-$2-$3')
    }
  }
}

When a distracted developer activate an invalid module (a module who not exists or don't has a manifest.json file) the stellar 💥

The idea is show a proper message like:

There is an invalid module active, named "invalidModule", fix this to start Stellar normally.