I'm triying to install but I get:

Error: Argument passed in must be a single String of 12 bytes or a string of 24 hex characters

I need a setTimeout for adding some per-collection functions (expressa.db.collection.bless = function(){}):

app.listen( port, function(){
  setTimeout( function(){
    expressa.db.mycollection.bless = function(doc){
       doc.disable = function(){
         doc.state = "disabled"
       }.bind(doc)
    }
  }, 500 ) // otherwise only expressa.db.settings is available (collections didn't settle down yet)
})

I think something like this is what we want:

expressa.db.on('ready',function(){
   expressa.db.mycollection.bless = function(doc){
       doc.disable = function(){
         doc.state = "disabled"
       }.bind(doc)
    }
}

app.listen( port )

While i don't think it will necessarilly be quick, i do think it would be relatively straightforward to generate a javascript client script containing all the default calls to all the collections.

Something similar to what is exposed by expressa in node ie: api.db.users.get but for the endpoints specified in the the readme. The idea being the client script does the more laborious url construction and the consumer on the client just does something like api.users.get( ... ) or await api['email_signups].update( ... ) etc.

Exposing all the collections is probably not a great idea, so a whitelisting approach could be used by setting a property on the collection schema 'for client'. Client script would then be generated when the database is created and when collections are added/removed.

Just an idea!

I want my expressa api to sit on the root of the server, everything seems to work except the /admin page is no longer accessible:

For example, running this:

var express = require('express');
var app = express();

var expressa = require('expressa');
app.use('/', expressa.api());            // optionally pass in settings
app.use('/admin', expressa.admin({ apiurl: '/' }));

app.listen(3000, function () {
    console.log('Example app listening on port 3000!');
});

then navigating to http://localhost:3000/admin/

results in the following error:

{"error":"unknown collection"}

Hi,

I had to migrate file-collections to postgres using expressa-cli, and found out all my tests failed.

REASON

It turns out all the database adapters return an id after a create() db-call, except postgres returns an object.
I introduced an environment-flag to not cause harm for existing postgres expressa-users for now.
However, I do believe we want to make this default behaviour, so we could, at somepoint bump the versionnumber to indicate a breaking change for postgres users (v0.4.x etc).

after installing via npm and adding necessary code changes, when I go to http://localhost:3000/admin/ i am redirected to install page but when I try to save changes, i get error.

there is no data/settings/development.json file being created.

Any help would be appreciated.

Hi,

in the Admin interface how do i connected collections by relationship?
for example i have POST and TAG collections, and in a post item i want to have a field/property named tags which is type array of ids that holds the ids from the TAG collection...?

thanks.

Hi,

I am following the instructions on the README to get a simple server and database set up. I have the following file server.js

`
var express = require('express');
var app = express();

var expressa = require('expressa');
app.use('/api', expressa.api()); // optionally pass in settings
app.use('/admin', expressa.admin({ apiurl: '/api/' }));

app.listen(3000, function () {
console.log('Example app listening on port 3000!');
});`

I have created an empty database 'test' in pgAdmin

I run my server, and navigate to localhost:3000/admin

But when i click i get the following error(s):

I am on a laptop using windows 10

Thanks

The 'Download All' button does not work in all locations its listed. In the console the error reads Uncaught TypeError: t.downloadCSV is not a function

I think we need a way to add custom endpoints that also have expressa features like req.user and req.hasPermission so that these custom endpoints can use the current permission model.

Hi,
from your readme I would have expected to have a login. But there are no credentials mentioned. So I fail attempting to get through the admin UI. Any help appreciated!

it seems /users/me seems to bypass the listeners (which could hide/delete sensitive properties).

I installed expressa following readme and got an error after install:

error adding collection permissions

Error: could not load data

    at /rbd/pnpm-volume/db2e9f30-6bfe-4850-b75e-e887ea2d0f78/node_modules/.registry.npmjs.org/jfs/0.2.6/node_modules/jfs/lib/Store.js:209:15

    at ReadFileContext.callback (/rbd/pnpm-volume/db2e9f30-6bfe-4850-b75e-e887ea2d0f78/node_modules/.registry.npmjs.org/jfs/0.2.6/node_modules/jfs/lib/Store.js:94:16)

    at FSReqWrap.readFileAfterOpen [as oncomplete] (fs.js:420:13)

adding collection permissions for settings

adding collection permissions for collection

adding collection permissions for role

adding collection permissions for users

adding collection permissions for post

And this is the resulting admin UI I get:

Error: could not load data
at /var/www/app/node_modules/jfs/lib/Store.js:209:15
at ReadFileContext.callback (/var/www/app/node_modules/jfs/lib/Store.js:94:16)
at FSReqWrap.readFileAfterOpen [as oncomplete] (fs.js:365:13)
j

Im in the situation where i would like to use data/settings/development.json on my buildserver (without production configvalues).

Any idea how to initialize the development-settings?

Most nodejs applications distinguish between production/development using the NODE_ENV environment flag:

if( process.env.NODE_ENV == "production" ){

}else{ // by default nodejs sets it to "development"

}

expressa could do something similar, by creating both development.json and production.json in data/settings/. upon first run...and load one based on the NODE_DEV value.

Example

index.js

    28  router.settings = {}
    29
-   30  db.settings.get('production')
+  30  db.settings.get( process.env.NODE_ENV )
    31          .then(function(data) {
    32                  router.settings = data;

NOTE: if this sounds ok let me know, and we'll prepare a PR

#node v6.9.1, npm 4.0.2, Windows 10 64 Pro

starting

node app.js

Example app listening on port 3000!
error reading settings
Error: could not load data
at C:\expressa\myapp\node_modules\jfs\lib\Store.js:209:15
at ReadFileContext.callback (C:\expressa\myapp\node_modules\jfs\lib\Store.js:94:16)
at FSReqWrap.readFileAfterOpen [as oncomplete] (fs.js:365:13)
validators loaded.

navigation to http://localhost:3000/admin

my err handler
TypeError: Cannot read property 'get' of undefined
at getById (C:\expressa\myapp\node_modules\expressa\index.js:249:27)
at C:\expressa\myapp\node_modules\expressa\index.js:106:2
at Layer.handle [as handle_request] (C:\expressa\myapp\node_modules\express\lib\router\layer.js:95:5)
at next (C:\expressa\myapp\node_modules\express\lib\router\route.js:131:13)
at Route.dispatch (C:\expressa\myapp\node_modules\express\lib\router\route.js:112:3)
at Layer.handle [as handle_request] (C:\expressa\myapp\node_modules\express\lib\router\layer.js:95:5)
at C:\expressa\myapp\node_modules\express\lib\router\index.js:277:22
at Function.process_params (C:\expressa\myapp\node_modules\express\lib\router\index.js:330:12)
at next (C:\expressa\myapp\node_modules\express\lib\router\index.js:271:10)
at addRolePermissionsMiddleware (C:\expressa\myapp\node_modules\expressa\role_permissions.js:8:11)

selecting filebased gives

updated settings collection storage
updated collection collection storage
updated role collection storage
error adding collection permissions
Error: could not load data
at C:\expressa\myapp\node_modules\jfs\lib\Store.js:209:15
at ReadFileContext.callback (C:\expressa\myapp\node_modules\jfs\lib\Store.js:94:16)
at FSReqWrap.readFileAfterOpen [as oncomplete] (fs.js:365:13)
updated users collection storage
error adding collection permissions
Error: could not load data
at C:\expressa\myapp\node_modules\jfs\lib\Store.js:209:15
at ReadFileContext.callback (C:\expressa\myapp\node_modules\jfs\lib\Store.js:94:16)
at FSReqWrap.readFileAfterOpen [as oncomplete] (fs.js:365:13)
updated post collection storage
adding collection permissions for settings
updated settings collection storage
adding collection permissions for collection
updated collection collection storage
adding collection permissions for role
updated role collection storage
adding collection permissions for users
updated users collection storage
adding collection permissions for post
updated post collection storage

when selecting mongodb

Example app listening on port 3000!
error reading settings
Error: could not load data
at C:\expressa\myapp\node_modules\jfs\lib\Store.js:209:15
at ReadFileContext.callback (C:\expressa\myapp\node_modules\jfs\lib\Store.js:94:16)
at FSReqWrap.readFileAfterOpen [as oncomplete] (fs.js:365:13)
validators loaded.
my err handler
TypeError: Cannot read property 'get' of undefined
at getById (C:\expressa\myapp\node_modules\expressa\index.js:249:27)
at C:\expressa\myapp\node_modules\expressa\index.js:106:2
at Layer.handle [as handle_request] (C:\expressa\myapp\node_modules\express\lib\router\layer.js:95:5)
at next (C:\expressa\myapp\node_modules\express\lib\router\route.js:131:13)
at Route.dispatch (C:\expressa\myapp\node_modules\express\lib\router\route.js:112:3)
at Layer.handle [as handle_request] (C:\expressa\myapp\node_modules\express\lib\router\layer.js:95:5)
at C:\expressa\myapp\node_modules\express\lib\router\index.js:277:22
at Function.process_params (C:\expressa\myapp\node_modules\express\lib\router\index.js:330:12)
at next (C:\expressa\myapp\node_modules\express\lib\router\index.js:271:10)
at addRolePermissionsMiddleware (C:\expressa\myapp\node_modules\expressa\role_permissions.js:8:11)
updated settings collection storage
updated collection collection storage
updated role collection storage
error adding collection permissions
TypeError: Parameter "url" must be a string, not undefined
at Url.parse (url.js:87:11)
at Object.urlParse [as parse] (url.js:81:5)
at module.exports (C:\expressa\myapp\node_modules\mongodb\lib\url_parser.js:17:23)
at connect (C:\expressa\myapp\node_modules\mongodb\lib\mongo_client.js:290:16)
at Function.MongoClient.connect (C:\expressa\myapp\node_modules\mongodb\lib\mongo_client.js:114:3)
at C:\expressa\myapp\node_modules\expressa\db\mongo.js:51:17
at Object.get (C:\expressa\myapp\node_modules\expressa\db\mongo.js:50:11)
at addCollectionPermissions (C:\expressa\myapp\node_modules\expressa\collection_permissions.js:51:23)
at C:\expressa\myapp\node_modules\expressa\index.js:118:11
at Array.map (native)
updated users collection storage
error adding collection permissions
TypeError: Parameter "url" must be a string, not undefined
at Url.parse (url.js:87:11)
at Object.urlParse [as parse] (url.js:81:5)
at module.exports (C:\expressa\myapp\node_modules\mongodb\lib\url_parser.js:17:23)
at connect (C:\expressa\myapp\node_modules\mongodb\lib\mongo_client.js:290:16)
at Function.MongoClient.connect (C:\expressa\myapp\node_modules\mongodb\lib\mongo_client.js:114:3)
at C:\expressa\myapp\node_modules\expressa\db\mongo.js:51:17
at Object.get (C:\expressa\myapp\node_modules\expressa\db\mongo.js:50:11)
at addCollectionPermissions (C:\expressa\myapp\node_modules\expressa\collection_permissions.js:51:23)
at C:\expressa\myapp\node_modules\expressa\index.js:118:11
at Array.map (native)
updated post collection storage
C:\expressa\myapp\node_modules\mongodb\lib\mongo_client.js:237
throw err
^

Error: Argument passed in must be a single String of 12 bytes or a string of 24 hex characters
at new ObjectID (C:\expressa\myapp\node_modules\bson\lib\bson\objectid.js:34:11)
at C:\expressa\myapp\node_modules\expressa\db\mongo.js:52:40
at connectCallback (C:\expressa\myapp\node_modules\mongodb\lib\mongo_client.js:315:5)
at C:\expressa\myapp\node_modules\mongodb\lib\mongo_client.js:234:11
at _combinedTickCallback (internal/process/next_tick.js:67:7)
at process._tickCallback (internal/process/next_tick.js:98:9)

...

For those who want to execute custom queries etc on the database, it would be better if the request was made from the existing Pool already created within expressa. The alternative is requiring pg in your project and creating another pool/connection which to me is inefficient and bypasses the contract between your server and expressa. I think all comms should happen via expressa.

Small side note -> i noticed you create a pool per collection, which could result in quite a few pools depending on application, and the documentation recommends limited number, even just 1: https://node-postgres.com/features/pooling

I tried updating some isolated properties of an object.
However, looking at the code it seems unfinished no?

the 'changed'-variable seems unused in router.post('/:collection/:id/update', > (req, res, next) { in index.js

This seems to have started since the 0.5.5 update. Basically if you have multiple listeners such as

var express = require('express');
var app = express();

var expressa = require('expressa');
var api = expressa.api();
app.use('/admin', expressa.admin({apiurl: '/api'}));
app.use('/api', api);            // optionally pass in settings

api.addListener('ready', function ready() {
  app.listen(3000, function () {
    console.log('Example app listening on port 3000!');
  });
})

api.addCollectionListener('get', 'users', async function usersGet(req, collection, data) {
})

api.addCollectionListener('post', 'users', async function usersPost(req, collection, doc) {
})

api.addLateCollectionListener('post', 'users', async function usersPostLate(req, collection, doc) {
})

api.addCollectionListener('put', 'users', async function usersPut(req, collection, doc) {
})

And then try fresh install via the /admin panel (using users storage type as 'postgresql', you will notice the following error:

From what i can tell this is a race condition, the install is somehow losing its usual order of operations, and in this case, expressa is trying to retrieve the 'Admin' role document before it has been created

The db abstraction layer should be unified more. Currently different types handle adding a duplicate key differently.

Quite low in terms of severity: Once you install a server, you are unable to log out and log back in again. You have to restart the server in order to be shown the log in screen at /admin

Hi,

In the documentation it says we are able to execute queries on a collection via the api:

/:collection/?query={..}

Where the query has to be in the form of a mongo-query. If my databse is jsonb postgres can i execute jsonb sql queries?

Hello,

I understand that the JSON file support offers a nice "on file database" solution, but I would like to propose offering SQLite support as well. JSON Files have a locking issue, and are limited by the file system. SQLite 3 offers most of the power of PostgreSql with the advantages of being lighter weight in every way. Along with offering a single easy to backup file as opposed to a complex database system that requires dedicated backup tools. SQLite is also a popular developer alternative to PostgresSql, as it allows the developer to reduce load on their machine while developing.

SQLite recently added support for json fields which should allow for a fork of "mongo-query-to-postgres-jsonb" that turns it into "mongo-query-to-sqlite-json1".

Not totally sure about it, but i noticed a missing password-property.
Im suspecting that adding some roles (thru expressa-admin) caused this.
I'll do some more research.

idea: would be neato to be able to setup collections by 👍

• adding an extra 'from payload'-button
• which runs the json payload thru https://www.npmjs.com/package/schemize

Expressa currently has a mixed policy when it comes to responding (results/errors in json or plain text).

I would opt for json-only responses, by creating an error function in index.js like this:

+ router.error = function(res,msg,code,info){
+  res.status(204).send({"error":msg, "code": code || 1,"info": info || ""}).end()
+ }

  router.use(>(req, res, next) {
·   req.settings = router.settings;
+   res.set('Content-Type','applicatioin/json'')
+   res.error = router.error
·   next()
  })

Now anywhere in the expressa code, or listeners, one could call res.error("some error") and the rest is taken care of.

Browser client-execution wouldn't crash on JSON parse-errors (in case of plaintext) or 404-statuses, they can recover appropriately by checking for the error-key in every response (and present a custom, more human errormessage based on the code-field)

WDYT?

In the era of node >= 10 is it necessary to use BlueBird?

Since native promises have come a long way, I don't see any of Bluebirds special features being used so could now be a good time to reconsider?

My other concern is expressa overwrites the global.Promise giving no way for the consumer to choose native promises themselves.

Hi,

So i installed a server and created an admin user. Then, logged in as that admin user in the /admin page, i created a new user with the predefined Authenticated role. I then logged out of the super user, and back in as the new 'Authenticated' user, and was still able to do everything i did as an admin user, creating collections, adding users etc. Is this correct? I was thinking because the Authenticated role only has these limited { "users: view own": true, "users: edit own": true, "users: delete own": true } permissions, that i shouldn't be able to?

Hi, i am actually pretty new to backend world.

I just want to figure out what fits my need. So i am curious, is it the same as keystonejs or different?. I am looking for API-first / headless cms actually. Is expressa good choice for my case?
Thanks.

Super small issue. Breadcrumb in the admin ui does not work

Hi,
I really like this project it is very nice.
I tried few API interfaces but they were very limited, this looks like a good start.

In order for me to save time to create API REST endpoints when i do mocks i use "json-server" package - https://github.com/typicode/json-server
its super quick, you have a JSON file structure, that from within that i auto generate the routes also with nested sub routes with relationship and rules.

i think that adding this as an "import feature" could auto create collections schema and routes will be great! any thoughts?

I have a user who could not edit their own document in another collection despite having the correct editown permission. I noticed the documents owners were coming in as undefined and i believe it is tracked down to the following:

In collections.js line 127

const owner = doc.owner && doc.meta.owner

Maybe supposed to be

const owner = doc.meta && doc.meta.owner

?

I understand how to add an expressa.addListener to each document, but I would like to know if there is a way to add a listener to expressa that happens right before the final response is sent.

I'd like to add a copyright message to the end of my entire result before it's sent to the client.

Can't quite figure it out. :/

I just noticed this on chrome v66:

the issue: /user/me endpoint get cached (which prevents account-switching).

how to reproduce

1. login as user X
2. logout
3. login as user Y
4. /user/me will return user X

Interestingly enough, this only happens on a server (which is behind SSL, not sure if that has something to do with it). Localhost seems to work as expected.

Im now trying this index.js patch:

router.get('/user/me', function(req, res, next) {                                                       
      req.params.collection = 'users';                                                                    
      req.params.id = req.uid;
+    res.setHeader("Cache-Control", "no-cache, no-store, must-revalidate");                              
+    res.setHeader("Pragma", "no-cache");                                                                
+    res.setHeader("Expires", 0);                                                                        
      getById(req, res, next);
      //res.redirect('/users/' + req.uid)
  }); 

I will keep you posted

Hi,

I updated my test app to 0.5.0.

I followed the exact steps in the readme but am unable to reach the admin page.

server.js

error in chrome

console

node 10.20.1

I pulled the latest version, but got this:

$ node --version
4.0.0
$ node app.js 
/mnt/data/home/sqz/projects/expressa/node_modules/promise-sequential/index.js:9
    let count = 0;
    ^^^

SyntaxError: Block-scoped declarations (let, const, function, class) not yet supported outside strict mode
    at exports.runInThisContext (vm.js:53:16)
    at Module._compile (module.js:413:25)
    at Object.Module._extensions..js (module.js:452:10)
    at Module.load (module.js:355:32)
    at Function.Module._load (module.js:310:12)
    at Module.require (module.js:365:17)
    at require (module.js:384:17)
    at Object.<anonymous> (/mnt/data/home/sqz/projects/expressa/index.js:8:18)
    at Module._compile (module.js:434:26)
    at Object.Module._extensions..js (module.js:452:10)

Lets try strict:

$ node --use-strict app.js 
/mnt/data/home/sqz/projects/expressa/node_modules/bcrypt-nodejs/bCrypt.js:519
        password = password + (minor >= 'a' ? "\000" : "");
                                                ^^

SyntaxError: Octal literals are not allowed in strict mode.
    at exports.runInThisContext (vm.js:53:16)
    at Module._compile (module.js:413:25)
    at Object.Module._extensions..js (module.js:452:10)
    at Module.load (module.js:355:32)
    at Function.Module._load (module.js:310:12)
    at Module.require (module.js:365:17)
    at require (module.js:384:17)
    at Object.<anonymous> (/mnt/data/home/sqz/projects/expressa/auth/index.js:2:14)
    at Module._compile (module.js:434:26)
    at Object.Module._extensions..js (module.js:452:10)

I tried looking for a newer bcrypt, but it's already at the latest version.
I think the promise-sequential requires strict mode, but bcrypt needs to run outside strict mode.
Not sure how to solve this, but i think we should find a solution as many services still run < node v4

can you reproduce:

• create an 'users'-collection
• uncheck anonymous permission for "view users"
• do anonymous request to /api/users in the browser

We need that the any kind of sensitive information should not be visible to anonymous, authenticated user-role. Only admin role should have access to that.
can you assist in resolving this issue?

not sure, but maybe this allows users with 'edit own: true'-permissions to overwrite non-owned collection-items by sending a PUT request with a modified {.....,meta:{owner:'hackeruid'}}

In the case of updating items, we might want expressa to retrieve the collection-item first from the db, so we'll know for sure we're dealing with a legit owner-property.

roles, collections and settings are all required to be stored in files. All of these are accessed very frequently, so they'll need to be cached in memory. I'm not sure yet how often the cached version should be updated, maybe only when expressa starts up.

'/:collection` however, does allow listeners.
I guess there's a notify missing

I was thinking it might be wise to limit the below call in allowFirstUserAsAdmin to top 1 result. Since it happens for every post query on the users collection it might be called frequently, theoretically getting worse as a site gets more users. Also as more users get added the json payload grows, and we just need a count.

const userCount = (await api.db.users.find()).length

I am not a mongo query expert but from some googling it seems this may be achievable with { $limit : 1 }

I'm planning on adding a basic password reset feature. Here's my plan

• create easy way for developer to request password reset url (so they can include in an email)
• add endpoint that renders a choose new password form
• add endpoint that processes the response and sets the user password

Hi,

Was creating new collections from the /admin panel and ran into the following problems:

1. Unable to give to give properties a description using json schema editor
2. When changing type to number and setting a min and max, the the request fails with a validation error but the collection creation succeeds
3. When creating a property of type 'object', after completing one object and returning to 'Add another field' on the root, the object you just created gets cleared (particularly furstrating after having created a long object!!)
4. There is an error in the console when clicking 'Add another field' (might be related to point 3)
   
5. After having created a collection with any storage type, and adding data to the collection, the _id property is not populated, should be by default right? When using postgres, the id column is also empty. This seems to work fine for users though
6. For new collections all the own permissions do not get auto added to the Admin role, should they?
7. New permissions are added with true and not 1 as with the default collections, this seems to be updated when the server is restarted

It seems they are more flaky when running on windows. Not sure yet how this can be solved. I think adding cypress to the CircleCI runner would help too to know if they're flaky in general.

Still trying to make something 😸
I don't get why it throw that since addListener is present in listerner.js.
Here's the beginning of server.js:

var express = require('express');
var app = express();
var expressa = require('expressa');
app.use(express.static('public'));
app.use('/api', expressa.api());            // optionally pass in settings
app.use('/admin', expressa.admin({ apiurl: '/api/', showCriticalFeaturesOnProduction: true }));
app.get('/', function(request, response) {
  response.sendFile(__dirname + '/views/index.html');
});
expressa.addListener('get', -101, function (req, collection, doc) {
...

I tried following https://github.com/thomas4019/expressa/blob/master/doc/relationships.md but didn't get expected results.
Is this property in meta? Or declared as object in root, then additional properties in root?
Do we need to change additionalProperties to true?

Thanks!

Any idea how could append them into the permission-mechanism and append them to the 'Permission'-table in expressa-admin?

// this will output all custom express endpoints
app._router.stack.map(>(endpoint){
  console.log( endpoint.regexp.toString() )
})

It would make expressa the first express middleware (I've seen) to apply role/permissions on programmatically added endpoints.

Background

The problem with all express applications is a lack of centrally securing all endpoints.
So cases like this:

app.use('/foo/bar',function(req,res,next){   .... })
app.get('/flop',function(req,res,next){ ..  })

Totally bypasses authentication/roles.
Ofcourse this can be solved like this:

app.get('/flop',myAuthMiddleware, function(req,res,next){ ..  })

However, then it still bypasses all finer grained permissions.
Since expressa already has finegrained controlled of permissions, it would be neato to add the non-expressa endpoints in there as well.

WDYT?

Using the new admin interface i setup a database with the users storage set to 'postgres'. And the set up succeeds but it seems the default username and password ([email protected] and 123) are what is carried into the database, and not the values i entered in the setup page

Tokens are currently valid forever. It would be good to add an expiration date to them and make this a configurable setting so admins can decide what works best for their app.