There is no operation to find out the total number of elements that a query returns without retrieving the element list from the server.
The current store-test version is 1.1.0 (aka the current spec version), most of the stores currently implement 1.0.0.

If this new feature request is accepted we should add in 1.2.0 a count functionality test. In time stores will implement it when they adhere to 1.2.0 specs.

@rjrodger @mirceaalexandru what do you think? Is there is a reason for not implementing this functionality until now?

npm ERR! notarget No matching version found for [email protected].

if you need to reference unpublished modules, use the github and tag reference format in package.json

Since seneca-store-test and seneca-mem-store are co-dependent, we will need to publish this as 4.0.0 with the github reference and then again as 4.0.1 with the correct version reference

because see senecajs/seneca-jsonfile-store#7

test organization should be similar to the existing tests

the build always needs to pass before we can release

specs for the remove command here https://github.com/rjrodger/seneca/blob/master/doc/data-store.md#cmdremove say that the load$ parameter defaults to false, but in the mem-store (which I consider like the reference implementation) it defaults to true.
(also different is the behaviour if all$ is true: no ent is returned)

Which one is correct?

specs https://github.com/rjrodger/seneca/blob/master/doc/data-store.md say that invalid skip and limit values should be silently ignore, however mem-store currently interpret a negative number as the index from the bottom.

Which one is the correct?

What types of errors should be returned by the stores upon failures?

At the moment I think the stores are just returning the error from the underlying driver, but I think it would be valuable to wrap those errors in seneca specific errors.
That would make easier to the consumer to check for certain conditions (i.e.: duplicate key, connection closed etc), not requiring knowledge of the driver/db specific error codes

I was thinking also for a pattern matching implementation in some of the new versions. I mean - would it be possible to extend the seneca-standard-query (or is it there already, and i missed it?) with a functionality like this:

entity.list$({stringProp:/some_portion_of_a_value/, ...}

I understand that implementing it with a regex would be very hard having in mind specific implementations of the db stores but, probably an implementation as simple as indexOf(pattern) > -1

It is a common functionality for all db stores.

I think that existing standard-query + count$ (feature request) + pattern marchers (this request) would already cover >90% of the web application needs

Example: https://github.com/senecajs/seneca-entity-depend/blob/main/.github/workflows/build.yml

There is another test we would like to have: a vanilla db connection and insert/query - using only the db API module - no seneca involved.

This is a sanity test to make the db api/module itself is working - allows us to exclude seneca issues when there are weird bugs.

We need a way to create fixtures during test setup and register the id for later use, in place of using create with id$ and a fixed id.

This way the test store will be suitable for the seneca-shard-store

This will eliminate the need to manually re-generate the test db each time the schema of the test db changes.

The idea is that when a PR is submitted for seneca-store-test, the CI shall fetch a bunch of store plugins (e.g. mongo-store, postgres-store, mem-store, etc.), and run their tests against the seneca-store-test suite.

Consider an entity foo with these attributes:

    {
      id$: 'to-be-updated',
      p1: 'v1',
      p2: 'v2',
      p3: 'v3'
    }

What is the expected value of p3, when updating the entity passing in new values only for p1 and p2, like in the following code?

      var foo = si.make({ name$: 'foo' })
      foo.id = 'to-be-updated'
      foo.p1 = 'z1'
      foo.p2 = 'z2'

      foo.save$(function (err, foo1) {
        console.log(foo1.p3)
      }

And what is the expected value of p3 in the store?

At the moment my impression is that sql based stores and nosql ones behave differently.
I'm sure that mem-store doesn't merge the object but completely replace them (thus p3 is undefined) and that mysql and oracle stores merges them (thus p3 = "v3").

I have a preference for merge solution (in my opinion you should explicitly pass a null/undefined attribute to have it cleared in the store too)

on postgres store the native$ tests like: list$({ native$: 'SELECT * FROM product ORDER BY price' } are passing although the native$ is not working. The list$(native$:...) does a select all instead of executing the query but the result accidentally matches the tests.

The tests need to be enhanced.

On issue #16 (comment) you said that list$ should take an array of ids as "opaque ids" and I coded that in the tests.

However I'm now updating the mem-store and noticed that for the same functionality on load$ the id is handled on seneca side (translating a single string into { id: original_string }.

I'd like to know if you prefer to add the same support for this in list$ on seneca side or leave it to the stores to handle such cases

some of the sql stores add some support for sql queries beyond the native$ method: they allow sql strings or an array containing sql string and params as input to the list$ method.

This has also been codified in the tests, but there's no indication of that in the specs.
Should we keep this functionality? Should it be present among tests?

let mf = () => seneca.make('foo')

let f01 = await mf().data$({x:1,y:22}).save$()
let f02 = await mf().data$({x:2,y:33}).save$()
let f03 = await mf().data$({x:2,y:44}).save$()
let f04 = await mf().data$({x:1,y:55,upsert$:['x']})
let foos = await.mf().list$()

// tests:
// f01.id != f02.id != f03.id
// f01.id == f04.id
// 3 == foos.length
// foos: [ {id:'<f01>', x:1, y:55 }, {id:'<f02>', x:2, y:33 }, {id:'<f03>', x:2, y:44 }, ]
// note: db does not have a unique index on x for this test

@lilsweetcaligula please review and examine for logical errors!

I noticed that list$ and load$ will ignore the entity on which they are called and only use the passed in query to filter records.

remove$ however works slightly different: it ignores the original entity and use the query only if passed in. If no query is passed in the entity is used instead:

var foo = seneca.make('foo')
foo.a = '1'
foo.remove({ b: '2' }, function () {
   // removes only where b = 2
})

whilst

var foo = seneca.make('foo')
foo.a = '1'
foo.remove(function () {
   // removes where a = 1
})

Is this the intended behaviour?
Should this be replicated on load and list too?

ps: I think this is actually handled at seneca (not store) level

const seneca = require('seneca')();
const productPlugin = require('./plugins/product');
seneca.use(productPlugin);
seneca.use('mongo-store', { uri: 'mongodb://120.0.0.1:27017/test' })
seneca.ready((err) => {
    seneca.act({ area: 'product', action: 'add', category: 'food', name: '方便面' }, (err, result) => {
        console.log(result);
    });
});

When i use the plugin 'mongo-store'.The console write a fetal log 'The definition function for the plugin mongo-store has failed: Cannot read property 'init' of undefined'.
I try to find the reason.So i Debug the file mongo-store.js.

// the code in the file 374 row.
// error: seneca.store is undefined.
var meta = seneca.store.init(seneca, opts, store)

I think the reason is seneca missing some api. So ceate the issues.

supports cases where db triggers might update record
see senecajs/seneca-entity#3

In order to detect the issue I reported in senecajs/seneca-mem-store#2,
I added this failing test to my copy of seneca-store-test v 1.0.0

    it('should not save modifications to entity after save completes', function (done) {
      scratch.foo2 = si.make({ name$: 'foo' })
      scratch.foo2.p3 = ['a']
      scratch.foo2.save$(verify(done, function (saved_foo) {
        assert.deepEqual(saved_foo.p3, ['a'])
        // now that foo is in the database, modify the original data
        scratch.foo2.p3.push('b')
        assert.deepEqual(saved_foo.p3, ['a'])
      }))
    })

I didn't want to create a PR for this until you agree that the issue I filed is a bug.

Separate into named functions and list linearly at top of test files

create branch for move to @seneca/store-test for this

@hapi/lab should be used everywhere
@hapi/code as well
separate PR

not urgent

prevents store implementation overriding existing field data that is not specified in saved entity fields - which is very bad

Add a test that verifies that the limit$ is by default 20
Add a test that verifies the nolimit option

While I was working on the seneca-sqlite-store, I stumbled upon a very interesting bug. Below I present the debug statements that help explain the problem:

// lib/sqlite-store.js
    save: function (args, done) {
      console.dir(args.ent, { depth: 4 }) // debug output

// node_modules/seneca-store-test/store-test.js
    describe('happy path', () => {
      it('is happy', async (fin) => {
        const foo_ent = si.entity('foo')
        const f01 = await foo_ent.data$({ x: 1, y: 22 }).save$()
        const f02 = await foo_ent.data$({ x: 2, y: 33 }).save$()

Running the 'happy path' test against the sqlite-store then produces the following output:

Output:
Entity { 'entity$': '-/-/foo', x: 1, y: 22 }
Entity {
  'entity$': '-/-/foo',
  x: 2,
  y: 33,
  id: '83e504c0-c72c-40f2-9af9-9214847d86cc'
}

That id field should really not be there. With equivalent debug statements in place in seneca-mongo-store, below is the output of the 'happy path' test:

Entity { 'entity$': '-/-/foo', x: 1, y: 22 }
Entity { 'entity$': '-/-/foo', x: 2, y: 33 }

The seneca-postgres-store also works fine on my fix-tests branch:

Entity { 'entity$': '-/-/foo', x: 1, y: 22 }
Entity { 'entity$': '-/-/foo', x: 2, y: 33 }

This may be related to the Seneca#entity method and/or the seneca-promisify package.

Received a bug report that says redis crashes when deleting not existing entities. Need to add tests to cover this scenario.
senecajs/seneca-redis-store#11

On issue #14 we agreed to require errors from the store to be wrapped into seneca specific errors, however I wasn't able to find a smart way to test them (apart the duplicate key error).

The main problem is that there isn't a guaranteed way to agnostically cause an error in the store underling driver. There are three possible path we could follow:

1. require stores to implement "hooks" to trigger errors. I don't like this approach because I don't like to write code just for the sake of testability, and the error would still be "fake-y" when testing
2. require the store implementor to pass in to the tests an additional seneca instance with the store configured in a way that will generate proper errors. I'm afraid this approach will make tests more obscure and complex and may discourage people from using seneca-store-test.
3. don't bother enforcing correct error behaviour in seneca-store-test. At a minimum we could provide a list of "assertion methods" to assert that the error has been properly created for a specific case and leave to the store implementor to use that (something along the lines of "assertValidConnectionError(err)" and "assertValidTimeoutError(err)").
   Those assertions should check for the correct error code and format

@rjrodger @geek what do you think?

npm run test should pass
see test/coverage.html for visual coverage report

      it('should not save modifications to entity after save completes', function (done) {
        var foo = si.make('foo')
        foo.p3 = [ 'a' ]
        foo.save$(verify(done, function (foo1) {
          assert.deepEqual(foo1.p3, [ 'a' ])
          // now that foo is in the database, modify the original data
          foo.p3.push('b')
          assert.deepEqual(foo1.p3, [ 'a' ])
        }))
      })

I think we should allow changing the object, is up to client of the store to change it or not. If change is required and after that another save$ is required, why do not let doing something like:

var foo = si.make('foo')
foo.p3 = [ 'a' ]
foo.save$(function (err, foo1) {
  // do something here
  foo1.p3.push('b')
  foo1.save$(function(err, foo2){...........})
})