tekartik / sembast.dart Goto Github PK

View Code? Open in Web Editor NEW

763.0 14.0 63.0 1.86 MB

Simple io database

License: BSD 2-Clause "Simplified" License

Dart 99.77% Shell 0.02% HTML 0.21%

sembast.dart's Introduction

sembast

NoSQL persistent embedded file system document-based database for Dart VM and Flutter with encryption support.

General

Yet another NoSQL persistent store database solution.

Pure dart solution working on Dart VM and Flutter using the file system as storage (1 database = 1 file). Works in memory (Browser, VM, Flutter, Node) for testing purpose

Supports single process io applications (Pure dart single file IO VM/Flutter storage supported)
Support transactions
Version management
Helpers for finding data
Web support (including Flutter Web) through sembast_web.
Can work on top of sqflite through sembast_sqflite.

Usage example:

notepad_sembast: Simple flutter notepad working on all platforms (web/mobile/desktop) (online demo)
demo_sembast: Simplest sembast demo based on the app template with added persistency. Online demo

Follow the guide.

Documentation

Documentation

sembast.dart's People

Stargazers

Watchers

Forkers

algobardo sroddy lo-tp tommyle thphuc goyourfly alejogamboa ronaldoslv optivolvethiyraashdavid f3ath ochibobo bharathmg-sharechat zeissus autodo-app christianedwardpadilla bookbot-kids chaddy012 abdurrahmanelrayes gideonaryeetey all4alex hungrymissilepod wetbikeboy2500 svetazelenova woshizilong supafoundation xanf-code grgbiki daniel-v do-wa lyf571321556 susobhan podcoder 0xero7 aldiemhaaa f-person mafreud coduggan arthur-becker susi-digital ilikerobots sashajo gustavo-henrique-em christiankleineidam rolies106 mrtsmglv mhmdrzahrv rkmaul shyam200 remejuan garyjdn malvernbright thetasinner laujangit stmerlhin i-atros wangqiang1588 geoai777 timshadel dharmeshkatariya eka12 hiyaai dora12stella

sembast.dart's Issues

Is sorting by Key ascending or descending working ?

I try to make a search filtered on document property (macAddress) and sorted by key (created by plugin)
I would like to get a returned record with the biggest key (sort order = false).

Setting false or true to the order parameters seems to return the same result.

How to return the last created document (with the biggest key) ?

Finder finder = new Finder(filter: Filter.equal('macAddress', macAddress), sortOrders: [SortOrder('key', false, true)], limit: 1);

finder.filter = new Filter.equal('macAddress', macAddress);
finder.sortOrder = new SortOrder('key', false, true);
finder.limit=1;

How to get last insert id?

when insert new data I want to get last id same time. How can i do that?

I really would like to query the sembast DB and retrieving a stream of data that updates itself once the underlying data change. I can see that there are some reactive wrappers in sqlite, to simulate the firestore realtime... I hope is possible to implement it, as that is the only main missing feature

v1.9.1 is very slow

I upgraded from 1.7.0 to 1.9.1 and the performance difference was marked. Saving data went from 5.5 seconds to 30+ seconds, the emulator burnt CPU like crazy. Anything I could/should do to debug further?

Question: is there an example of storing images and encrypting it

First of all great work. Huge respect. I owe you a big deal.

Well have a question though...

Is there an example of store an image/picture and encrypt it? I haven't been able to find any examples.

storage

What is the storage capacity?

Preparing for 2.0

The plan for sembast v2 is to remove the old API in favor of the new 'sounded' Store API.

The following class will be removed:

Store
Record
CompositeFilter
DatabaseExecutor
StoreExecutor
StoreTransaction
CompositeFilter
FilterOperation
FilterPredicate
ByKeyFilter

This particularly means that the convenient (unsounded) db.put/delete/get will not exist anymore in favor of RecordRef.put/delete/get allowing to type (or not) the key and the value)

The plan is to make an initial version 1.19.0-dev where the old APIs are deprecated allowing for people to make a smooth transition to 2.0.0 where they will be removed.

As of today (1.17.1), the future deprecated APIs are commented using // @deprecated v2 so that you can easily find them.

If you have any concern (one will likely be the documentation), please comment here.

remove all documents in store

Hello

What is the best way to remove all documents in a store?

Doesn't work in Singleton

Hello, then I'm trying to do something like this:

class NotesHelper {
   final NotesHelper _instance;
   Database _db;

   Future<NotesHelper> instance() async {
      if (_instance == null) {
         _instance = NotesHelper();
         _instance._db = await databaseFactoryIo.openDatabase(dbPath);
       }
       return _instance;
    }
}

Instance is created, but _db is always null and database doesn't create. There is no errors in log. I tried to move Database opening to another method and call it in instance, but no luck.
And if I am trying to make Database static, then it couldn't be opened too.
Could you help?

Ordering by property of nested object

By exemple:
Records:
{firstname: 'fred', lastName: 'toto', address: {city: 'paris'}}

The filter:
SortOrder('address.city')

type 'Utf8Decoder' is not a subtype of type 'StreamTransformer<Uint8List, String>' of 'streamTransformer'

Hi,

First of all thank you for putting in such an effort.

Here is the issue:
I just upgraded to the latest flutter version on 'master' and I am now getting the following error when calling databaseFactoryIo.openDatabase(databasePath):

type 'Utf8Decoder' is not a subtype of type 'StreamTransformer<Uint8List, String>' of 'streamTransformer'

All was running smoothly until the update.

Error when using find: "_CastError (type 'int' is not a subtype of type 'String' in type cast)"

I am using the following code in a part of my app to look up a record. I can debug and inspect the database and see that it is fine: I have merely one document which has a property with 'uid' set to a string.

var store = StoreRef<String, String>.main();
  var finder = Finder(filter: Filter.equals('uid', uid));
  var records = new List<RecordSnapshot<String, String>>();
  try {
    records = await store.find(db, finder: finder);
  } catch (e) {
    print("alexa this is so sad play despacito");
  }

But I get the error thrown when performing the find function:
_CastError (type 'int' is not a subtype of type 'String' in type cast)

This appears to be happening in the record_ref_impl.dart file in this function

@override
  RecordRef<RK, RV> cast<RK, RV>()

This is how I put a record into the database:

  Directory dir = await getApplicationDocumentsDirectory();
  String dbPath = dir.path + "local.db";
  Database db = await databaseFactoryIo.openDatabase(dbPath);
  db.put({'uid' : uid, 'settings' : settings, 'name' : name});
  db.close();

Auto-increment after deleting all files

Is there a way of making the integer auto-increment keys to revert back to one (key-1) after deleting all files in a store. (i.e delete method without providing a finder). Thanks.

process isolation

This looks pretty interesting, and thanks for putting this out there...

I am using Dart for Desktop and Mobile, and will try to use semblast for my prototype, but wanted to ask a few things about process...

On mobile, have your tried to run this with Flutter ? On mobile i have tried two architecture:

running the DD as a Service, witht he UI running in a webview (and Angular2/Dart)
running everything in the same procces with Flutter.

So, it begs the question, what have you tried.

On Desktop, i use Electron / Dart. It shoudl be easy to use semblast running in a different process that the Electron runtime kicks off. But it would mean that i need to use semblast with a Dart based Http Service. Have you tried using Semblast with a Http Service fronting it ?

Lastly, are you thinking about extending semblast so that it can also be durable, in that it use the disk also ?

thanks in advance

ImmutableMap<String, dynamic>' is not a subtype of type 'Map<String, String>' in type cast

Hey there! Thanks for this much needed library for dart!

I tried using the new Store API to find some records, by filtering based on keys, but it throws an error.

import 'package:sembast/sembast.dart';
import 'package:sembast/sembast_io.dart';

main() async {
  final db = await databaseFactoryIo.openDatabase("test.db");
  final store = StoreRef<String, Map<String, String>>.main();

  for (var i = 0; i < 10; i++) {
    await store.record(i.toString()).put(db, {"value": i.toString()});
  }

  await store.find(
    db,
    finder: Finder(
      filter: Filter.custom((record) {
        return record.key != "1";
      }),
    ),
  );
}

produces the following error:-

Unhandled exception:
type 'ImmutableMap<String, dynamic>' is not a subtype of type 'Map<String, String>' in type cast
#0      new SembastRecordSnapshot.fromRecord (package:sembast/src/record_snapshot_impl.dart:65:34)
#1      _SembastStoreRef&Object&StoreRefMixin.find.<anonymous closure> (package:sembast/src/store_ref_impl.dart:98:26)
#2      MappedListIterable.elementAt (dart:_internal/iterable.dart:414:29)
#3      ListIterable.toList (dart:_internal/iterable.dart:219:19)
#4      _SembastStoreRef&Object&StoreRefMixin.find (package:sembast/src/store_ref_impl.dart:99:11)
<asynchronous suspension>
#5      main (file:///home/dev/FlutterProjects/meghshala_app_flutter/test.dart:12:15)
<asynchronous suspension>
#6      _startIsolate.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:300:19)
#7      _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:171:12)

P.S. This works fine using the regular db.findRecords() API

[Help] How to import the export/Import helper functions?

Hello, I am a beginner with using Dart and Flutter, thanks for your useful package.
But I encountered a problem when I wanted to use these helper functions to test some examples(here) in document , Have these functions been exported to the package in the latest version?

Broken database file after app crash in iOS

App crash by some random bug, which cause the database file to be broken. Looks like sambast could not handle broken database file.

This is the exception I am getting when try to start the app again.

2017-10-16 21:32:52.794543-0700 Runner[1871:656406] [VERBOSE-2:dart_error.cc(16)] Unhandled exception:
FormatException: Unexpected character (at character 134)
...4918-8024-33f979b57a71","name":"g"}}{"key":"51b1d43e-3dbd-421c-918f-a5d3...

#0      _ChunkedJsonParser.fail (dart:convert-patch/dart:convert/convert_patch.dart:1361)
#1      _ChunkedJsonParser.parse (dart:convert-patch/dart:convert/convert_patch.dart:857)
#2      _parseJson (dart:convert-patch/dart:convert/convert_patch.dart:29)
#3      JsonDecoder.convert (dart:convert/json.dart:504)
#4      JsonCodec.decode (dart:convert/json.dart:135)
#5      Database.open.<anonymous closure>.<anonymous closure>.<anonymous closure> (package:sembast/src/sembast_database.dart:748)
#6      Stream.forEach.<anonymous closure>.<anonymous closure> (dart:async/stream.dart:793)
#7      _runUserCode (dart:async/stream_pipe.dart:11)
#8      Stream.forEach.<anonymous closure> (dart:async/stream.dart:793)
#9      _rootRunUnary (dart:async/zone.dart:1132)
#10     _CustomZone.runUnary (dart:async/zone.dart:1029)
#11     _CustomZone.runUnaryGuarded (dart:async/zone.dart:931)
#12     _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:330)
#13     _BufferingStreamSubscription._add (dart:async/stream_impl.dart:257)
#14     _SinkTransformerStreamSubscription._add (dart:async/stream_transformers.dart:68)
#15     _EventSinkWrapper.add (dart:async/stream_transformers.dart:15)
#16     _StringAdapterSink.add (dart:convert/string_conversion.dart:268)
#17     _LineSplitterSink._addLines (dart:convert/line_splitter.dart:156)
#18     _LineSplitterSink.addSlice (dart:convert/line_splitter.dart:131)
#19     StringConversionSinkMixin.add (dart:convert/string_conversion.dart:189)
#20     _SinkTransformerStreamSubscription._handleData (dart:async/stream_transformers.dart:120)
#21     _rootRunUnary (dart:async/zone.dart:1132)
#22     _CustomZone.runUnary (dart:async/zone.dart:1029)
#23     _CustomZone.runUnaryGuarded (dart:async/zone.dart:931)
#24     _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:330)
#25     _BufferingStreamSubscription._add (dart:async/stream_impl.dart:257)
#26     _SinkTransformerStreamSubscription._add (dart:async/stream_transformers.dart:68)
#27     _EventSinkWrapper.add (dart:async/stream_transformers.dart:15)
#28     _StringAdapterSink.add (dart:convert/string_conversion.dart:268)
#29     _StringAdapterSink.addSlice (dart:convert/string_conversion.dart:273)
#30     _Utf8ConversionSink.addSlice (dart:convert/string_conversion.dart:348)
#31     _Utf8ConversionSink.add (dart:convert/string_conversion.dart:341)
#32     _ConverterStreamEventSink.add (dart:convert/chunked_conversion.dart:86)
#33     _SinkTransformerStreamSubscription._handleData (dart:async/stream_transformers.dart:120)
#34     _rootRunUnary (dart:async/zone.dart:1132)
#35     _CustomZone.runUnary (dart:async/zone.dart:1029)
#36     _CustomZone.runUnaryGuarded (dart:async/zone.dart:931)
#37     _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:330)
#38     _BufferingStreamSubscription._add (dart:async/stream_impl.dart:257)
#39     _StreamController&&_SyncStreamControllerDispatch._sendData (dart:async/stream_controller.dart:770)
#40     _StreamController._add (dart:async/stream_controller.dart:641)
#41     _StreamController.add (dart:async/stream_controller.dart:587)
#42     _FileStream._readBlock.<anonymous closure> (dart:io/file_impl.dart:103)
#43     _rootRunUnary (dart:async/zone.dart:1132)
#44     _CustomZone.runUnary (dart:async/zone.dart:1029)
#45     _FutureListener.handleValue (dart:async/future_impl.dart:129)
#46     _Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:635)
#47     _Future._propagateToListeners (dart:async/future_impl.dart:664)
#48     _Future._completeWithValue (dart:async/future_impl.dart:477)
#49     _Future._asyncComplete.<anonymous closure> (dart:async/future_impl.dart:509)
#50     _rootRun (dart:async/zone.dart:1124)
#51     _CustomZone.run (dart:async/zone.dart:1021)
#52     _CustomZone.bindCallback.<anonymous closure> (dart:async/zone.dart:947)
#53     _microtaskLoop (dart:async/schedule_microtask.dart:41)
#54     _startMicrotaskLoop (dart:async/schedule_microtask.dart:50)

Filtering/Querying by multiple fields

Can someone point me to the correct way to filter by multiple fields? For example:

Give this data previously stored:

{
  'timestamp': '6846876879',
  'type: 'message'
  'address: '+1 789 789 7788',
  'message: 'ABC'
}

I would like to filter/query entries that are {'type': 'message', 'address': '+1 789 789 7788'}

is any migration feature?

i can't find this feature on api reference, could you help me how can i have this feature?

OS Error: Read-only file system, errno = 30

Just following along the "tutorial" setup, and this happens.

Code

void initDb() async {
    store = StoreRef.main();
    db = await databaseFactoryIo.openDatabase(dbPath, version: 1);
}

Result

E/flutter (25210): [ERROR:flutter/lib/ui/ui_dart_state.cc(148)] Unhandled Exception: FileSystemException: Cannot create file, path = '/sample.db' (OS Error: Read-only file system, errno = 30)
E/flutter (25210): #0      _wrap.<anonymous closure> (package:sembast/src/io/file_system_io.dart:102:7)
E/flutter (25210): #1      _rootRunUnary (dart:async/zone.dart:1132:38)
E/flutter (25210): #2      _CustomZone.runUnary (dart:async/zone.dart:1029:19)
E/flutter (25210): #3      _FutureListener.handleError (dart:async/future_impl.dart:144:20)
E/flutter (25210): #4      Future._propagateToListeners.handleError (dart:async/future_impl.dart:651:47)
E/flutter (25210): #5      Future._propagateToListeners (dart:async/future_impl.dart:672:24)
E/flutter (25210): #6      Future._completeWithValue (dart:async/future_impl.dart:483:5)
E/flutter (25210): #7      Future._asyncComplete.<anonymous closure> (dart:async/future_impl.dart:513:7)
E/flutter (25210): #8      _rootRun (dart:async/zone.dart:1124:13)
E/flutter (25210): #9      _CustomZone.run (dart:async/zone.dart:1021:19)
E/flutter (25210): #10     _CustomZone.runGuarded (dart:async/zone.dart:923:7)
E/flutter (25210): #11     _CustomZone.bindCallbackGuarded.<anonymous closure> (dart:async/zone.dart:963:23)
E/flutter (25210): #12     _microtaskLoop (dart:async/schedule_microtask.dart:41:21)
E/flutter (25210): #13     _startMicrotaskLoop (dart:async/schedule_microtask.dart:50:5)

Manifest

Manifest looks ok to me:

<uses-permission android:name="android.permission.INTERNET"/>
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"/>
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE"/>

Contains Filter not available

The db is containing model with firstName, lastName.
I need to query by firstName containing a pattern

Encrypting the database file

Is there a way to encrypt the database? I have a requirement of a secure database

Store.findRecords failing on flutter with dart 2

Thanks for the great package !

My problem: the findRecords method is failing on Flutter with Dart 2.

How to reproduce

Create a new Flutter project
Use the following pubspec.yaml:

name: sembast_flutter_tests
description: Test of the sembast package under Flutter

dependencies:
  flutter:
    sdk: flutter
  sembast: ^1.7.0
  path_provider: any
  cupertino_icons: ^0.1.2

dev_dependencies:
  flutter_test:
    sdk: flutter

flutter:
  uses-material-design: true

Put the following code in lib/main.dart:

import "package:sembast/sembast_io.dart";
import "package:sembast/sembast.dart";
import 'package:path/path.dart';
import 'package:path_provider/path_provider.dart';

void main() {
  getApplicationDocumentsDirectory().then((dataDir) {
    final dbPath = join(dataDir.path, "test.db");
    ioDatabaseFactory.openDatabase(dbPath).then((db) {
      final store = db.getStore("test store");
      final record = new Record(
        store,
        {"field1": "value for field 1"},
        1
      );
      db.putRecord(record).then((record) {
        Store store = db.getStore("test store");
        final finder = new Finder(limit: 1);
        store.findRecords(finder).then((records) {
          print(records);
        });
      });
    });
  });
}

Execute flutter run. I have the following output:

Using hardware rendering with device Android SDK built for x86. If you get graphics artifacts, consider enabling software rendering with "--enable-software-rendering".
Launching lib/main.dart on Android SDK built for x86 in debug mode...
Initializing gradle...                                       0.9s
Resolving dependencies...                                    0.8s
Running 'gradlew assembleDebug'...                           9.7s
Built build/app/outputs/apk/debug/app-debug.apk.
Installing build/app/outputs/apk/app.apk...                  1.4s
I/FlutterActivityDelegate(13860): onResume setting current activity to this
I/flutter (13860): test store
E/flutter (13860): [ERROR:topaz/lib/tonic/logging/dart_error.cc(16)] Unhandled exception:
E/flutter (13860): type 'Future<dynamic>' is not a subtype of type 'Future<List<Record>>' in type cast
E/flutter (13860): #0      Object._as (dart:core/runtime/libobject_patch.dart:67:25)
E/flutter (13860): #1      SembastStore.findRecords (package:sembast/src/store_impl.dart:115:8)
E/flutter (13860): #2      main.<anonymous closure>.<anonymous closure>.<anonymous closure> (file:///home/andrei/src/sembast_flutter_tests/lib/main.dart:20:15)
E/flutter (13860): #3      _RootZone.runUnary (dart:async/zone.dart:1381:54)
E/flutter (13860): #4      _FutureListener.handleValue (dart:async/future_impl.dart:129:18)
E/flutter (13860): #5      Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:633:45)
E/flutter (13860): #6      Future._propagateToListeners (dart:async/future_impl.dart:662:32)
E/flutter (13860): #7      Future._complete (dart:async/future_impl.dart:467:7)
E/flutter (13860): #8      _SyncCompleter.complete (dart:async/future_impl.dart:51:12)
E/flutter (13860): #9      SembastDatabase.inTransaction.<anonymous closure>.<anonymous closure> (package:sembast/src/database_impl.dart)
E/flutter (13860): <asynchronous suspension>
E/flutter (13860): #10     SynchronizedLock._runTask.<anonymous closure>.<anonymous closure> (package:synchronized/src/synchronized_impl.dart:189:29)
E/flutter (13860): #11     _rootRun (dart:async/zone.dart:1126:13)
E/flutter (13860): #12     _CustomZone.run (dart:async/zone.dart:1023:19)
E/flutter (13860): #13     runZoned (dart:async/zone.dart:1501:17)
E/flutter (13860): #14     SynchronizedLock._runTask.<anonymous closure> (package:synchronized/src/synchronized_impl.dart:187:14)
E/flutter (13860): #15     new Future.sync (dart:async/future.dart:222:31)
E/flutter (13860): #16     SynchronizedLock._runTask (package:synchronized/src/synchronized_impl.dart:186:16)
E/flutter (13860): #17     LockBase._createAndRunTask.run (package:synchronized/src/synchronized_impl.dart:61:14)
E/flutter (13860): #18     LockBase._createAndRunTask (package:synchronized/src/synchronized_impl.dart:71:17)
E/flutter (13860): #19     SynchronizedLock.synchronized (package:synchronized/src/synchronized_impl.dart:222:12)
E/flutter (13860): #20     SembastDatabase.inTransaction.<anonymous closure> (package:sembast/src/database_impl.dart:89:19)
E/flutter (13860): #21     new Future.<anonymous closure> (dart:async/future.dart:174:37)
E/flutter (13860): #22     Timer._createTimer.<anonymous closure> (dart:async/runtime/libtimer_patch.dart:21:15)
E/flutter (13860): #23     _Timer._runTimers (dart:isolate/runtime/libtimer_impl.dart:382:19)
E/flutter (13860): #24     _Timer._handleMessage (dart:isolate/runtime/libtimer_impl.dart:416:5)
E/flutter (13860): #25     _RawReceivePortImpl._handleMessage (dart:isolate/runtime/libisolate_patch.dart:165:12)
Syncing files to device Android SDK built for x86...         0.9s

🔥  To hot reload your app on the fly, press "r". To restart the app entirely, press "R".
An Observatory debugger and profiler on Android SDK built for x86 is available at: http://127.0.0.1:8111/
For a more detailed help message, press "h". To quit, press "q".

flutter doctor -v shows the following:

[✓] Flutter (Channel master, v0.4.5-pre.52, on Linux, locale en_US.UTF-8)
    • Flutter version 0.4.5-pre.52 at /home/andrei/opt/flutter
    • Framework revision 8717a5e6f7 (2 days ago), 2018-05-19 03:19:35 +0300
    • Engine revision 1179c38a42
    • Dart version 2.0.0-dev.55.0.flutter-43635d3372

[✓] Android toolchain - develop for Android devices (Android SDK 27.0.3)
    • Android SDK at /home/andrei/Android/Sdk
    • Android NDK at /home/andrei/Android/Sdk/ndk-bundle
    • Platform android-27, build-tools 27.0.3
    • ANDROID_HOME = /home/andrei/Android/Sdk
    • Java binary at: /opt/android-studio/jre/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1024-b01)
    • All Android licenses accepted.

[✓] Android Studio (version 3.1)
    • Android Studio at /opt/android-studio
    • Flutter plugin version 24.2.1
    • Dart plugin version 173.4700
    • Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1024-b01)

[✓] Connected devices (1 available)
    • Android SDK built for x86 • emulator-5554 • android-x86 • Android 8.0.0 (API 26) (emulator)

• No issues found!

What I really don't understand is why the same code is running without problem under Dart VM with the same dart version as above...

Thanks for the help.

Documentation: How do I update a record?

I hope this is the correct place to ask my question - if I should rather post it on stackoverflow, please tell me and I will do so!

I am currently playing around with Sembast in Flutter and I am trying to use it to store bookmarks in a Manga reader app.

I am writing bookmarks like this:

var newRecord = new Record(bookmarkStore, {
  'mangaId': bookmark.mangaId,
  'chapterId': bookmark.chapterId,
  'page': bookmark.page,
});

await db.inTransaction(() async {
  await db.putRecord(newRecord);
});

But I haven't found any way to update a record (I only want to store the latest bookmark for each mangaId - I tried first deleting the old record like this:

Record record = await bookmarkStore.findRecord(new Finder()
    ..filter = new Filter.equal("mangaId", bookmark.mangaId));

if (record != null) {
  debugPrint('deleting old bookmark');
  await db.delete(record.key);
}

But somehow the old record still remains in the database...

Should use deep equality for equals filter?

I've found out that I can't match against values that are List or Map for equals Filter because it is using ==

see filter.dart:

88    switch (operation) {
89      case FilterOperation.equals:
90        return fieldValue == value;
        ...

Can it be

DeepCollectionEquality().equals(fieldValue, value)

instead? Thank you!

About relationship with tables?

I dont know more about Nosql db.
I have some quesitons.

store = table? Can we create multible stores?
How do we relate between tables?

is there support for custom primary keys?

I am storing items from an API response that already have a primary key attached. is there a way to make sembast use that key instead of the ones it auto generates?

Unhandled Exception: Invalid argument(s)

Hi,
First of all thank you so much for such a great library. I am getting this exception when i am trying to store my objects in DB. I am using json_serializable package for serialization, but whenever i try to add something in DB it throws an exception. Below are my model classes:

Sembast Version:

  sembast: ^1.16.0+3

Job:

import 'package:cubivue_app/models/jobsummary/job_summary.dart';
import 'package:cubivue_app/models/task/task.dart';

import 'package:json_annotation/json_annotation.dart';

/// This allows the `User` class to access private members in
/// the generated file. The value for this is *.g.dart, where
/// the star denotes the source file name.
part 'job.g.dart';

/// An annotation for the code generator to know that this class needs the
/// JSON serialization logic to be generated.
@JsonSerializable(anyMap: true)
class Job {
  String headerSubTitleLeft;
  String headerSubTitleRight;
  String headerTitleLeft;
  String headerTitleRight;
  String jobId;
  String jobTileSubTitle;
  String jobTileTitle;
  JobSummary jobSummary;
  List<Task> tasks;

  Job({
    this.headerSubTitleLeft,
    this.headerSubTitleRight,
    this.headerTitleLeft,
    this.headerTitleRight,
    this.jobId,
    this.jobSummary,
    this.jobTileSubTitle,
    this.jobTileTitle,
    this.tasks,
  });

  /// A necessary factory constructor for creating a new User instance
  /// from a map. Pass the map to the generated `_$UserFromJson()` constructor.
  /// The constructor is named after the source class, in this case User.
  factory Job.fromJson(Map<String, dynamic> json) => _$JobFromJson(json);

  /// `toJson` is the convention for a class to declare support for serialization
  /// to JSON. The implementation simply calls the private, generated
  /// helper method `_$UserToJson`.
  Map<String, dynamic> toJson() => _$JobToJson(this);
}

Task:

import 'package:cubivue_app/models/actions/action.dart';

import 'package:json_annotation/json_annotation.dart';

/// This allows the `User` class to access private members in
/// the generated file. The value for this is *.g.dart, where
/// the star denotes the source file name.
part 'task.g.dart';

/// An annotation for the code generator to know that this class needs the
/// JSON serialization logic to be generated.
@JsonSerializable(anyMap: true)
class Task {
  List<TaskAction> actions;
  String subTitle;
  String taskListId;
  String taskId;
  String title;

  Task({
    this.actions,
    this.subTitle,
    this.taskListId,
    this.taskId,
    this.title,
  });

  /// A necessary factory constructor for creating a new User instance
  /// from a map. Pass the map to the generated `_$UserFromJson()` constructor.
  /// The constructor is named after the source class, in this case User.
  factory Task.fromJson(Map<String, dynamic> json) => _$TaskFromJson(json);

  /// `toJson` is the convention for a class to declare support for serialization
  /// to JSON. The implementation simply calls the private, generated
  /// helper method `_$UserToJson`.
  Map<String, dynamic> toJson() => _$TaskToJson(this);
}

Job Summary:

import 'package:flutter/foundation.dart';

import 'package:json_annotation/json_annotation.dart';

/// This allows the `User` class to access private members in
/// the generated file. The value for this is *.g.dart, where
/// the star denotes the source file name.
part 'job_summary.g.dart';

/// An annotation for the code generator to know that this class needs the
/// JSON serialization logic to be generated.
@JsonSerializable(anyMap: true)
class JobSummary {
  final String date;
  final String name;
  final String phone;
  final String time;

  JobSummary({
    this.date,
    this.name,
    this.phone,
    this.time,
  });

  /// A necessary factory constructor for creating a new User instance
  /// from a map. Pass the map to the generated `_$UserFromJson()` constructor.
  /// The constructor is named after the source class, in this case User.
  factory JobSummary.fromJson(Map<String, dynamic> json) =>
      _$JobSummaryFromJson(json);

  /// `toJson` is the convention for a class to declare support for serialization
  /// to JSON. The implementation simply calls the private, generated
  /// helper method `_$UserToJson`.
  Map<String, dynamic> toJson() => _$JobSummaryToJson(this);
}

Action:

import 'package:json_annotation/json_annotation.dart';

/// This allows the `User` class to access private members in
/// the generated file. The value for this is *.g.dart, where
/// the star denotes the source file name.
part 'action.g.dart';

/// An annotation for the code generator to know that this class needs the
/// JSON serialization logic to be generated.
@JsonSerializable(anyMap: true)
class TaskAction {
  String actionId;
  String actionTypeId;
  String caption;
  bool required;

  TaskAction({
    this.actionId,
    this.actionTypeId,
    this.caption,
    this.required,
  });

  /// A necessary factory constructor for creating a new User instance
  /// from a map. Pass the map to the generated `_$UserFromJson()` constructor.
  /// The constructor is named after the source class, in this case User.
  factory TaskAction.fromJson(Map<String, dynamic> json) => _$TaskActionFromJson(json);

  /// `toJson` is the convention for a class to declare support for serialization
  /// to JSON. The implementation simply calls the private, generated
  /// helper method `_$UserToJson`.
  Map<String, dynamic> toJson() => _$TaskActionToJson(this);
}

I tried converting my json manually and it worked. The problem occurs only when i use json_serializable. Help would be appreciated :)

DB Call

  Future<int> insert(Job job) async {
    print('[JobDataSource][insert] insert job: ${job.jobId}');
    return await _jobsStore.add(await _db, job.toJson());
  }

Exception:

ERROR:flutter/lib/ui/ui_dart_state.cc(148)] Unhandled Exception: Invalid argument(s): value Instance of 'JobSummary' unsupported type JobSummary
E/flutter (31372): #0      cloneValue (package:sembast/src/utils.dart:174:3)
E/flutter (31372): #1      cloneValue.<anonymous closure> (package:sembast/src/utils.dart:157:49)
E/flutter (31372): #2      MapMixin.map (dart:collection/maps.dart:163:28)
E/flutter (31372): #3      cloneValue (package:sembast/src/utils.dart:156:18)
E/flutter (31372): #4      makeImmutableRecord (package:sembast/src/record_impl.dart:313:45)
E/flutter (31372): #5      SembastStore.txnPutRecordSync (package:sembast/src/store_impl.dart:449:25)
E/flutter (31372): #6      SembastStore.txnPutSync (package:sembast/src/store_impl.dart:107:14)
E/flutter (31372): <asynchronous suspension>
E/flutter (31372): #7      SembastStore.txnAdd (package:sembast/src/store_impl.dart:93:19)
E/flutter (31372): <asynchronous suspension>
E/flutter (31372): #8      _SembastStoreRef&Object&StoreRefMixin.add.<anonymous closure> (package:sembast/src/store_ref_impl.dart:73:12)
E/flutter (31372): #9      SembastDatabase.inTransaction.<anonymous closure> (package:sembast/src/database_impl.dart:1064:34)
E/flutter (31372): #10     SembastDatabase.transaction.<anonymous closure>.<anonymous closure> (package:sembast/src/database_impl.dart:911:57)
E/flutter (31372): #11     new Future.sync (dart:async/future.dart:224:31)
E/flutter (31372): #12     SembastDatabase.transaction.<anonymous closure> (package:sembast/src/database_impl.dart:911:24)
E/flutter (31372): <asynchronous suspension>
E/flutter (31372): #13     BasicLock.synchronized (package:synchronized/src/basic_lock.dart:31:26)
E/flutter (31372): <asynchronous suspension>
E/flutter (31372): #14     SembastDatabase.transaction (package:sembast/src/database_impl.dart:897:28)
E/flutter (31372): <asynchronous suspension>
E/flutter (31372): #15     SembastDatabase.inTransaction (package:sembast/src/database_impl.dart:1064:7)
E/flutter (31372): #16     _SembastStoreRef&Object&StoreRefMixin.add (package:sembast/src/store_ref_impl.dart:69:25)
E/flutter (31372): <asynchronous suspension>
E/flutter (31372): #17     JobDataSource.insert (package:cubivue_app/data/local/datasources/job/job_datasource.dart:20:29)
E/flutter (31372): <asynchronous suspension>
E/flutter (31372): #18     JobDataSource.insertMany.<anonymous closure> (package:cubivue_app/data/local/datasources/job/job_datasource.dart:25:34)
E/flutter (31372): #19     List.forEach (dart:core-patch/growable_array.dart:278:8)
E/flutter (31372): #20     JobDataSource.insertMany (package:cubivue_app/data/local/datasources/job/job_datasource.dart:25:17)
E/flutter (31372): <asynchronous suspension>
E/flutter (31372): #21     Repository.getMyJobs (package:cubivue_app/data/repository.dart:259:10)
E/flutter (31372): <asynchronous suspension>
E/flutter (31372): #22     _JobStore.getMyJobs (package:cubivue_app/stores/job/job_store.dart:76:17)
E/flutter (31372): <asynchronous suspension>
E/flutter (31372): #23     JobStore.getMyJobs.<anonymous closure> (package:cubivue_app/stores/job/job_store.g.dart:136:51)
E/flutter (31372): #24     _rootRun (dart:async/zone.dart:1124:13)
E/flutter (31372): #25     _ZoneDelegate.run (dart:async/zone.dart:711:19)
E/flutter (31372): #26     AsyncAction._run (package:mobx/src/api/async/async_action.dart:42:29)
E/flutter (31372): #27     _CustomZone.run (dart:async/zone.dart:1021:19)
E/flutter (31372): #28     AsyncAction.run (package:mobx/src/api/async/async_action.dart:25:26)
E/flutter (31372): <asynchronous suspension>
E/flutter (31372): #29     JobStore.getMyJobs (package:cubivue_app/stores/job/job_store.g.dart:136:35)
E/flutter (31372): #30     _AllJobsScreenState.initState.<anonymous closure> (package:cubivue_app/ui/allJobs/all_jobs.dart:28:18)
E/flutter (31372): #31     new Future.delayed.<anonymous closure> (dart:async/future.dart:316:39)
E/flutter (31372): #32     _rootRun (dart:async/zone.dart:1120:38)
E/flutter (31372): #33     _CustomZone.run (dart:async/zone.dart:1021:19)
E/flutter (31372): #34     _CustomZone.runGuarded (dart:async/zone.dart:923:7)
E/flutter (31372): #35     _CustomZone.bindCallbackGuarded.<anonymous closure> (dart:async/zone.dart:963:23)
E/flutter (31372): #36     _rootRun (dart:async/zone.dart:1124:13)
E/flutter (31372): #37     _CustomZone.run (dart:async/zone.dart:1021:19)
E/flutter (31372): #38     _CustomZone.bindCallback.<anonymous closure> (dart:async/zone.dart:947:23)
E/flutter (31372): #39     Timer._createTimer.<anonymous closure> (dart:async-patch/timer_patch.dart:21:15)
E/flutter (31372): #40     _Timer._runTimers (dart:isolate-patch/timer_impl.dart:382:19)
E/flutter (31372): #41     _Timer._handleMessage (dart:isolate-patch/timer_impl.dart:416:5)
E/flutter (31372): #42     _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:171:12)

Way to find items by Filter.matches() with ignore case

is there a way to filter out the data, one the basis of some match in the name of item, yes there is
Filter.matches() works for it, but it really does not allow any thing like enable/disabling case sensitive check. or am I missing something?

Store uint8 array?

Is it possible to store an array of uint8 (byte array) to this database?
I can just encode a given data to base64 to store in string but it would be nice to have this type as I use protocol buffer.

Record exists but cannot update values.

Trying to update records with custom key. I'm trying to update the "season" key inside my record.
It finds the records and returns a CastMap in the response.
But there is no change in the values.

The comments in the .update() says

/// If it does not exist, return null. if value is a map, keys with dot values
/// refer to a path in the map, unless the key is specifically escaped

First I didn't get what does it means by dot values? How do I structure the path?

Secondly, what does it mean by key. Is it record key or store json key? If it means record key, then I'll probably need to update the entire json value right? Anyway to avoid that?

var store = intMapStoreFactory.store(movieData.contentType);
      Map<String, dynamic> jsonValues = {"seasons": movieData.seasonObject};
      sembastDB.db.transaction((txn) async {
        var response =
            await store.record(int.parse(movieData.id)).update(txn, jsonValues);
        print('Response -> $response');
        print('Response -> ${response.runtimeType}');
      });

This is my original record saved in the database

{
  "id": "48866",
  "title": "The 100",
  "overview": "100 years in the future, when the Earth has been abandoned due to radioactivity, the last surviving humans live on an ark orbiting the planet — but the ark won't last forever. So the repressive regime picks 100 expendable juvenile delinquents to send down to Earth to see if the planet is still habitable.",
  "posterurl": "https://image.tmdb.org/t/p/w300//wBzNjurA8ijJPF21Ggs9nbviIzi.jpg",
  "posterpath": "/posters/48866.jpg",
  "backdropurl": "https://image.tmdb.org/t/p/w780//qYTIuJJ7fIehicAt3bl0vW70Sq6.jpg",
  "backdroppath": "/backdrops/48866.jpg",
  "latest_episode_air_date": "2019-05-07",
  "latest_episode_to_air": {
    "episode_number": 2,
    "id": 1686688
  },
  "season_length": 6,
  "episodes_length": "",
  "status": "Returning Series",
  "type": "",
  "episode_runtime": 43,
  "last_watched": "",
  "seasons": {
    "1": [
      1,
      2,
      3,
      4,
      5,
      6,
      7,
      8,
      9,
      10,
      11,
      12,
      13
    ],
  }
}

Bulk Update support

Is it possible to make a bulk update based on a query / filter ?
By example setting active value to true for all records with macAddress = '123'

How to insert a record with custom key?

var store = intMapStoreFactory.store();
var key = await store.add(db, {'offline': true});
var value = await store.record(key).get(db);

The above method generates an int key.
How do I create my own key ? and use it to get the record ?

Bad UTF-8 encoding 0x0 (at offset 32) error when initialising database using databaseFactoryIo.openDatabase(dbPath)

When I run final database = await databaseFactoryIo.openDatabase(dbPath); to initialise my database I get the following error in iOS. Works fine in Android.

[VERBOSE-2:ui_dart_state.cc(148)] Unhandled Exception: FormatException: Bad UTF-8 encoding 0x0 (at offset 32) _Utf8Decoder.convert (dart:convert/utf.dart:444:15) _Utf8ConversionSink.addSlice (dart:convert/string_conversion.dart:309:14) _Utf8ConversionSink.add (dart:convert/string_conversion.dart:305:5) _ConverterStreamEventSink.add (dart:convert/chunked_conversion.dart:72:18) _SinkTransformerStreamSubscription._handleData (dart:async/stream_transformers.dart:120:24) _rootRunUnary (dart:async/zone.dart:1132:38) _CustomZone.runUnary (dart:async/zone.dart:1029:19) _CustomZone.runUnaryGuarded (dart:async/zone.dart:931:7) _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:336:11) _BufferingStreamSubscription._add (dart:async/stream_impl.dart:263:7) _SinkTransformerStreamSubscription._add (dart:async/stream_transformers.dart:68:11) _EventSinkWrapper.add (dart:async/stream_transformers.dart:15:11<…>

Filtering on a Field that is an Array

Hi there!

Is it possible to filter on a field that's an array? For instance, let's say I have a field called TextArray, and in all of my records, that field contains 3 string elements (different sentences). Would I be able to make a Filter that matches by Regular Expression searching that TextArray field?

Thank you, and great work, by the way!

does it possible merge or join store for Finder??

I have 2 stores in aaa.db
ex) StoreRef('dataA') , StoreRef('dataB')

For sortorders value
when i use Finder I'd like to instance merge(like snapshot) stores to one
How can i instance merge or join stores for Finder?

I think two ways like this;;
but not working..

ex1)
var twoStoreMergeForFind = await StoreRef('dataA', dataB').find(db, finder: finder);

ex2)
var myDatas = ['dataA', 'datab'];
var storeMergeTwoData = StoreRef((joinAll(myDatas)));
var twoStoreMergeForFind = await storeMergeTwoData.find(db, finder: finder);

does it possbile join or merge mutiple stores for Finder ???

Multiple entries of the same record on the db

I'm unable to execute put nor update without generating additional entries on the database. I've tried the following:

`import 'package:path/path.dart';
import 'package:sembast/sembast.dart';
import 'package:sembast/sembast_io.dart';

main(List arguments) async {
Database db = await databaseFactoryIo
.openDatabase(join(".dart_tool", "sembast", "example", "record_demo.db"));

int key = 1;
StoreRef<int, Map<String, dynamic>> store = intMapStoreFactory.store();
await store.record(key).put(db, {'offline': true}, merge: true);
await store.record(key).put(db, {'offline': false}, merge: true);
}
`

This generated in the db:

{"version":1,"sembast":1} {"key":1,"value":{"offline":true}} {"key":1,"value":{"offline":false}}

As far as I understand this shouldn't be happen. This is irrespective of the type of key int or String. I've spotted this with a String key but it seems that int is the same.

Update issues

Hey

Before I start, amazing work! This library works really well.

I'm struggling somewhat with my updates though. I've tried a few ways and it us probably me being dumb. I have successfully created a local repository that can add, get, delete. I seem to be stuck on update though and cannot figure out if it's the way I am doing things?

Here is how I successfully store a record:-

@override Future<ApiResponse> add(UserFood item) async { final Map<String, dynamic> values = convertItemToStorageMap(item); final Record recordToAdd = Record(_store, values, item.objectId); final Record recordFromDB = await _db.putRecord(recordToAdd); return ApiResponse(true, 200, convertRecordToItem(recordFromDB), null); }

As you can see I am storing an object as a map, using the objectId as a key to store my record to be retrieved.

Here is how I get:-

@override Future<ApiResponse> getById(String id) async { final Record record = await _store.getRecord(id); if (record != null) { final UserFood userFood = convertRecordToItem(record); return ApiResponse(true, 200, userFood, null); } else { return errorResponse; } }

Now the issue seems to be with update, I have tried this in similar ways but cannot figure this out. I have tried using _db.update, and _store update but both just do not work.

'@OverRide
Future update(UserFood item) async {
final Map<String, dynamic> values = convertItemToStorageMap(item);
final Record recordToUpdate = Record(_store, values, item.objectId);
final Record recordFromDb = await _db.update(recordToUpdate, item.objectId);

if (recordFromDb == null) {
  return errorResponse;
}

return ApiResponse(true, 200, recordFromDb.value, null);

It either returns null or that it cannot find the record. Is update suppose to return null or return the value you have tried to store?

Question: Does Sembast support indexing?

Hi,

This DB is what I was looking for, but there are not documentation about performance or the posibility of index some properties. In my app, I have to store around of 500, 000 documents, and they have to be indexed, I'm now using Jaguar Orm that uses SQLite as DB engine for this purpose, but the implementation is so so tricky and I want to use Sembast. So the question is: Do you have or will you have some mechanism to index the storage, or this is not necessary and it will have good performance in this scenario?

Usage as a event sourcing storage

Hey, I've been looking at this project with an interest in using it as a event sourcing storage for small datasets - in the order of thousands of records per database.

My idea is to synchronize the small DB between different users of the same database (just a few users/db) via events so it's never necessary to send the whole database to new users except first time they connect... so users should see each other's changes, which should happen infrequently...

Do you think this database would fit the bill? I like that it is written entirely in Dart, so I don't need to rely on external software and could potentially use both on web clients and mobile phones.

Thanks for making this project public and open source by the way, the idea is nice and the code, from what I've seen, of high quality!

meta dependency

Hi!
A quick one: your 1.13.3 depends on meta ^1.1.7 but Flutter is still on 1.1.6 (not above), which makes the version solving to fail, for Flutter users.
Workaround is to use your sembast dependency at ^1.13.2 while waiting for Flutter to upgrade its meta dependency...
All the best,

Filtering an array property

Can I filter something like Filter.matches("words.text", "^1", anyInList: true) ?

{
  "words": [
    "a": {
      "text": "123"
    },
    "b": {
      "text": "456"
    }
  ]
}

I can't filter data with EnumType.

I have i lot of records in my store. I want seperate them with enum type. I can save it to db but when i want to filter them it doesnt show me any result.

Enum Type :

enum ItemState { health, diaper, sleep, measure, feed }

  Future<List<WeightNHeight>> getAllSortedByName() async {
    final babyId = await Preferences.getId();
    final finder = Finder(
      filter: Filter.and([
        Filter.equal("babyId", babyId),
        Filter.equal('state', ItemState.measure),
      ]),
      sortOrders: [
        SortOrder("date", false),
      ],
    );

    final recordSnapshots = await _itemStore.find(
      await _db,
      finder: finder,
    );
    return recordSnapshots.map((snapshot) {
      final items = WeightNHeight.fromMap(snapshot.value);
      // An ID is a key of a record from the database.
      items.id = snapshot.key;
      return items;
    }).toList();
  }

Problem with DB encryption on Release (1.5.4)

Hi,
I was using the sembast with the xxtea codec and it was working great for debug/release on the latest stable version of flutter (1.2.1). But after upgrading to the latest stable version (1.5.4) the release version doesn't work it throws [2] Invalid codec signature when I open the DB. Do you have any idea why this is happening?
Thank you!

Initial value during subscription

When I am subscribed to the data
_store.query(finder: finder).onSnapshots(_db))
is it possible somehow receive surrent snapshot as initial data? I get notified only after changes to the store

Thanks,

Encyption codec verification does not work correctly

It seems like the process for codec verification is this:

When the database is opened with the codec, encrypt {'signature': ${codec.signature}} with the codec, and store that value.
The next time the database is opened, calculate the signature on the passed value and compare to the stored value. If there is a mismatch, stop.

The problem is that this approach only works if the encryption algorithm always produces the same output when given the same input, which is actually a very undesirable property for an encryption algorithm to have. This rules out any algorithm that uses a random salt, for instance.

Proposed solutions:

compare the bare text signatures passed, rather than the encrypted versions
Switch (2) above to the following: decrypt the stored signature with the passed codec and compare to the passed signature.

{ 
 id: string,  // it's uuid string
 name: string,
 type: string
}