hhblaze / dbreeze Goto Github PK

View Code? Open in Web Editor NEW

512.0 43.0 57.0 712.55 MB

C# .NET NOSQL ( key value store embedded ) ACID multi-paradigm database management system.

License: BSD 2-Clause "Simplified" License

C# 99.96% Batchfile 0.04%

nosql database embedded c-sharp text search search-engine net acid transaction

dbreeze's People

Contributors

Stargazers

Watchers

Forkers

enginekit dylanliu1 notyours180 vebin hakanl starofflame sangkyunyoon tombert hbulzy dronab wondial codemasterblackperl veridit codefork noid-protocol xtmhm2000 guojianbin harikiraninfosys kouweizhong wangchengqun rallyfoundation mpvyard axzxs2001 awesomedotnetcore valmac ikvm sdyiheng jerrygaohk shaikhanas1993 lvzhuye husseinzahran iracding zapoutix erisonliang eksotama dystudio zendbit hthubert nielshenriksen arthuryil cm4ker devbox10 nanofabricfx cool8868 jangocheng lanicon seanz617 bubdm sosodennis luca-g ahmadfikrimasyhur waynemunro piyongcai bangush itfenom fredatgithub ramankheti

dbreeze's Issues

System.OverflowException when backward iterating

We are getting OverflowExceptions intermittently. It does not happen often but it's been reproduced a number of times by different machines meaning it is not an isolated incident. You'll find the stack below.

We're using <string, byte[]> for key/value store (string for key, byte[] since it's a custom serializer using JSON.NET).

Only occurs during execution. If we reload the database from the disk, the error no longer occurs.

System.OverflowException: Arithmetic operation resulted in an overflow.
   at DBreeze.Storage.FSR.Table_Read(Boolean useCache, Int64 offset, Int32 count) in c:\dev\DBreeze\DBreeze\Storage\FSR.cs:line 1040
   at DBreeze.Storage.FSR.Table_Read(Boolean useCache, Byte[] offset, Int32 count) in c:\dev\DBreeze\DBreeze\Storage\FSR.cs:line 854
   at DBreeze.Storage.StorageLayer.Table_Read(Boolean useCache, Byte[] offset, Int32 quantity) in c:\dev\DBreeze\DBreeze\Storage\StorageLayer.cs:line 134
   at DBreeze.LianaTrie.LTrieWriteCache.ReadKey(Boolean useCache, Byte[] pointer) in c:\dev\DBreeze\DBreeze\LianaTrie\LTrieWriteCache.cs:line 1187
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 70
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<ItBwd>d__0.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 92
   at DBreeze.LianaTrie.Iterations.Backward.<IterateBackward>d__10.MoveNext() in c:\dev\DBreeze\DBreeze\LianaTrie\Iterations\Backward.cs:line 176
   at DBreeze.Transactions.Transaction.<SelectBackward>d__2b`2.MoveNext() in c:\dev\DBreeze\DBreeze\Transactions\Transaction.cs:line 1520

Getting error on UWP (RPi Win 10 IoT)

I'm getting this error when starting my app.
Failed during startup: DBreeze.Exceptions.DBreezeException: Creation of the database folder failed! ---> DBreeze.Exceptions.DBreezeException: Database is not operable, please find out the problem and restart the engine! FSR INIT FAILED: C:\Data\Users\DefaultAccount\AppData\Local\Packages\IoStorm_mr3ez18jctte6\LocalCache\Cache_DBreezeSchema ---> System.IO.IOException: IO_SharingViolation_File, C:\Data\Users\DefaultAccount\AppData\Local\Packages\IoStorm_mr3ez18jctte6\LocalCache\Cache_DBreezeSchema

Any clue? This has worked before, not sure exactly what have changed, I upgraded to DBreeze 1.84 and I'm on build 15063 of Windows 10 IoT.

Full Text

I have 2 full text questions.

How to get it to work in the first place. In one block of code I've added names to a text name search and in another part I do the search
// the insert
tran.TextInsert("AcctTextNameSearch", item.AccountID.ToBytes(), item.FinEntity + " " + item.Contact.Name1.ToLower() + " " + item.Contact.Name2.ToLower(), "");

// the search - returns nothing although I confirmed data - the AName parameter is sent as lower
public void NameSearch(string AName)
{
using (var tran2 = engine.GetTransaction())
{
var tsm = tran2.TextSearch("AcctTextNameSearch");
foreach (var w in tsm.BlockAnd(AName).GetDocumentIDs())
{
Console.WriteLine(w.ToString());
}
}
}

I'm not familiar with how to "build" the search criteria not using the fluent style
So I'm wondering how to build multiple conditions. So let's say I have fields to search on name, address, city, state, zip and any or all of them can be filled in. If they are left blank I want to exclude them but if the user entered data to search, I want to add it to the text search

Thanks,
Bill

implement meaningful exception messages

Currently, all db related exceptions display almost the same message and tells to restart the engine. It would be good practice to implement full error messages. Moreover, it will help in debugging.

is there any GUI viewer for data files?

Document Growth

Is there a way to allow for document growth? I'm designing models and determining best way to go.

Either to not worry about growth of a document as it is written to new location and then copy/reduce the database periodically
or
Separate the document detail lines from the document but then lose the fetch of a whole document in one pass.

Any experience on which provides best performance? Or is there a "best practice"?

Safe Backup of the Database files

How do we get a safe external backup of the DB?

By "safe" I mean, if the DB is in use, then we want to make a snapshot of the data files while there are no updates in progress. Because the DB is spread across multiple files, we could potentially save the files out of sync.

I see the internal incremental backup options, but I'd like to backup with a standard system, like command prompt backup of DB.

.Net core support

Hello, we are talking about your DB here. thangchung/awesome-dotnet-core#94 (comment)
What are your thoughts?

Remove binaries from repo

Binaries (like bin and pdb files, etc) don't belong in the git repo.

Using multible DBreezeIndexes creates multiple entries in the DB

So let's suppose we have a Database of Songs and I use these indexes:

Indexes = new List<DBreezeIndex>
{
                        new DBreezeIndex(1, record.Path) { PrimaryIndex = true }, //PI Primary Index
                       new DBreezeIndex(2, record.FolderPath), //SI - Secondary Index
                       new DBreezeIndex(3, record.LeadArtist),
                       new DBreezeIndex(4, record.Album),
                       new DBreezeIndex(5, record.Title),
}

There are *5 Indexes here and so DBreeze adds them 5 times. So when I go on and retrieve my songs using this:

var records = tran.SelectForward<byte[], byte[]>(tableName);

I get every song x 5 and that messes things up not to mention the RAM and CPU Load.

Is there any way to avoid this without decreasing the number of Indexes?

The other question I have is this: How do I query for songs by their album, title, artist?

Thank you providing such an awesome and efficient database engine!

Update:
So, I just ran a test and it seems the values are being duplicated 5 times with the same key (primary key by the looks of it).
Update 2:
Ran another test and I was wrong. The key wasn't the same but the primary key was being appended to secondary indexes. I turned that off and now the indexes are all separate. Still, this doesn't solve the main problem of multiple entries.

Custom Comparator

Is it possible to use a custom comparator for my objects.
I use DBreeze just to store <byte[], byte[]>, but I have a special order to maintain.

Thanks for this great project!

System.IO.Compression.ZLibException: 'The underlying compression routine could not be loaded correctly.' in UWP

This exception occurs on Windows 10 Mobile:
DllNotFoundException: Unable to load DLL 'clrcompression.dll': The specified module could not be found. (Exception from HRESULT: 0x8007007E)

On this line of code:

tran.TextInsert(TextTableName, record.Id.To_8_bytes_array_BigEndian(), record.GetTextSearchKey());

Here are some references:
http://stackoverflow.com/questions/41185291/unzipping-throws-an-the-underlying-compression-routine-could-not-be-loaded-corr

RemoveKey - Unable to get it to work?

HI I've got a section of code (see below) that uses RemoveKey to remove a single record from the DB.

    public void DeleteKey(string tableCode, long itemId)
    {
        using (var t = engine.GetTransaction())
        {
            t.SynchronizeTables(tableCode, "TS_" + tableCode, tableCode + "LANG");

            NBrightData nbi1 = null;
            var obj = t.Select<byte[], byte[]>(tableCode, DBreezeIdx.idx_ItemId.ToIndex(itemId)).ObjectGet<NBrightData>();
            if (obj != null)
            {
                nbi1 = (NBrightData)obj.Entity;
            }
            Console.WriteLine(nbi1.ItemId + " " + nbi1.Lang + " " + nbi1.ModifiedDate + " " + nbi1.ModuleId + " " + nbi1.ParentItemId + " " + nbi1.PortalId + " " + nbi1.Key + " " + nbi1.TableCode + " " + nbi1.TextData + " " + nbi1.UserId + " " + nbi1.XrefItemId + Environment.NewLine);

            t.RemoveKey<long>(tableCode, itemId);
            t.Commit();

            obj = t.Select<byte[], byte[]>(tableCode, DBreezeIdx.idx_ItemId.ToIndex(itemId)).ObjectGet<NBrightData>();
            if (obj != null)
            {
                nbi1 = (NBrightData)obj.Entity;
            }
            Console.WriteLine(nbi1.ItemId + " " + nbi1.Lang + " " + nbi1.ModifiedDate + " " + nbi1.ModuleId + " " + nbi1.ParentItemId + " " + nbi1.PortalId + " " + nbi1.Key + " " + nbi1.TableCode + " " + nbi1.TextData + " " + nbi1.UserId + " " + nbi1.XrefItemId + Environment.NewLine);

        }
    }

The above code runs OK, but I get the record existing before the Remove and also after...

OUTPUT

DELETE 3 -----------------------------------

3 01/01/2017 00:00:00 -1 0 1 R3 DATA text 1 0

I've tried multiple veriations of code with commit and table sync. But the read of the table always returns the record which should have bene removed?

In 1 situation I've have the code throw an exception, telling me the transaction doesn't exits, but the read still works and displayed the record 3.

I can't find any working examples for the RemoveKey,

Can you tell me what I'm doing wrong?

Performance questions

As I work in performance critical environment where every millisecond counts, I have the following questions after thoroughly reading the Documentation.

If I want to retrieve all the records (no filters or anything) really, really quickly what will be faster? SelectForward, SelectForwardStartsFrom with int.MinValue or SelectForwardFromTo with int.MinValue & int.MaxValue?
If I understand correctly, the DBCustomSerializer serializes each object before inserting right? And deserializes each object on ObjectGet, am I right? I have observed (through performance profiling) that this takes way too much CPU as it is serializing & deserializing again and again. Is there any way to make this less CPU expensive (for example, serializing & deserializing only once or not at all?)
How do I remove a record completely from the database? As I understand from the documentation, the row is just set to invisible but if I want to remove it completely, what would I have to do? (I understand this can be a performance heavy operation).
Any advice regarding how to decrease the overall DB size without compromising performance?
Is ObjectGetNewIdentity an expensive operation (when doing on each new insert) and if so, can it be replaced with some other alternative that is faster?

Thanks.

Indexing large term does not work

If I truncate the string, it works.

[Fact]
        public void TestIndex()
        {
            using (DBreeze.DBreezeEngine engine = new DBreeze.DBreezeEngine(nameof(TestIndex)))
            {
                var id = RandomUtils.GetBytes(20);
                using (var tx = engine.GetTransaction())
                {
                    tx.TextAppend("InvoiceSearch", id, "bb5fcafcd25708af510d7bd3894be6c5c9a4497f92e92864448e82d7e67fde79");
                    tx.Commit();
                }
                using (var tx = engine.GetTransaction())
                {
                    AssertFind(id, tx, "bb5fcafcd25708af510d7bd3894be6c5c9a4497f92e92864448e82d7e67fde79");
                }
            }
        }

        private static void AssertFind(byte[] id, DBreeze.Transactions.Transaction tx, string term)
        {
            var isContained = tx.TextSearch("InvoiceSearch").Block(term)
                                    .GetDocumentIDs()
                                    .Any(docid => docid.SequenceEqual(id));
            Assert.True(isContained, term + " should be contained");
        }

Any chance for a client UI

like Mongo viewer or management studio?

Can DBreeze handle Guid as an index type?

Hello,

Can Guid be used in an index in DBreeze?

We create a new Session object. Once set Session.SessionId is fixed and never changes. SessionLogId is created anew each time SessionLog is instantiated. The unique identifiers from a database point of view are Guid types (SessionId and SessionLogId set by 3rd party).

This is simplified example. There is a one too many relationship between Session and SessionLogs. There is only one unique Session but many SessionLog.


public class Session
{
	public Session(Guid sessionId)
	{
		SessionId = sessionId;
	}

    public Guid SessionId { get; }
}


public class SessionLog
{
	public SessionLog(Guid sessionId)
	{
		SessionId = sessionId;
		SessionLogId  = Guid.NewGuid();
	}
		
	public Guid SessionId { get; }
	public Guid SessionLogId { get; } 
}


//This works. Data is saved correctly. (Serializer not shown)
     public void SaveSession(SessionLog sessionLog)
        {
            try
            {
                using (var tran = databaseEngine.GetTransaction())
                {
                    var status = tran.ObjectInsert("SessionLogs", new DBreezeObject<SessionLog>()
                    {
                        NewEntity = true,
                        Entity = sessionLog,

                        Indexes = new List<DBreezeIndex>
                        {
                              new DBreezeIndex(1, sessionLog.SessionId) { PrimaryIndex = true },
                              new DBreezeIndex(2, sessionLog.SessionLogId), // to make row unique
                        }
                    });

                    tran.Commit();
                }
            }
            catch (Exception ex)
            {             
            }
        }

Table Sessions(Guid SessionId)
"0b86d287-3776-4140-b394-d0d492fb66a4"
"f046d56f-3f70-4555-ba7b-eec884bf323b"

Table SessionsLogs
Sessions(Guid SessionId, Guid SessionLogId)
"0b86d287-3776-4140-b394-d0d492fb66a4", "813663fc-2b4a-45f4-ac5c-93b2e6c63099"
"0b86d287-3776-4140-b394-d0d492fb66a4", "d6d647cd-25fc-4216-93b2-3286ed7c5da8"
"0b86d287-3776-4140-b394-d0d492fb66a4", "9d111f41-868c-4589-9882-8e53701080dd"
"0b86d287-3776-4140-b394-d0d492fb66a4", "813663fc-2b4a-45f4-ac5c-93b2e6c63099"
"f046d56f-3f70-4555-ba7b-eec884bf323b", "60876d6c-cd16-430b-a44b-892ec6be0903"

But how to fetch the data? In SQL I would do this to get the logs from the table shown.

select * from SessionsLogs where sessionId = "0b86d287-3776-4140-b394-d0d492fb66a4"
(4 rows returned)

But I cannot seem to use SelectForwardFromTo as there is no Min and max in Guid. I have tried many Select* variations but I either get 1 row, 0 rows or duplicate rows.

Many Thanks, John

Could not load file or assembly ‘System.Xml.XmlSerializer

On linux I got this exception, I added ref to nuget package System.Xml.XmlSerializer, Version=4.0.11.0 and it seems to have done the trick... unsure why it happens though, your code has no dependency on XmlSerializer.

System.IO.FileNotFoundException: Could not load file or assembly ‘System.Xml.XmlSerializer, Version=4.0.10.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a’. The system cannot find the file specified.
File name: ‘System.Xml.XmlSerializer, Version=4.0.10.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a’
   at DBreeze.Transactions.TransactionsCoordinator.Commit(Int32 transactionThreadId)
   at DBreeze.Transactions.Transaction.Commit()
   at ElementsExplorer.Repository.MarkAsUsed(KeyInformation info) in /home/zwoc/workspace/ElementsExplorer/ElementsExplorer/Repository.cs:line 203
   at ElementsExplorer.Controllers.MainController.<Sync>d__5.MoveNext() in /home/zwoc/workspace/ElementsExplorer/ElementsExplorer/Controllers/MainController.cs:line 46
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.AspNetCore.Mvc.Internal.ObjectMethodExecutor.<CastToObject>d__38`1.MoveNext()

DBreeze 1.75 doesn't work in UWP

I have Visual Studio 2015 U3. If I create a blank Universal Windows Application and add DBreeze (v 1.75) then I'm getting errors restoring the packages:

Restoring packages for 'App6'.
Restoring packages for c:\users\user\documents\visual studio 2015\Projects\App6\App6\project.json...
Package DBreeze 1.75.0 is not compatible with uap10.0 (UAP,Version=v10.0). Package DBreeze 1.75.0 supports:

monoandroid (MonoAndroid,Version=v0.0)
net35 (.NETFramework,Version=v3.5)
net40 (.NETFramework,Version=v4.0)
net45 (.NETFramework,Version=v4.5)
netcoreapp1.0 (.NETCoreApp,Version=v1.0)
One or more packages are incompatible with UAP,Version=v10.0.
Package DBreeze 1.75.0 is not compatible with uap10.0 (UAP,Version=v10.0) / win10-arm. Package DBreeze 1.75.0 supports:
monoandroid (MonoAndroid,Version=v0.0)
net35 (.NETFramework,Version=v3.5)
net40 (.NETFramework,Version=v4.0)
net45 (.NETFramework,Version=v4.5)
netcoreapp1.0 (.NETCoreApp,Version=v1.0)
One or more packages are incompatible with UAP,Version=v10.0 (win10-arm).
Package DBreeze 1.75.0 is not compatible with uap10.0 (UAP,Version=v10.0) / win10-arm-aot. Package DBreeze 1.75.0 supports:
monoandroid (MonoAndroid,Version=v0.0)
net35 (.NETFramework,Version=v3.5)
net40 (.NETFramework,Version=v4.0)
net45 (.NETFramework,Version=v4.5)
netcoreapp1.0 (.NETCoreApp,Version=v1.0)
One or more packages are incompatible with UAP,Version=v10.0 (win10-arm-aot).
Package DBreeze 1.75.0 is not compatible with uap10.0 (UAP,Version=v10.0) / win10-x64. Package DBreeze 1.75.0 supports:
monoandroid (MonoAndroid,Version=v0.0)
net35 (.NETFramework,Version=v3.5)
net40 (.NETFramework,Version=v4.0)
net45 (.NETFramework,Version=v4.5)
netcoreapp1.0 (.NETCoreApp,Version=v1.0)
One or more packages are incompatible with UAP,Version=v10.0 (win10-x64).
Package DBreeze 1.75.0 is not compatible with uap10.0 (UAP,Version=v10.0) / win10-x64-aot. Package DBreeze 1.75.0 supports:
monoandroid (MonoAndroid,Version=v0.0)
net35 (.NETFramework,Version=v3.5)
net40 (.NETFramework,Version=v4.0)
net45 (.NETFramework,Version=v4.5)
netcoreapp1.0 (.NETCoreApp,Version=v1.0)
One or more packages are incompatible with UAP,Version=v10.0 (win10-x64-aot).
Package DBreeze 1.75.0 is not compatible with uap10.0 (UAP,Version=v10.0) / win10-x86. Package DBreeze 1.75.0 supports:
monoandroid (MonoAndroid,Version=v0.0)
net35 (.NETFramework,Version=v3.5)
net40 (.NETFramework,Version=v4.0)
net45 (.NETFramework,Version=v4.5)
netcoreapp1.0 (.NETCoreApp,Version=v1.0)
One or more packages are incompatible with UAP,Version=v10.0 (win10-x86).
Package DBreeze 1.75.0 is not compatible with uap10.0 (UAP,Version=v10.0) / win10-x86-aot. Package DBreeze 1.75.0 supports:
monoandroid (MonoAndroid,Version=v0.0)
net35 (.NETFramework,Version=v3.5)
net40 (.NETFramework,Version=v4.0)
net45 (.NETFramework,Version=v4.5)
netcoreapp1.0 (.NETCoreApp,Version=v1.0)
One or more packages are incompatible with UAP,Version=v10.0 (win10-x86-aot).
Package restore failed for 'App6'.
Package restore failed. Rolling back package changes for 'App6'.
========== Finished ==========
Time Elapsed: 00:00:00.8687082

Huge data folder size overhead in multiple overwrite workload

We use DBreeze 1.083.2017.0302 in our product with custom serializer to a byte array for data and manually split it across few tables (external logic, 6 tables right now). All tables consist of key (int) and byte[] (custom serializer result) pairs.

The workload can be:

Insert new "batch" of rows (rare)
Remove "batch" of rows and insert new ones with different keys (very often).
Overwrite all or part of rows with new data (depends but can be very often).

Write size can vary significantly but a rough average is about 5-15 KB per 1 row in write "batch".
Each write/insert/update row request wrapped to the individual transaction per 1 row, so they, not real batch, just logically grouped operations.
All data in workload goes to one of "manual" tables.
Technical_SetTable_OverwriteIsNotAllowed options is not set and has default value.

We noticed that DBreeze folder size right now around 7 GB but if we count size of stored data it will be just 75 MB (2 466 476 rows).

Are we missing some options / configuration for our workload, or maybe we have to implement some kind of "VACUUM" command?

Why duplicated projects?

I see that lot's of stuff is duplicated accross several projects (#9).

What is the most up to date? Do you want me to make a PR to cleanup this mess?

Question: Number of tables

I need to do a 1 key => N results.
I am using the key as the table name to achieve this purpose. But there might have thousands of such keys.
Does creating that many table is bad for DBreeze ?

Exception times to times

I get this exception time to time. The DB is not corrupted. (restarting the program and running again the command work)

DBreeze.Exceptions.DBreezeException was unhandled by user code
  HResult=-2146233088
  Message=Getting table "@utCoins" from the schema failed!
  Source=DBreeze
  StackTrace:
       at DBreeze.Transactions.TransactionsCoordinator.GetTable_WRITE(String tableName, Int32 transactionThreadId)
       at DBreeze.Transactions.Transaction.GetWriteTableFromBuffer(String tableName)
       at DBreeze.Transactions.Transaction.Insert[TKey,TValue](String tableName, TKey key, TValue value, Byte[]& refToInsertedValue, Boolean& WasUpdated, Boolean dontUpdateIfExists)
       at DBreeze.Transactions.Transaction.Insert[TKey,TValue](String tableName, TKey key, TValue value)
       at Stratis.Bitcoin.FullNode.Consensus.DBreezeCoinView.SaveChanges(ChainedBlock newTip, IEnumerable`1 unspentOutputs) in C:\Users\\Documents\StratisBitcoinFullNode-master (3)\StratisBitcoinFullNode-master\Stratis.Bitcoin.FullNode\Consensus\CoinViews\DBreezeCoinView.cs:line 126
       at Stratis.Bitcoin.FullNode.Consensus.CommitableCoinView.Commit() in C:\Users\\Documents\StratisBitcoinFullNode-master (3)\StratisBitcoinFullNode-master\Stratis.Bitcoin.FullNode\Consensus\CoinViews\CommitableCoinView.cs:line 157
       at Stratis.Bitcoin.FullNode.Consensus.BackgroundCommiterCoinView.<FlushAsync>b__21_0() in C:\Users\\Documents\StratisBitcoinFullNode-master (3)\StratisBitcoinFullNode-master\Stratis.Bitcoin.FullNode\Consensus\CoinViews\BackgroundCommiterCoinView.cs:line 98

It seems to happen here: https://github.com/stratisproject/StratisBitcoinFullNode/blob/master/Stratis.Bitcoin.FullNode/Consensus/CoinViews/DBreezeCoinView.cs#L126

Using overlapped IO for batch insert and query

At least for queries, there is no way to fetch data in batch efficiently by using overlapped IO.
This is needed for several reasons for doing that:

When the DB is on external hard drive, there is some latency. Querying in parallel would divide this latency.
Some hard drive support parallel read. (ie multiple head hard drives, and SSD)
A partition in RAID 0 supports parallel read (stripping)

Unable to run on Windows 10 IoT .NET Core RC2 - RPi

I used to run DBreeze fine on my RPi, but after the RC2 update it doesn't work any more, getting an error on line 27 in Engine\Scheme.cs. This is the exception:

Additional information: Could not load file or assembly 'System.Threading, Version=4.0.10.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)

I tried this in a blank new UWP project. It works if I run it in the x86 emulator, but not when I'm deploying it to RPI2/3 (ARM).

Performance of DBreeze compared to SQLite, LevelDB or LMDB

Can you say something about the performance of DBreeze compared to more common databases like SQLite, LevelDB or LMDB in the same scenario? Is DBreeze competitive?

Compress data?

Hi, I want to store larger arrays (65536) of ushort into a table. Does DBreeze support compression to save disk space or do I have to compress myself?

Terminating errors in product which uses dbreeze

Hi DBreeze Team,

I am using a vendor product (ShareGate) which uses DBreeze. A few times a week I receive a terminating error:

Problem signature: Problem Event Name: PowerShell NameOfExe: powershell.EXE FileVersionOfSystemManagementAutomation: 10.0.10586.117 InnermostExceptionType: System.IO.IOException OutermostExceptionType: DBreeze.Exceptions.DBreezeException DeepestPowerShellFrame: unknown DeepestFrame: DBreeze.DBreezeEngine.InitDb ThreadName: CrossP..0fac55c14b4) OS Version: 6.3.9600.2.0.0.272.7 Locale ID: 1033

The product includes an accompanying message: "DataPortal.Fetch failed (Operation Timed Out)."

I know this is a long shot, but can you provide any insight into what's happening? I have gone through the vendor a few times for more info on how to correct or avoid this error with no luck.

Thanks!

Remove duplicates

It seems that all source files are duplicated into DBreeze.UWP and NETCore folders. It would probably be better to link the files, hard to maintain multiple copies, risky and error-prone.

Source in github or codeplex?

Is the latest/correct source code in Github or Codeplex?

OverflowException

On 1.92.0

Received this report from a user.

Good for logging?

Hi I am currently looking for an embedded solution for logging; more specific, it would need to handle the logs of incoming mobile text messages (sms) in a Callback scenario. So whenever a person sends a message to our virtual number a post is made to an url. What then happens is further business logic, but I want to save the callback info (json object) into Dbreeze. So it's insert only, and the read operations are more for diagnostics. "give me all messages between date x and date y for incoming number z". So I would need to add some indexes on number, date, etc.
So the data will only grow over time, maybe a million records per year.
Would DBreeze a good solution for this? Or should I create different tables per month for example?
Thanks for your advice!

Remove binaries (nupkg) files from GIT

Probably best to not have binaries in GIT. You can use releases in github if you need to preserve these.

Can you also exclude .suo and bin/obj folders, I recommend a good .gitignore to get started.

Feature-Request: Option to Restart DBEngine

How this would help?
In some cases (like when you dispose the DBEngine only to start it again), it would be good to have the option to restart the engine instead.

Current Behaviour:
The DBEngine needs to be disposed and initialized like this:

//_db is the DBEngine
_db.Dispose();
_db =null;
  var dbConfig = new DBreezeConfiguration
                {
                    DBreezeDataFolderName = dbPath,
                    Storage = DBreezeConfiguration.eStorage.DISK
                };
                _db = new DBreezeEngine(dbConfig);

As you can see, it would be good to just have this:

_db.Restart();

The difference will be in NOT creating a new instance on every disposal but just re initializing the old one. Currently there is no method for this or to make this happen in SHORT precise way.

Something else that could be done is to have an InitDB function, which would reduce the above code down to:

_db.Dispose();
_db.InitDB();

Thanks.

Return data pointer in DBreezeObjectInsertResult after call ObjectInsert method

Hi guys,

Is it possible to return direct pointer to inserted data when ObjectInsert method is used? We need to store this pointer in another table to perform synchronization operations. An example is given below:

public void InsertOrUpdateObject<TEntity>(List<TEntity> entityList)
{
string tableName = typeof(TEntity).Name.ToLower();
string syncTableName = tableName + "sync";

using (Transaction tran = _engine.GetTransaction())
{
    tran.SynchronizeTables(tableName, syncTableName);
    foreach (var entity in entityList)
    {
        var id = tran.ObjectGetNewIdentity<long>(tableName);
        List<DBreezeIndex> indexes = GetIndexedByIndicies(entity);

        var newObject = new DBreezeObject<TEntity>
        {
            Indexes = indexes,
            Entity = entity
        };
        DBreezeObjectInsertResult<TEntity> res = tran.ObjectInsert(tableName, newObject);
        //it will be good to have data pointer in DBreezeObjectInsertResult
        tran.Insert<byte[], byte[]>(syncTableName, 1.ToIndex(_synchronizationManager.SyncTransactionId, id), res.DataPointer);
    }
    tran.Commit();
}

}

Regards

Strange performance numbers

I have been looking at DBreeze recently and saw the great insert performance numbers. But I am getting very strange results from a pretty simple benchmark i wrote.

Here is my code

private static void Run(int runCount, string table, DBreezeEngine db)
{
    Console.WriteLine("Testing " + runCount);

    string simulatedRow = $@"C:\This\Is\A\Fake\Directory, {Guid.NewGuid()}, 0.00, 0, 2, 3";

    var sw = Stopwatch.StartNew();

    using (var t = db.GetTransaction())
    {
        for (int i = 0; i < runCount; i++)                
            t.Insert(table, i, simulatedRow);
        t.Commit(); 
    }

    sw.Stop();

    Console.WriteLine("Insert took {0:N2}. Rate = {1:N3}", sw.Elapsed.TotalMilliseconds, runCount / sw.Elapsed.TotalSeconds);
    Console.WriteLine();
}

Starting with an empty directory (using on disk storage), i run the following

(Different tables for each test)

Run(100, "t1", db);
Run(1000, "t2", db);
Run(10000, "t3", db);
Run(100000, "t4", db);
Run(1000000, "t5", db);

and get

Testing 100
Insert took 110.10. Rate = 908.285

Testing 1000
Insert took 80.76. Rate = 12,381.984

Testing 10000
Insert took 113.05. Rate = 88,452.993

Testing 100000
Insert took 405.44. Rate = 246,645.924

Which is great!

But if i run (reusing the same table)

Run(100, "t1", db);
Run(1000, "t1", db);
Run(10000, "t1", db);
Run(100000, "t1", db);

My results are very different. The last test is about 66x slower.

Testing 100
Insert took 133.47. Rate = 749.257

Testing 1000
Insert took 309.03. Rate = 3,235.900

Testing 10000
Insert took 2,686.21. Rate = 3,722.711

Testing 100000
Insert took 26,852.87. Rate = 3,723.996

If find that if i delete the table in between each run the speed goes back to normal.

Is it known that multiple transaction against the same table cause this performance drop or am i seeing something strange?

Tested on Windows 8.1 x64, .Net 4.5.2, AnyCPU

implement Version property for version check

There should be a hardcoded "Version" property and another "GetVersion()" method. Basically, the use case will include checking the DB version against the hardcoded one and performing necessary updates or showing an error that an old version is being used or whatever the client wants.

Creating a new instance of the Engine not thread safe.

A static dictionary is initiated without locks.

Not sure if this is intentional.

See comments in our code: https://github.com/stratisproject/StratisBitcoinFullNode/blob/master/Stratis.Bitcoin/Utilities/DBreezeSingleThreadSession.cs#L85

Which DLL for Mono 5.0?

Hi want to use DBreeze in Mono 5.0, but I don't know which DLL I have to use. I have 10 different DLL's here. Can you tell me, what I can use?

DBreeze single Database File

Hi Dear Developer
I want to save video file to a database
finally after testing many databases, I found DBreeze as the most fastest.
but I have a question.
Is it possible to have one database file like SQLite or LiteDb ? or copy other files to a temp folder ?
my DBreeze DB have more than one file as I've attached

.

Transaction doesn't exist exception

Here's my code for inserting records:

 using (var tran = _engine.GetTransaction())
                {
                    if (records.Any())
                    {
                        tran.Technical_SetTable_OverwriteIsNotAllowed(_tableName); //exception hits here. Removing this DOESN'T solve the issue.

                        foreach (var record in records.ToList())
                        {
                            record.Id = tran.ObjectGetNewIdentity<long>(_tableName);
                            var ir = tran.ObjectInsert(_tableName, new DBreezeObject<IDbRecord>
                            {
                                Indexes = new List<DBreezeIndex>
                                        {
                                            new DBreezeIndex(1, record.Id) {PrimaryIndex = true }
                                        },
                                NewEntity = true,
                                //Changes Select-Insert pattern to Insert (speeds up insert process)
                                Entity = record //Entity itself
                            },
                                true);
                            //Using text-search engine for the free text search
                            tran.TextInsert(_textTableName, record.Id.To_8_bytes_array_BigEndian(), record.GetTextSearchKey());
                        }

                        tran.Commit();
                    }
                }

Here's the exception details:

Stacktrace:

   at DBreeze.Transactions.TransactionsCoordinator.GetTable_WRITE(String tableName, Int32 transactionThreadId)
   at DBreeze.Transactions.Transaction.GetWriteTableFromBuffer(String tableName)
   at DBreeze.Transactions.Transaction.Technical_SetTable_OverwriteIsNotAllowed(String tableName)
   at BreadPlayer.Database.KeyValueStoreDatabaseService.<>c__DisplayClass15_0.<InsertRecords>b__0()
   at System.Threading.Tasks.Task.Execute()

Please help, thanks.

Feature Request: Password Protection

Password Protection is a very necessary feature. It can be used in storing passwords, or sensitive data or anything else that requires data encryption.

netstandard version higher then netcore

version 1.86.2017.622

// Decompiled with JetBrains decompiler
// Type: DBreeze.DBreezeEngine
// Assembly: DBreeze, Version=1.86.2017.622, Culture=neutral, PublicKeyToken=174b57f1c155f3c6
// MVID: B46DCEE3-CD5C-40AE-85DF-5294C37891F4
// Assembly location: C:\Users\[]\.nuget\packages\dbreeze\1.86.0\lib\netstandard1.6\DBreeze.dll

version 1.86.2017.522

// Decompiled with JetBrains decompiler
// Type: DBreeze.DBreezeEngine
// Assembly: DBreeze, Version=1.86.2017.522, Culture=neutral, PublicKeyToken=174b57f1c155f3c6
// MVID: 84506ED6-C797-4974-98B1-EEA237624C23
// Assembly location: C:\Users\n[]\.nuget\packages\dbreeze\1.86.0\lib\netcoreapp1.0\DBreeze.dll

Kernel32.dll not found on Linux

dotnet core 2.0 with 1.88.2 on linux

Failed   NBXplorer.Tests.UnitTest1.RepositoryCanTrackAddresses
Error Message:
 DBreeze.Exceptions.DBreezeException : Getting table "@utKeysByScript" from the schema failed!
---- DBreeze.Exceptions.TableNotOperableException : DBreeze.Scheme
-------- DBreeze.Exceptions.DBreezeException : Rollback of the table "DBreeze.Scheme" failed!
------------ DBreeze.Exceptions.DBreezeException : Restore rollback file "RepositoryCanTrackAddresses/_DBreezeSchema" failed!
---------------- System.DllNotFoundException : Unable to load DLL 'kernel32.dll': The specified module or one of its dependencies could not be found.
 (Exception from HRESULT: 0x8007007E)
Stack Trace:
   at DBreeze.Transactions.TransactionsCoordinator.GetTable_WRITE(String tableName, Int32 transactionThreadId)
   at DBreeze.Transactions.Transaction.GetWriteTableFromBuffer(String tableName)
   at DBreeze.Transactions.Transaction.Insert[TKey,TValue](String tableName, TKey key, TValue value, Byte[]& refToInsertedValue, Boolean& WasUpdated, Boolean dontUpdateIfExists)
   at DBreeze.Transactions.Transaction.Insert[TKey,TValue](String tableName, TKey key, TValue value)
   at NBXplorer.Repository.InsertKeyInformation(Transaction tx, Script scriptPubKey, KeyInformation info) in /mnt/e/Sources/NBXplorer/NBXplorer/temp/NBXplorer/NBXplorer/Repository.cs:line 347
   at NBXplorer.Repository.MarkAsUsed(KeyInformation info) in /mnt/e/Sources/NBXplorer/NBXplorer/temp/NBXplorer/NBXplorer/Repository.cs:line 262
   at NBXplorer.Tests.UnitTest1.RepositoryCanTrackAddresses(RepositoryTester tester) in /mnt/e/Sources/NBXplorer/NBXplorer/temp/NBXplorer/NBXplorer.Tests/UnitTest1.cs:line 51
   at NBXplorer.Tests.UnitTest1.RepositoryCanTrackAddresses() in /mnt/e/Sources/NBXplorer/NBXplorer/temp/NBXplorer/NBXplorer.Tests/UnitTest1.cs:line 31
----- Inner Stack Trace -----
   at DBreeze.Scheme.GetTable(String userTableName)
----- Inner Stack Trace -----
   at DBreeze.LianaTrie.LTrieRootNode.RollBack()
   at DBreeze.LianaTrie.LTrie.RollBack()
----- Inner Stack Trace -----
   at DBreeze.LianaTrie.LTrieWriteCache.RollBack()
   at DBreeze.LianaTrie.LTrieRootNode.RollBack()
----- Inner Stack Trace -----
   at DBreeze.Storage.FSR.FlushFileBuffers(IntPtr hFile)
   at DBreeze.Storage.FSR.NET_Flush(FileStream mfs)
   at DBreeze.Storage.FSR.Rollback()

See discussion here
stratisproject/StratisBitcoinFullNode#3243

The error

[2019-02-07 20:44:42.6119 36] ERROR: Stratis.Bitcoin.Features.Consensus.ProvenBlockHeaders.ProvenBlockHeaderStore+<SaveAsync>d__21.MoveNext Error saving the batch DBreeze.Exceptions.TableNotOperableException: DBreeze.TranJournal ---> DBreeze.Exceptions.DBreezeException: Rollback of the table "DBreeze.TranJournal" failed! ---> DBreeze.Exceptions.DBreezeException: Restore rollback file "/Deployment/NodeData/stratis/StratisMain/provenheaders/_DBreezeTranJrnl" failed! ---> System.IO.IOException: Remote I/O error
   at System.IO.FileStream.WriteNative(ReadOnlySpan`1 source)
   at System.IO.FileStream.FlushWriteBuffer()
   at System.IO.FileStream.Seek(Int64 offset, SeekOrigin origin)
   at System.IO.FileStream.set_Position(Int64 value)
   at DBreeze.Storage.FSR.Rollback()
   --- End of inner exception stack trace ---
   at DBreeze.LianaTrie.LTrieWriteCache.RollBack()
   at DBreeze.LianaTrie.LTrieRootNode.RollBack()
   --- End of inner exception stack trace ---
   at DBreeze.LianaTrie.LTrieRootNode.RollBack()
   at DBreeze.LianaTrie.LTrie.RollBack()
   --- End of inner exception stack trace ---
   at DBreeze.Transactions.TransactionsCoordinator.Commit(Int32 transactionThreadId)
   at DBreeze.Transactions.Transaction.Commit()
   at Stratis.Bitcoin.Features.Consensus.ProvenBlockHeaders.ProvenBlockHeaderRepository.<>c__DisplayClass16_0.<PutAsync>b__0() in H:\InternalTestnet_Data\Sources\StratisBitcoinFullNode\src\Stratis.Bitcoin.Features.Consensus\ProvenBlockHeaders\ProvenBlockHeaderRepository.cs:line 160
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location where exception was thrown ---
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot)
--- End of stack trace from previous location where exception was thrown ---
   at Stratis.Bitcoin.Features.Consensus.ProvenBlockHeaders.ProvenBlockHeaderStore.SaveAsync() in H:\InternalTestnet_Data\Sources\StratisBitcoinFullNode\src\Stratis.Bitcoin.Features.Consensus\ProvenBlockHeaders\ProvenBlockHeaderStore.cs:line 276
[2019-02-07 20:44:48.3362 36] INFO: Stratis.Bitcoin.Utilities.AsyncLoop+<>c__DisplayClass16_0+<<StartAsync>b__0>d.MoveNext FlushChain stopping.
[2019-02-07 20:44:50.1569 43] INFO: Stratis.Bitcoin.FullNode.<StartPeriodicLog>b__76_0 ======Node stats====== 02/07/2019 20:44:50

Thanks.

Insert fails on Windows 10 IoT (UWP)

I'm trying to use DBreeze on a Raspberry Pi 3 running Windows 10 IoT. It works fine in the Windows version, but fails when running on the Pi. Here's the exception:

"DBreeze.Exceptions.DBreezeException: Getting table \"@utKeyValue\" from the schema failed! ---> DBreeze.Exceptions.TableNotOperableException: DBreeze.Scheme ---> DBreeze.Exceptions.DBreezeException: Rollback of the table \"DBreeze.Scheme\" failed! ---> DBreeze.Exceptions.DBreezeException: Restore rollback file \"C:\\Data\\Users\\DefaultAccount\\AppData\\Local\\Packages\\068e5313-8012-428f-883e-4e8bc4bb5c4e_mr3ez18jctte6\\LocalCache\\_DBreezeSchema\" failed! ---> System.DllNotFoundException: Unable to load DLL 'kernel32.dll': The specified module could not be found. (Exception from HRESULT: 0x8007007E)
   at DBreeze.Storage.FSR.FlushFileBuffers(IntPtr hFile)
   at DBreeze.Storage.FSR.NET_Flush(FileStream mfs)
   at DBreeze.Storage.FSR.Rollback()
   --- End of inner exception stack trace ---
   at DBreeze.LianaTrie.LTrieWriteCache.RollBack()
   at DBreeze.LianaTrie.LTrieRootNode.RollBack()
   --- End of inner exception stack trace ---
   at DBreeze.LianaTrie.LTrieRootNode.RollBack()

  at DBreeze.LianaTrie.LTrie.RollBack()
   --- End of inner exception stack trace ---
   at DBreeze.Scheme.GetTable(String userTableName)
   --- End of inner exception stack trace ---
   at DBreeze.Transactions.TransactionsCoordinator.GetTable_WRITE(String tableName, Int32 transactionThreadId)
   at DBreeze.Transactions.Transaction.GetWriteTableFromBuffer(String tableName)
   at DBreeze.Transactions.Transaction.Insert[TKey,TValue](String tableName, TKey key, TValue value, Byte[]& refToInsertedValue, Boolean& WasUpdated, Boolean dontUpdateIfExists)
   at DBreeze.Transactions.Transaction.Insert[TKey,TValue](String tableName, TKey key, TValue value)
   at App1.App.TestDatabase()
   at App1.App.OnLaunched(LaunchActivatedEventArgs e)"

hhblaze / dbreeze Goto Github PK

dbreeze's People

Contributors

Stargazers

Watchers

Forkers

dbreeze's Issues

OUTPUT

DELETE 3 -----------------------------------

Stacktrace:

Recommend Projects

Recommend Topics

Recommend Org