lysevi / nkvdb Goto Github PK

I just modified the read_write_example to insert the time values in descending order.
And also printed what's been read, to my surprise, there are time values I didn't ask for in the output and, on the other hand, time values I did ask for are missing.

diff --git a/examples/read_write_example.cpp b/examples/read_write_example.cpp
index 4c6a084..d1de1fe 100644
--- a/examples/read_write_example.cpp
+++ b/examples/read_write_example.cpp
@@ -30,7 +30,7 @@ int main(int argc, char *argv[]) {
   clock_t write_t0 = clock();
   mdb::Meas meas = mdb::Meas::empty();

-  for (int i = 0; i < 3000000; ++i) {
+  for (int i = 3000000; i-- > 0;) {
     meas.value = i;
     meas.id = i % 10;
     meas.source = meas.flag = 0;
@@ -47,11 +47,14 @@ int main(int argc, char *argv[]) {
   clock_t read_t0 = clock();

   mdb::Meas::MeasList output;
-  auto reader = ds->readInterval(0, writes_count);
+  auto reader = ds->readInterval(3, 30);

   // or meases->readAll(&output);
   while (!reader->isEnd()) {
          reader->readNext(&output);
+         for (auto i = output.cbegin(); i != output.cend(); ++i) {
+                 printf("got time %lu\n", i->time);
+         }
   }

So I asked for times between 3 and 30 and got:

write 3000000 values...
write time: 2.84058
read all values...
got time 0
got time 1
got time 2
got time 24
got time 3
read time :0.192736

format and use case.

steps:

• byChange - by defaul
• byTime - with step value in seconds

map/reduce api

meta test

file (json?) placed in storage dir.

• page size
• dynamic cache
• cache size
• cache pool size
• step between index writes (It affects the number of measurements in one millisecond)

mdb looks like something I need but I'm skeptical of using it without a published storage format.

mode when storage remove old pages

data compression

more correct

• btree index
• all reads must use with index.

in index and page header.

I modified the read_write_example.cpp in the following way:

#include <ctime>
#include <iostream>
#include <cstdlib>
#include <nkvdb.h>

const std::string storage_path = "exampleStorage";

bool enable_dyn_cache = false;
size_t cache_size =   nkvdb::defaultcacheSize;
size_t cache_pool_size=  nkvdb::defaultcachePoolSize;

const unsigned writes_count = 2000000;
const size_t arr_size = 17;

int main(int argc, char *argv[]) {

    nkvdb::Storage::Storage_ptr ds =  nkvdb::Storage::Create(storage_path);
    ds->enableCacheDynamicSize(enable_dyn_cache);
    ds->setPoolSize(cache_pool_size);
    ds->setCacheSize(cache_size);
    std::cout << "write " << writes_count << " values..." << std::endl;

    nkvdb::Meas::PMeas array = new nkvdb::Meas[arr_size];

    for (unsigned s = 0; s < (unsigned)( writes_count/arr_size); ++s)
    {
        auto now = time(0);
        for (size_t i = 0; i < arr_size; ++i)
        {
            array[i].id = s%5;
            array[i].time = now;
            array[i].setValue(s);
        }
        ds->append(array, arr_size);
    }
    delete[] array;
    return 0;
}

According to the number used for arr_size, the program will crash or not.
I also tried to make cache_size and cache_pool_size multiples of arr_size, but it doesn't help.
It must be a problem of buffer overflow due to a division between integers, I guess.

The include file, storage.h is too generic. Why not mdb.h?

If I write a value of "x" at time "a," then I write the same value at time "b" sequentially afterwards, will this require more storage in mdb?

Having a public interface that allows you to "close" Storage complicates reasoning about the state of an instantiated Storage object. The destructor already calls Close(), so that half of making Storage an RAII class is already done. Simply removing the Close() method from the public interface achieves a greater level of class invariance.

If you'd like some help making this change, let me know and I'll throw up a pull request.