ActualSize much less than real file size. about mmkv HOT 26 CLOSED

It's possible. It's not about how many items you store, but how big your items are.

from mmkv.

Should actualSize equals file size?

from mmkv.

No.

from mmkv.

And I don't recommend storing big data inside MMKV, at lease not in current release.
Checkout the FAQ for more information.

from mmkv.

Thank you. We didn't stored big date.
I check the big mmkv file with hex friend, only first 4M is valid data, remaining 130M is "0".

from mmkv.

That's weird. Can you send a reproducible demo project to me? [email protected]

from mmkv.

I don't know the reproduce step, I got the file from online gray version.

The first line is like this, the actual size is same with valid data size.
69B93F00 A6DE021C 31303634 32373733 37396C61 73745F75

The last line is like this, which means “ "mRealTime":false}] ”, seems like the end of a string set.
226D5265 616C5469 6D65223A 66616C73 657D5D00 0000

I am checking the value size key by key.

from mmkv.

By the way, why are you using MULTI_PROCESS_MODE in a single process App?

from mmkv.

It's a multi process App, I mean maybe we used SINGLE_PROCESS_MODE in some MULTI_PROCESS_MODE cases,

By the way, the last value mentioned before is actually a Json String of List, is there any possible the calculate of Json size occurred error?

from mmkv.

Json is just a String, nothing special. Is there any chance that there're some big items stored in MMKV and deleted afterward?

from mmkv.

I am not very sure of that, need further check, what's the exact definition of "big"?

from mmkv.

Items with size of 10K+ is consider big.

from mmkv.

Do you mean remove big items would not trigger file resize?
I add some log in MiniPBCoder::decodeOneMap:
{
int i = 0;
int j = 0;
while (!m_inputData->isAtEnd()) {
const auto &key = m_inputData->readString();
if (key.length() > 0) {
i++;
auto value = m_inputData->readData();
if (value.length() > 0) {
j += value.length();
dic[key] = move(value);
} else {
dic.erase(key);
}
}
}
MMKVInfo("total %d keys, value size %d", i, j);
}
loading [] with 4176233 size in total, file size is 134217728
loading [] with crc 4170586551 sequence 4
total 3412 keys, value size 4081561
loaded [***] with 1111 values

from mmkv.

I think I have found the reason, the futureUsage is over estimated:
size_t futureUsage = newSize * std::max<size_t>(8, (m_dic.size() + 1) / 2);

My last value size is 130K, m_dic.size() is 1000+, then the file will resize to 130M, but most of it will never been used.

This means, if I store thousands of keys in a file, it will very easy to cause OOM.

from mmkv.

Oh I see...Let me think about that.

from mmkv.

Fixed with this commit.

from mmkv.

I think it's good enough for me, thank you. How far will this release?

By the way, I found there is no reduce file size operation except "clearAll", if I store a lot of key-values in one file and then delete most of them in some time, it seems will waste some disk space.

Should I avoid this kind of usage as far as possible?

from mmkv.

That's a good suggestion. A trim operation should be supported.

And by the way, the next minor release is not on schedule yet. I'll let you know by commenting on this issue when that happens.

from mmkv.

Waiting for good news, thank you very much!

from mmkv.

Release with v1.0.13, check it out.

from mmkv.

Wonderful!

from mmkv.

It seems use average item size to calc future usage does not merge into android code.

from mmkv.

It seems use average item size to calc future usage does not merge into android code.

I can't believe these, looks like there's a mistake on my local git repo.
Will fix soon.

from mmkv.

Fixed with this commit. A new version will likely come out next week.

from mmkv.

Look forward to it, thank you.

from mmkv.

Released with v1.0.14.

from mmkv.