Comments (12)
from lmdbjava.
Just confirmed that dropping commit 3524995 brings back previous behaviour and my test pass.
In case it's relevant when I'm opening database I do
Dbi<ByteBuffer> db = _env.openDbi(hexId, MDB_CREATE);
from lmdbjava.
Further tests show that if I open db like this
Comparator<ByteBuffer> cmp = new MyComparatorCopiedFromByteBufferProxy();
Dbi<ByteBuffer> db = _env.openDbi(hexId, cmp, MDB_CREATE);
where MyComparatorCopiedFromByteBufferProxy
has implementation copied from org.lmdbjava.ByteBufferProxy$AbstractByteBufferProxy
then my test are passing as well on 0.9.0
private static class MyComparatorCopiedFromByteBufferProxy implements Comparator<ByteBuffer> {
@Override
public int compare(ByteBuffer o1, ByteBuffer o2) {
return compareBuff(o1, o2);
}
/**
* Lexicographically compare two buffers.
*
* @param o1 left operand (required)
* @param o2 right operand (required)
* @return as specified by {@link Comparable} interface
*/
public static int compareBuff(final ByteBuffer o1, final ByteBuffer o2) {
requireNonNull(o1);
requireNonNull(o2);
if (o1.equals(o2)) {
return 0;
}
final int minLength = Math.min(o1.limit(), o2.limit());
final int minWords = minLength / Long.BYTES;
final boolean reverse1 = o1.order() == LITTLE_ENDIAN;
final boolean reverse2 = o2.order() == LITTLE_ENDIAN;
for (int i = 0; i < minWords * Long.BYTES; i += Long.BYTES) {
final long lw = reverse1 ? reverseBytes(o1.getLong(i)) : o1.getLong(i);
final long rw = reverse2 ? reverseBytes(o2.getLong(i)) : o2.getLong(i);
final int diff = Long.compareUnsigned(lw, rw);
if (diff != 0) {
return diff;
}
}
for (int i = minWords * Long.BYTES; i < minLength; i++) {
final int lw = Byte.toUnsignedInt(o1.get(i));
final int rw = Byte.toUnsignedInt(o2.get(i));
final int result = Integer.compareUnsigned(lw, rw);
if (result != 0) {
return result;
}
}
return o1.remaining() - o2.remaining();
}
}
from lmdbjava.
It looks like the problem is related to changing what is a default comparator for ByteBuffer.
In pre 0.9.0 it was a custom lmdbjava implementation which was doing unsigned comparison.
In 0.9.0 comparator is selected based on the flag MDB_INTEGERKEY
See method in ByteBufferProxy
@Override
protected Comparator<ByteBuffer> getComparator(final DbiFlags... flags) {
final int flagInt = mask(flags);
if (isSet(flagInt, MDB_INTEGERKEY)) {
return this::compareCustom;
}
return this::compareDefault;
}
In my case, since I'm not passing MDB_INTEGERKEY new comparator is used based on ByteBuffer compareTo (signed bytes)
protected final int compareDefault(final ByteBuffer o1,
final ByteBuffer o2) {
return o1.compareTo(o2);
}
This in itself is a behaviour change which breaks lmdbjava for me.
Supplying MDB_INTEGERKEY also does not work, not clear to me why
But I was under the impression MDB_INTEGERKEY should be used where keys are actually integers, not byte[]
From DbiFlags
/**
* Numeric keys in native byte order: either unsigned int or size_t. The keys
* must all be of the same size.
*/
MDB_INTEGERKEY(0x08),
Not sure if supplying that flag has some side effects in native lmdb library.
As mentioned before a workaround for me is to use a comparator which is a copy of the code from AbstractByteBufferProxy
from lmdbjava.
Related Issues (20)
- Compatibility with Java 19 - Modularize? Minimizing use of unsupported JDK parts? HOT 3
- Is it possible to make Dbi constructor accessible from outside? HOT 1
- Version 0.8.3 broken...? HOT 20
- ByteBuf.nioBuffer() returns a buffer with all zeros HOT 2
- Is lmdb has some problems which exists in go version? HOT 5
- mdb_page_flush crash HOT 2
- Having issues allocating a reasonable map size on Windows HOT 1
- There was an error in the forked process HOT 2
- Why does LMDB need to know how large our DB might be? HOT 3
- Entries not having expected values HOT 6
- New release date? HOT 5
- lmdb-java throws UnsatisfiedLinkError on M2 macOS HOT 4
- CursorIterable/KeyRange broken in v0.9.0 HOT 2
- Crash with 0.7.0 when maxNumReaders exceeded
- Page Full Exception on Apple Silicon
- EINVAL 22 while deleting entries with a cursor HOT 1
- EIO Input/Output error (5) HOT 2
- `could not load FFI provider` (Centos 7 and lmdbjava 0.8.3)
- Env$MapFullException while deleting data from cursor HOT 9
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from lmdbjava.