Code Monkey home page Code Monkey logo

binaryoffheaphashmap's Introduction

Binary off-heap hashmap for Java 8

The BOHMap is a hashmap implementation for Java 8 that uses off-heap memory for storing map entries (both key and value).

This allows the map to grow past the available JVM heap space, and at the same time will not impose any significant GC cost for the entries stored in the map. Essentially you can have millions of entries on a very small JVM heap.

It has no external third-party library dependencies, and is 100% Java code, i.e., no native code needed.

The B in BOHMap stands for binary; All keys and values must be in binary form. In order to make it more user friendly from a POJO standpoint, a small serialization wrapper is also made available. This wrapper, called OHMap, will benefit from the same off-heap nature of BOHMap but will allow you to use any serializable Java object as both key and value.

By default the OHMap will use standard Java serialization via ObjectOutputStream and ObjectInputStream, but this can be substituted by any serialization framework of your choice.

A set of tests are also included, but if they’ve missed any use cases and you find a bug, please let me know, thanks.

I've put together a short blog post with some performance numbers: http://blog.cfelde.com/2014/04/only-the-good-die-young-or-move-off-heap/

binaryoffheaphashmap's People

Contributors

cfelde avatar vkazanov avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

binaryoffheaphashmap's Issues

BOHMap can crash JVM

BOHMap calculates offset into the partition using the following code:

 final int hash = Math.abs(hashFunction.apply(keyData));
 final long offset = hash % partitionCount;

If hashFunction returns Integer.MIN_VALUE (i.e. -2147483648) Math.abs will return a negative number. A native offset into a buffer most of the time will crash the process, if not worse.

Check out, for example, this explanation.

This can be easily fixed by doing something like this instead:

final long offset = Math.abs(hashFunction.apply(keyData) % partitionCount);

I actually have a proper fix. Should I open a PR?

Error while iteration on map values after removal

Hi,
I experienced strange behaviour of OHMap after removal of element. Here is sample code to generate issue:

    String KEY = "KEY";
    String VALUE = "VALUE";
    OHMap<String, String> ohMap = new OHMap<>(10);
    ohMap.put(KEY, VALUE);
    ohMap.remove(KEY);
    Iterator<String> iterator = ohMap.values().iterator();
    if (iterator.hasNext()) {
        iterator.next();
    }

It generates JVM crash:

# 
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007f2a1f9c9aed, pid=25433, tid=0x00007f2a1e608700
#
# JRE version: OpenJDK Runtime Environment (8.0_242-b08) (build 1.8.0_242-8u242-b08-0ubuntu3~18.04-b08)
# Java VM: OpenJDK 64-Bit Server VM (25.242-b08 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# V  [libjvm.so+0xa81aed]

or Exception:

java.lang.NegativeArraySizeException
	at com.cfelde.bohmap.BOHMap$ValuesIterator.next(BOHMap.java:1009)
	at com.cfelde.bohmap.BOHMap$ValuesIterator.next(BOHMap.java:944)
	at com.cfelde.bohmap.OHMap$Values$1.next(OHMap.java:282)
	at com.cfelde.bohmap.OHMap$Values$1.next(OHMap.java:274)

It looks like iterator#hasNext return true, but Iterator#next causes exception/crash.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.