Code Monkey home page Code Monkey logo

Comments (6)

august0715 avatar august0715 commented on May 22, 2024

另外一个的问题是:你从Hbase获取的元数据,都是先转化为String再去与类信息匹配的。建议你将类信息直接转化为byte数组,这样子从hbase获取的元数据就可以直接匹配了,这样子省去了转化String的过程。而且,key、value转化的字节数组一般都较小,使用Bytes类里数组比较方法更好。总体性能能提升不少。也就是说,将TypeInfo类中的columnInfosMap,改成:
Map<byte[], Map<byte[], ColumnInfo>> columnInfosMap = new TreeMap<byte[], Map<byte[], ColumnInfo>>(Bytes.BYTES_COMPARATOR);

from simplehbase.

zhang-xzhi avatar zhang-xzhi commented on May 22, 2024

简单看了下ppt中举的例子,发现个问题:rowkey存储的数据,貌似在qualifier里面也存储了一份。是么?

这个只是例子这么做的,实际上要看HTable的设计,可以在rowkey和qualifier中存2份,也可以rowkey中保存了,qualifier中不存,主要是看应用场景。

from simplehbase.

zhang-xzhi avatar zhang-xzhi commented on May 22, 2024

Map> columnInfosMap = new TreeMap>(Bytes.BYTES_COMPARATOR);

这个我下来测下,看看性能影响多大,谢谢这个建议。

from simplehbase.

august0715 avatar august0715 commented on May 22, 2024

这样子做,还有两个好处:
1.能实现“原始的qualifier为byte数组”的支持。有的人就是想直接定义qualifier为byte数组,而不是字符串。
2.少去了字节数组转换为String导致的gc

from simplehbase.

zhang-xzhi avatar zhang-xzhi commented on May 22, 2024

第2个好处是肯定的。

第1个我不太确定,什么场景下需要直接定义qualifier为byte数组。

from simplehbase.

zhang-xzhi avatar zhang-xzhi commented on May 22, 2024

我测了一下,性能差别不是很大。
测试代码TestUsingMapWithKeyStringAndBytes。
loop=1000000 get from string map avgtime=2.69E-4 get from bytes map avgtime=7.7E-5

from simplehbase.

Related Issues (17)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.