Comments (3)
sduskis
commented on June 24, 2024
Sorry about this issue. We fixed this problem when another user reported it in #582. The 0.2.3 release will include this change. I'm aiming to get it out next week. For now, the 0.2.3 snapshot is your best bet. You'll have to add the following to your pom.xml to access it:
<repositories>
<repository>
<id>snapshots-repo</id>
<url>https://oss.sonatype.org/content/repositories/snapshots</url>
<releases><enabled>false</enabled></releases>
<snapshots><enabled>true</enabled></snapshots>
</repository>
</repositories>
from java-bigtable-hbase.
ankurcha
commented on June 24, 2024
Hi @sduskis,
I tried that and not that I have my grpc service working I got an (i believe) related issue with dataflow.
2016-01-15T21:01:51.852Z: Basic: S07: (8b8d1c3e7dcc314b): Executing operation ResourceRows/combine_dimensional_metrics/GroupByKey/Read+ResourceRows/combine_dimensional_metrics/Combine.GroupedValues+ResourceRows/combine_dimensional_metrics/Combine.GroupedValues/Extract+ResourceRows/map_aggregates_to_rows+PersistToBigTable/ConvertToPut+PersistToBigTable/CloudBigtableIO.CloudBigtableWriteTransform/ParDo(CloudBigtableSingleTableWrite)
2016-01-15T21:01:53.646Z: Error: (1fce635d4d1afade): java.lang.VerifyError: class com.google.protobuf.HBaseZeroCopyByteString overrides final method newInput.()Ljava/io/InputStream;
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:760)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at org.apache.hadoop.hbase.util.ByteStringer.<clinit>(ByteStringer.java:44)
at org.apache.hadoop.hbase.protobuf.ProtobufUtil.getMutationBuilderAndSetCommonFields(ProtobufUtil.java:1248)
at org.apache.hadoop.hbase.protobuf.ProtobufUtil.toMutation(ProtobufUtil.java:1170)
at org.apache.hadoop.hbase.protobuf.ProtobufUtil.toMutation(ProtobufUtil.java:1159)
at org.apache.hadoop.hbase.protobuf.ProtobufUtil.toMutation(ProtobufUtil.java:1146)
at com.google.cloud.bigtable.dataflow.HBaseMutationCoder.encode(HBaseMutationCoder.java:49)
at com.google.cloud.bigtable.dataflow.HBaseMutationCoder.encode(HBaseMutationCoder.java:40)
at com.google.cloud.dataflow.sdk.coders.StandardCoder.getEncodedElementByteSize(StandardCoder.java:168)
at com.google.cloud.dataflow.sdk.coders.StandardCoder.registerByteSizeObserver(StandardCoder.java:186)
at com.google.cloud.dataflow.sdk.util.WindowedValue$FullWindowedValueCoder.registerByteSizeObserver(WindowedValue.java:629)
at com.google.cloud.dataflow.sdk.util.WindowedValue$FullWindowedValueCoder.registerByteSizeObserver(WindowedValue.java:540)
at com.google.cloud.dataflow.sdk.runners.worker.MapTaskExecutorFactory$ElementByteSizeObservableCoder.registerByteSizeObserver(MapTaskExecutorFactory.java:508)
at com.google.cloud.dataflow.sdk.util.common.worker.OutputObjectAndByteCounter.update(OutputObjectAndByteCounter.java:126)
at com.google.cloud.dataflow.sdk.runners.worker.DataflowOutputCounter.update(DataflowOutputCounter.java:61)
at com.google.cloud.dataflow.sdk.util.common.worker.OutputReceiver.process(OutputReceiver.java:46)
at com.google.cloud.dataflow.sdk.runners.worker.ParDoFnBase$1.output(ParDoFnBase.java:174)
at com.google.cloud.dataflow.sdk.util.DoFnRunner$DoFnContext.outputWindowedValue(DoFnRunner.java:329)
at com.google.cloud.dataflow.sdk.util.DoFnRunner$DoFnProcessContext.output(DoFnRunner.java:483)
at com.brightcove.rna.transforms.ConvertBigTablePutFn.processElement(ConvertBigTablePutFn.java:32)
Any Idea what causes this? I seemed to me that there were overlapping dependencies but it doesn't particularly look right (as no other dependencies were changed).
from java-bigtable-hbase.
sduskis
commented on June 24, 2024
Oh, the joys of maven dependencies. In 0.2.1, we recommended adding HBase as a separate dependency due to a transitory dependency issue; I'm guessing you did that. HBase brings in a ton of old dependencies, including older versions of protobuf. In 0.2.2, we recommend removing hbase as a direct dependency due to all of these issues. It seems like more people are using other gRPC based systems, and at this point, the hbase dependency is more of a burden than a help.
Can you remove the hbase dependency if you have a direct dependency? Perhaps also move the bigtable dependency down to the end of the dependency list in your pom.xml file.
from java-bigtable-hbase.
Related Issues (20)
- bigtable.hbase.mirroring.TestReadVerificationSampling: testNoReadsVerificationOnGets failed HOT 1
- bigtable.hbase.mirroring.TestBufferedMutator: testBufferedMutatorPerformsMutations[mutateConcurrently: false] failed HOT 1
- bigtable.hbase.mirroring.TestReadVerificationSampling: testPartialReadsVerificationOnGets failed HOT 1
- bigtable.hbase.mirroring.TestBufferedMutator: testBufferedMutatorPrimaryErrorHandling[mutateConcurrently: true] failed HOT 1
- bigtable.hbase.mirroring.TestReadVerificationSampling: testAllReadsVerificationOnGets failed HOT 1
- bigtable.hbase.mirroring.TestBufferedMutator: testBufferedMutatorPerformsMutations[mutateConcurrently: true] failed HOT 1
- bigtable.hbase2_x.replication.HbaseToCloudBigtableBidirectionalReplicationEndpointTest: testDropsReplicatedEntry failed HOT 1
- bigtable.hbase.TestRetryBehavior: testRpcWillRetryOnAbort[multi-put] failed HOT 1
- Is Java 17 supported by this API? HOT 1
- bigtable.hbase.TestRetryBehavior: testRpcWillRetryOnAbort[single-put] failed HOT 1
- bigtable.hbase.wrappers.veneer.TestBulkMutationVeneerApi: testWhenBatcherIsClosed failed HOT 1
- CONTRIBUTING.md authentication step should refer to correct section
- finishBundle() should timeout after a long time
- Confused about release rules
- bigtable.hbase.TestColumnFamilyAdmin: testRemoveColumn failed HOT 1
- ConcurrentMirroringBUfferedMutator mutate() doesnt always rethrow exceptions
- bigtable.hbase.TestListTables: testListTableNamesWithEmptyElement failed HOT 1
- bigtable.hbase.TestListTables: testTableNames failed HOT 1
- bigtable.hbase.TestCreateTable: testTableNames failed HOT 1
- java.lang.IllegalStateException: Could not find an appropriate constructor for com.google.cloud.bigtable.hbase2_x.BigtableConnection
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from java-bigtable-hbase.