Support hbase 1.1 about java-bigtable-hbase HOT 22 CLOSED

Possible solutions:

1. different projects/bigtable-hbase jars per hbase version.
2. Have a class that implements the Admin core logic - BigtableAdminImpl. Use reflection magic to implement the Admin interface used at runtime (either the hbase 1.0 or hbase 1.1 Admin). The logic for invoke() would be something like: see if the requested method is implemented in BigtableAdminImpl; if so, pass along the request to the impl; else, throw UnsupportedOperationException with the name of the method.

from java-bigtable-hbase.

@AngusDavis: What are your thoughts on this issue?

from java-bigtable-hbase.

I'm thinking that since hbase is using xx.yy.zz, and keeping a given API for xx.yy, we should adopt a similar versioning scheme, as we will quickly be on several branches to keep up.

Our next version for the 1.0 line should be 1.0.00, for the 1.1 line should be 1.1.00, and for the 2.0 line, 2.0.00. Anything else will just get very confusing. Note - I hate this, but I don't see how we can deal with an open source project and not do this.

from java-bigtable-hbase.

@sduskis - Between the first and second option, I'd go with the first. That said, I might have a derivation second option that won't be as much black magic.

Question - If we split out into a second project - would you specify com.google.bigtable.hbase.Connection11 to get an HBase 1.1 connection object?

from java-bigtable-hbase.

To self reply - I was originally thinking that by not changing the Connection class name, you'd end up having some tricky things to debug when dealing with classpath inconsistencies. Then I realized that my slightly alternate proposal would rely on HBase's VersionInfo being correct as seen on the classpath and that if the classpath is fubar, it's going to be a pain to debug almost no matter what.

from java-bigtable-hbase.

I prefer option #1 as it is cleaner. Option #2 may not even work. Since
that method is the only difference, I really prefer a simpler approach that
works.
On May 13, 2015 7:09 PM, "AngusDavis" [email protected] wrote:

To self reply - I was originally thinking that by not changing the
Connection class name, you'd end up having some tricky things to debug when
dealing with classpath inconsistencies. Then I realized that my slightly
alternate proposal would rely on HBase's VersionInfo being correct as seen
on the classpath and that if the classpath is fubar, it's going to be a
pain to debug almost no matter what.

—
Reply to this email directly or view it on GitHub
#284 (comment)
.

from java-bigtable-hbase.

@kevinsi4508 I'm pretty sure that I can get option 2 to work. I'm not that sure that option 1 is cleaner from a user perspective. Being able to drop in a single jar that will work with various versions of hbase is quite a usability win. Like you're saying, however, it's harder to understand from the perspective of a developer.

@AngusDavis I'm looking forward to seeing your solution.

from java-bigtable-hbase.

I agree with you regarding user and developer's perspectives. If option #2
works, I am fine with it.

@kevinsi4508 https://github.com/kevinsi4508 I'm pretty sure that I can
get option 2 to work. I'm not that sure that option 1 is cleaner from a
user perspective. Being able to drop in a single jar that will work with
various versions of hbase is quite a usability win. Like you're saying,
however, it's harder to understand from the perspective of a developer.

@AngusDavis https://github.com/AngusDavis I'm looking forward to seeing
your solution.

—
Reply to this email directly or view it on GitHub
#284 (comment)
.

from java-bigtable-hbase.

@carterpage Suggested option 3: branch a fork per hbase release. We'd create a separate git project for shared artifacts.

from java-bigtable-hbase.

What I was thinking didn't pan out very well. I'd like to take some more time to consider this, though. Obviously I'd like minimal pain for all involved (us and our users) and so far I'm not liking the options much.

Maintaining N branches seems doable. I wouldn't want multiple github projects, though.

What would the deprecation and feature policy be? At what point would we declare a branch to be something we won't add features or fixes to? Would master track master hbase (e.g., HBase-2)?

from java-bigtable-hbase.

So the reason I liked a branch per release was so we could track changes as they merge down from 1.0 to 1.1 to 2.0.

That said, it's really nice to have one clean tag for our 0.1.6 release with everything in one place. So, proposal 4, one github project, containing multiple modules:

bigtable-core/ - generates bigtable-core-0.1.6
(the others depend upon this)

bigtable-hbase-1.0/ generates bigtable-hbase-1.0-0.1.6
bigtable-hbase-1.1/ generates bigtable-hbase-1.1-0.1.6

In general, we keep releasing versions of each minor branch as long as HBase is supporting them, or a product manage decision is made to continue supporting a particular version. There will be a rare exception where we find a particularly nasty security bug and have to release a new version of an otherwise unsupported release, but that should be an exceptional case.

I was earlier nervous about the idea of copying identical changes across all three branches, but as Solomon pointed out, since they are simply shims that will be delegating anything remotely interesting to bigtable-core, I'm less concerned.

from java-bigtable-hbase.

(option 4 is really option 1.1, since it's a very slight adjustment to solomon's original proposal)

from java-bigtable-hbase.

Your Option 4 is what I meant by Option 1.

from java-bigtable-hbase.

Sorry, I hate when people do that. I'm cool with #1.

from java-bigtable-hbase.

I'm not convinced they will always be thin shims and everything interesting will always be in bigtable-core.

If a non-backwards compatible change is made anywhere other than Table, Admin, or BufferedMutator, the changes become a lot more substantial.

from java-bigtable-hbase.

That's not to say we shouldn't do option 1, but I don't think we should under-estimate it.

from java-bigtable-hbase.

Yes. I fear Option 1 both in the short term and the long term. Especially as someone who will be maintaining the shims, the jars that go with them, the documentation around it and bdutil changes.

from java-bigtable-hbase.

Here's the black magic option for your perusal (it took about 30 minutes or so to code) - #291

from java-bigtable-hbase.

For option 3, I'd have to figure out how to release the shared artifacts like bigtable-protos and bigtable-grpc-interface only once. We wouldn't want bigtable-protos-hbase-1.0- and bigtable-protos-hbase-1.1- with the same contents. That wouldn't make sense. Therefore, I couldn't call mvn release:release for the whole project on each branch. I'd have to release bigtable-protos-hbase- and other shared artifacts in one branch (somehow...) and then release each branch's bigtable-hbase-- in the other branches.

I'm not sure how all of that would work by default. If we get it to work, it won't be a trivial release process.

from java-bigtable-hbase.

from java-bigtable-hbase.

I think that the single jar import is a separate concern. There are 2 different ways in which we currently allow a user to pull in just 1 dependency:

1. In maven, all dependencies will be included through the norman
2. In cases like bdutil, we produce a -shaded version of the jar that bundles all of the dependencies in a single jar, including protos, grpc, guava and etc.

We'll make sure both ways are available in whatever solution we decide on for this issue.

from java-bigtable-hbase.

Here's an implementation of option 1: #294

It has a bit of magic in that the hbase 1.1 project over writes a class in the bigtable-hbase 1.0 project.

from java-bigtable-hbase.