Warning: a recent release failed

Google Cloud Bigtable Client for Java

Java idiomatic client for Cloud Bigtable.

Quickstart

If you are using Maven with BOM, add this to your pom.xml file:

<dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>com.google.cloud</groupId>
      <artifactId>libraries-bom</artifactId>
      <version>26.25.0</version>
      <type>pom</type>
      <scope>import</scope>
    </dependency>
  </dependencies>
</dependencyManagement>

<dependencies>
  <dependency>
    <groupId>com.google.cloud</groupId>
    <artifactId>google-cloud-bigtable</artifactId>
  </dependency>

If you are using Maven without the BOM, add this to your dependencies:

<dependency>
  <groupId>com.google.cloud</groupId>
  <artifactId>google-cloud-bigtable</artifactId>
  <version>2.39.0</version>
</dependency>

If you are using Gradle 5.x or later, add this to your dependencies:

implementation platform('com.google.cloud:libraries-bom:26.37.0')

implementation 'com.google.cloud:google-cloud-bigtable'

If you are using Gradle without BOM, add this to your dependencies:

implementation 'com.google.cloud:google-cloud-bigtable:2.39.0'

If you are using SBT, add this to your dependencies:

libraryDependencies += "com.google.cloud" % "google-cloud-bigtable" % "2.39.0"

Authentication

See the Authentication section in the base directory's README.

Authorization

The client application making API calls must be granted authorization scopes required for the desired Cloud Bigtable APIs, and the authenticated principal must have the IAM role(s) required to access GCP resources using the Cloud Bigtable API calls.

Getting Started

Prerequisites

You will need a Google Cloud Platform Console project with the Cloud Bigtable API enabled.

Follow these instructions to get your project set up. You will also need to set up the local development environment by installing the Google Cloud Command Line Interface and running the following commands in command line: gcloud auth login and gcloud config set project [YOUR PROJECT ID].

Installation and setup

You'll need to obtain the google-cloud-bigtable library. See the Quickstart section to add google-cloud-bigtable as a dependency in your code.

About Cloud Bigtable

Cloud Bigtable

See the Cloud Bigtable client library docs to learn how to use this Cloud Bigtable Client Library.

About Cloud Bigtable

Cloud Bigtable is Google's NoSQL Big Data database service. It's the same database that powers many core Google services, including Search, Analytics, Maps, and Gmail.

Be sure to activate the Cloud Bigtable API and the Cloud Bigtable Admin API under APIs & Services in the GCP Console to use Cloud Bigtable from your project.

See the Bigtable client library documentation (Admin API and Data API) to learn how to interact with Cloud Bigtable using this Client Library.

Concepts

Cloud Bigtable is composed of instances, clusters, nodes and tables.

Instances

Instances are containers for clusters.

Clusters

Clusters represent the actual Cloud Bigtable service. Each cluster belongs to a single Cloud Bigtable instance, and an instance can have up to 4 clusters. When your application sends requests to a Cloud Bigtable instance, those requests are actually handled by one of the clusters in the instance.

Nodes

Each cluster in a production instance has 3 or more nodes, which are compute resources that Cloud Bigtable uses to manage your data.

Tables

Tables contain the actual data and are replicated across all of the clusters in an instance.

Clients

The Cloud Bigtable API consists of:

Data API

Allows callers to persist and query data in a table. It's exposed by BigtableDataClient.

Admin API

Allows callers to create and manage instances, clusters, tables, and access permissions. This API is exposed by: BigtableInstanceAdminClient for Instance and Cluster level resources.

See BigtableTableAdminClient for table management.

See BigtableDataClient for the data client.

See BigtableInstanceAdminClient for the instance admin client.

See BigtableTableAdminClient for the table admin client.

Calling Cloud Bigtable

The Cloud Bigtable API is split into 3 parts: Data API, Instance Admin API and Table Admin API.

Here is a code snippet showing simple usage of the Data API. Add the following imports at the top of your file:

import com.google.cloud.bigtable.data.v2.BigtableDataClient;
import com.google.cloud.bigtable.data.v2.models.Query;
import com.google.cloud.bigtable.data.v2.models.Row;

Then, to make a query to Bigtable, use the following code:

// Instantiates a client
String projectId = "my-project";
String instanceId = "my-instance";
String tableId = "my-table";

// Create the client.
// Please note that creating the client is a very expensive operation
// and should only be done once and shared in an application.
BigtableDataClient dataClient = BigtableDataClient.create(projectId, instanceId);

try {
  // Query a table
  Query query = Query.create(tableId)
      .range("a", "z")
      .limit(26);

  for (Row row : dataClient.readRows(query)) {
    System.out.println(row.getKey());
  }
} finally {
  dataClient.close();
}

The Admin APIs are similar. Here is a code snippet showing how to create a table. Add the following imports at the top of your file:

import static com.google.cloud.bigtable.admin.v2.models.GCRules.GCRULES;
import com.google.cloud.bigtable.admin.v2.BigtableTableAdminClient;
import com.google.cloud.bigtable.admin.v2.models.CreateTableRequest;
import com.google.cloud.bigtable.admin.v2.models.Table;

Then, to create a table, use the following code:

String projectId = "my-instance";
String instanceId = "my-database";

BigtableTableAdminClient tableAdminClient = BigtableTableAdminClient
  .create(projectId, instanceId);

try {
  tableAdminClient.createTable(
      CreateTableRequest.of("my-table")
        .addFamily("my-family")
  );
} finally {
  tableAdminClient.close();
}

TIP: If you are experiencing version conflicts with gRPC, see Version Conflicts.

Client side metrics

Cloud Bigtable client supports publishing client side metrics to Cloud Monitoring under the bigtable.googleapis.com/client namespace.

This feature is available once you upgrade to version 2.16.0 and above. Follow the guide on https://cloud.google.com/bigtable/docs/client-side-metrics-setup to enable.

Since version 2.38.0, client side metrics is enabled by default. This feature collects useful telemetry data in the client and is recommended to use in conjunction with server-side metrics to get a complete, actionable view of your Bigtable performance. There is no additional cost to publish and view client-side metrics in Cloud Monitoring.

Opt-out client side metrics

You can opt-out client side metrics with the following settings:

BigtableDataSettings settings = BigtableDataSettings.newBuilder()
        .setProjectId("my-project")
        .setInstanceId("my-instance")
        .setMetricsProvider(NoopMetricsProvider.INSTANCE)
        .build();

Use a custom OpenTelemetry instance

If your application already has OpenTelemetry integration, you can register client side metrics on your OpenTelemetry instance. You can refer to CustomOpenTelemetryMetricsProvider on how to set it up.

Client request tracing: OpenCensus Tracing

Cloud Bigtable client supports OpenCensus Tracing, which gives insight into the client internals and aids in debugging production issues. By default, the functionality is disabled. For example to enable tracing using Google Stackdriver:

If you are using Maven, add this to your pom.xml file

<dependency>
  <groupId>io.opencensus</groupId>
  <artifactId>opencensus-impl</artifactId>
  <version>0.31.1</version>
  <scope>runtime</scope>
</dependency>
<dependency>
  <groupId>io.opencensus</groupId>
  <artifactId>opencensus-exporter-trace-stackdriver</artifactId>
  <version>0.31.1</version>
  <exclusions>
    <exclusion>
      <groupId>io.grpc</groupId>
      <artifactId>*</artifactId>
    </exclusion>
    <exclusion>
      <groupId>com.google.auth</groupId>
      <artifactId>*</artifactId>
    </exclusion>
  </exclusions>
</dependency>

If you are using Gradle, add this to your dependencies

compile 'io.opencensus:opencensus-impl:0.24.0'
compile 'io.opencensus:opencensus-exporter-trace-stackdriver:0.24.0'

If you are using SBT, add this to your dependencies

libraryDependencies += "io.opencensus" % "opencensus-impl" % "0.24.0"
libraryDependencies += "io.opencensus" % "opencensus-exporter-trace-stackdriver" % "0.24.0"

At the start of your application configure the exporter:

import io.opencensus.exporter.trace.stackdriver.StackdriverTraceConfiguration;
import io.opencensus.exporter.trace.stackdriver.StackdriverTraceExporter;

StackdriverTraceExporter.createAndRegister(
  StackdriverTraceConfiguration.builder()
      .setProjectId("YOUR_PROJECT_ID")
      .build());

You can view the traces on the Google Cloud Platform Console Trace page.

By default traces are sampled at a rate of about 1/10,000. You can configure a higher rate by updating the active tracing params:

import io.opencensus.trace.Tracing;
import io.opencensus.trace.samplers.Samplers;

Tracing.getTraceConfig().updateActiveTraceParams(
    Tracing.getTraceConfig().getActiveTraceParams().toBuilder()
        .setSampler(Samplers.probabilitySampler(0.01))
        .build()
);

Disable Bigtbale traces

If your application already has OpenCensus Tracing integration and you want to disable Bigtable traces, you can do the following:

public static class MySampler extends Sampler {

    private final Sampler childSampler;

    MySampler(Sampler child) {
        this.childSampler = child;
    }

    @Override
    public boolean shouldSample(@Nullable SpanContext parentContext,
                                @Nullable Boolean hasRemoteParent,
                                TraceId traceId,
                                SpanId spanId,
                                String name,
                                List<Span> parentLinks) {
        if (name.contains("Bigtable")) {
            return false;
        }
        return childSampler.shouldSample(parentContext, hasRemoteParent, traceId, spanId, name, parentLinks);
    }

    @Override
    public String getDescription() {
        return "from my sampler";
    }
}

And use this sampler in your trace config:

Tracing.getTraceConfig().updateActiveTraceParams(
        Tracing.getTraceConfig().getActiveTraceParams().toBuilder()
                .setSampler(new MySampler(Samplers.probabilitySampler(0.1)))
                .build()
);

Version Conflicts

google-cloud-bigtable depends on gRPC directly which may conflict with the versions brought in by other libraries, for example Apache Beam. This happens because internal dependencies between gRPC libraries are pinned to an exact version of grpc-core (see here). If both google-cloud-bigtable and the other library bring in two gRPC libraries that depend on the different versions of grpc-core, then dependency resolution will fail. The easiest way to fix this is to depend on the gRPC bom, which will force all the gRPC transitive libraries to use the same version.

Add the following to your project's pom.xml.

    <dependencyManagement>
      <dependencies>
        <dependency>
          <groupId>io.grpc</groupId>
          <artifactId>grpc-bom</artifactId>
          <version>1.28.0</version>
          <type>pom</type>
          <scope>import</scope>
        </dependency>
      </dependencies>
    </dependencyManagement>

Container Deployment

While deploying this client in Google Kubernetes Engine(GKE) with CoS. Please make sure to provide CPU configuration in your deployment file. With default configuration JVM detects only 1 CPU, which affects the number of channels with the client, resulting in performance repercussion.

Also, The number of grpc-nio-worker-ELG-1-# thread is same as number of CPUs. These are managed by a single grpc-default-executor-# thread, which is shared among multiple client instances.

For example:

appVersion: v1
...
spec:
  ...
  container:
    resources:
      requests:
        cpu: "1" # Here 1 represents 100% of single node CPUs whereas other than 1 represents the number of CPU it would use from a node.

see Assign CPU Resources to Containers for more information.

Samples

Samples are in the samples/ directory.

Sample	Source Code	Try it
Native Image Bigtable Sample	source code
Authorized View Example	source code
Configure Connection Pool	source code
Filters	source code
Hello World	source code
Instance Admin Example	source code
Key Salting	source code
Quickstart	source code
Reads	source code
Table Admin Example	source code
Write Batch	source code
Write Conditionally	source code
Write Increment	source code
Write Simple	source code
Batch Delete Example	source code
Conditional Delete Example	source code
Delete Column Family Example	source code
Delete From Column Example	source code
Delete From Column Family Example	source code
Delete From Row Example	source code
Delete Table Example	source code
Drop Row Range Example	source code

Troubleshooting

To get help, follow the instructions in the shared Troubleshooting document.

Supported Java Versions

Java 8 or above is required for using this client.

Google's Java client libraries, Google Cloud Client Libraries and Google Cloud API Libraries, follow the Oracle Java SE support roadmap (see the Oracle Java SE Product Releases section).

For new development

In general, new feature development occurs with support for the lowest Java LTS version covered by Oracle's Premier Support (which typically lasts 5 years from initial General Availability). If the minimum required JVM for a given library is changed, it is accompanied by a semver major release.

Java 11 and (in September 2021) Java 17 are the best choices for new development.

Keeping production systems current

Google tests its client libraries with all current LTS versions covered by Oracle's Extended Support (which typically lasts 8 years from initial General Availability).

Legacy support

Google's client libraries support legacy versions of Java runtimes with long term stable libraries that don't receive feature updates on a best efforts basis as it may not be possible to backport all patches.

Google provides updates on a best efforts basis to apps that continue to use Java 7, though apps might need to upgrade to current versions of the library that supports their JVM.

Where to find specific information

The latest versions and the supported Java versions are identified on the individual GitHub repository github.com/GoogleAPIs/java-SERVICENAME and on google-cloud-java.

Versioning

This library follows Semantic Versioning.

Contributing

Contributions to this library are always welcome and highly encouraged.

See CONTRIBUTING for more information how to get started.

Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms. See Code of Conduct for more information.

License

Apache 2.0 - See LICENSE for more information.

CI Status

Java Version	Status
Java 8
Java 8 OSX
Java 8 Windows
Java 11

Java is a registered trademark of Oracle and/or its affiliates.

	/**
	* Wraps the protobuf {@link ReadRowsRequest}.
	*
	* <p>WARNING: Please note that the project id & instance id in the table name will be overwritten
	* by the configuration in the BigtableDataClient.
	*/
	public static Query fromProto(@Nonnull ReadRowsRequest request) {
	Preconditions.checkArgument(request != null, "ReadRowsRequest must not be null");

	Query query = new Query(NameUtil.extractTableIdFromTableName(request.getTableName()));
	query.builder = request.toBuilder();

	return query;
	}

googleapis / java-bigtable Goto Github PK

java-bigtable's Introduction

Google Cloud Bigtable Client for Java

Quickstart

Authentication

Authorization

Getting Started

Prerequisites

Installation and setup

About Cloud Bigtable

About Cloud Bigtable

Concepts

Instances

Clusters

Nodes

Tables

Clients

Data API

Admin API

Calling Cloud Bigtable

Client side metrics

Opt-out client side metrics

Use a custom OpenTelemetry instance

Client request tracing: OpenCensus Tracing

Disable Bigtbale traces

Version Conflicts

Container Deployment

Samples

Troubleshooting

Supported Java Versions

For new development

Keeping production systems current

Legacy support

Where to find specific information

Versioning

Contributing

License

CI Status

java-bigtable's People

Contributors

Stargazers

Watchers

Forkers

java-bigtable's Issues

Edited/Blocked

Open

Ignored or Blocked

Detected dependencies

Environment details

Steps to reproduce

Code example

Stack trace

Any additional information below

What the problem is.

What you want to happen.

Environment details

Issue

Steps to reproduce

Code example

External references such as API reference guides

Any additional information below

Environment details

Issue

Steps to reproduce

Code example

External references such as API reference guides

Any additional information below

Environment details

Steps to reproduce

Code example

Stack trace

Any additional information below

Recommend Projects

Recommend Topics

Recommend Org