Code Monkey home page Code Monkey logo

hazelcast-jdbc's Introduction

Hazelcast JDBC Driver

Hazelcast JDBC Driver allows Java applications to connect to Hazelcast using the standard JDBC API.

Supported Hazelcast version

Hazelcast 5

The JDBC driver version 5.x is compatible with Hazelcast version 5.x. For Hazelcast 4.2, use the JDBC Driver 4.2.

Download the Driver

Binaries

The driver comes in two packages:

  1. hazelcast-jdbc-{version}.jar (open source)
  2. hazelcast-jdbc-enterprise-{version}.jar (enterprise)

The open source one isn't able to connect to Hazelcast EE clusters, but has an open source licence. The latter one uses proprietary licence, and can be used for connecting to both OS and EE clusters.

Download directly the JAR file from Releases page (open source) or Hazelcast repository (enterprise).

Maven Central

Stable versions

Open source:

<dependency>
    <groupId>com.hazelcast</groupId>
    <artifactId>hazelcast-jdbc</artifactId>
    <version>5.4.0</version>
</dependency>

Enterprise (located in Hazelcast Maven repository):

<dependencies>
    <dependency>
        <groupId>com.hazelcast</groupId>
        <artifactId>hazelcast-jdbc-enterprise</artifactId>
        <version>5.4.0</version>
    </dependency>
</dependencies>

<repositories>
    <repository>
        <id>hazelcast-release</id>
        <name>Hazelcast Repository</name>
        <url>https://repository.hazelcast.com/release/</url>
        <releases>
            <enabled>true</enabled>
        </releases>
        <snapshots>
            <enabled>false</enabled>
        </snapshots>
    </repository>
</repositories>

Release 4.2 (for Hazelcast 4.2, SQL in Beta)

<dependency>
    <groupId>com.hazelcast</groupId>
    <artifactId>hazelcast-jdbc</artifactId>
    <version>4.2</version>
</dependency>

Snapshot versions

To download the latest snapshot build you need to add the dependency and also the snapshot repository.

Open source:

<dependencies>
    <dependency>
        <groupId>com.hazelcast</groupId>
        <artifactId>hazelcast-jdbc</artifactId>
        <version>5.5.0-SNAPSHOT</version>
    </dependency>
</dependencies>

<repositories>
    <repository>
        <id>snapshot-repository</id>
        <name>Maven Snapshot Repository</name>
        <url>https://oss.sonatype.org/content/repositories/snapshots</url>
        <releases>
            <enabled>false</enabled>
        </releases>
        <snapshots>
            <enabled>true</enabled>
        </snapshots>
    </repository>
</repositories>

Enterprise:

<dependencies>
    <dependency>
        <groupId>com.hazelcast</groupId>
        <artifactId>hazelcast-jdbc-enterprise</artifactId>
        <version>5.5.0-SNAPSHOT</version>
    </dependency>
</dependencies>

<repositories>
    <repository>
        <id>hazelcast-snapshots</id>
        <name>Hazelcast Snapshot Repository</name>
        <url>https://repository.hazelcast.com/snapshot/</url>
        <releases>
            <enabled>false</enabled>
        </releases>
        <snapshots>
            <enabled>true</enabled>
        </snapshots>
    </repository>
</repositories>

Connection URL

The URL must have the following structure, parts in [] are optional:

jdbc:hazelcast://host[:port][,host[:port]...]/[?property1=value1[&property2=value2]...]

To connect to a Viridian Serverless cluster, use:

jdbc:hazelcast://<cluster-id>/?discoveryToken=<yourDiscoveryToken>&cloudUrl=https://api.viridian.hazelcast.com&sslEnabled=true

Note: Viridian Serverless clusters require additional client configuration for TLS.

To connect to the legacy Cloud service, use:

jdbc:hazelcast://cluster-id/discoveryToken=value[&property1=value1[&property2=value2]...]

where:

  • jdbc:hazelcast:: (Required) is a sub-protocol and is a constant.
  • host: (Required) server address (or addresses separated with comma) to connect to, or the cluster id if the cluster is in Hazelcast Viridian/Cloud (in which case you can find it in the Connect Client - Advanced Setup dialog).
  • port: (Optional) Server port. Defaults to 5701.
  • propertyN: (Optional) List of connection properties in the key-value form.

Connection properties

The following list contains the properties supported by the Hazelcast JDBC Driver.

Common properties

Property Type Description
user String Hazelcast cluster username
password String Hazelcast cluster password
clusterName String Hazelcast cluster name
discoveryToken String Hazelcast Cloud discovery token
cloudUrl String Hazelcast Cloud URL

SSL properties

Property Type Description
sslEnabled Boolean Enable SSL for client connection
trustStore String Path to truststore file. Alias for trustCertCollectionFile
trustCertCollectionFile String Path to truststore file. Alias for trustStore
trustStorePassword String Password to unlock the truststore file
keyStore String Path to your keystore file
keyFile String Path to your keystore file
keyStorePassword String Password to access the key from your keystore file
protocol String Name of the algorithm which is used in your TLS/SSL; default to TLS
keyCertChainFile String Path to an X.509 certificate chain file in PEM format
factoryClassName String Fully qualified class name for the implementation of SSLContextFactory
javax.net.ssl.* String Properties starting with this prefix are passed to SSLConfig directly.

GCP properties

Property Type Description
gcpPrivateKeyPath String A filesystem path to the private key for GCP service account in the JSON format; if not set, the access token is fetched from the GCP VM instance
gcpHzPort String A range of ports where the plugin looks for Hazelcast members; if not set, the default value 5701-5708 is used
gcpProjects String A list of projects where the plugin looks for instances; if not set, the current project is used
gcpRegion String A region where the plugin looks for instances; if not set, the zones property is used; if it and zones property not set, all zones of the current region are used
gcpLabel String A filter to look only for instances labeled as specified; property format: key=value
gcpUsePublicIp Boolean Use public IP Address

AWS properties

Property Type Description
awsAccessKey String Access key of your AWS account; if not set, iam-role is used
awsSecretKey String Secret key of your AWS account; if not set, iam-role is used
awsIamRole String IAM Role attached to EC2 instance used to fetch credentials (if awsAccessKey/awsSecretKey not specified); if not set, default IAM Role attached to EC2 instance is used
awsTagKey/awsTagValue String Filter to look only for EC2 Instances with the given key/value; multi values supported if comma-separated (e.g. KeyA,KeyB); comma-separated values behaves as AND conditions
awsRegion String Region where Hazelcast members are running; default is the current region
awsHostHeader String ec2, ecs, or the URL of a EC2/ECS API endpoint; automatically detected by default
awsSecurityGroupName String Filter to look only for EC2 instances with the given security group
awsConnectionTimeoutSeconds Integer Connection timeout when making a call to AWS API; default to 10
awsReadTimeoutSeconds Integer Read timeout when making a call to AWS API; default to 10
awsConnectionRetries Integer Number of retries while connecting to AWS API; default to 3
awsHzPort String A range of ports where the plugin looks for Hazelcast members; default is 5701-5708
awsUsePublicIp Boolean Use public IP Address

Azure properties

Property Type Description
azureInstanceMetadataAvailable Boolean This property must be configured as false in order to be able to use the Azure properties. It is true by default.
azureClientId String Azure Active Directory Service Principal client ID
azureClientSecret String Azure Active Directory Service Principal client secret
azureTenantId String Azure Active Directory tenant ID
azureSubscriptionId String Azure subscription ID
azureResourceGroup String Name of Azure resource group which the Hazelcast instance is running in
azureScaleSet String name of Azure VM scale set. If this setting is configured, the plugin will search for instances over the resources only within this scale set
azureUsePublicIp Boolean Use public IP Address

Kubernetes properties

Property Type Description
k8sServiceDns String Service DNS, usually in the form of SERVICE-NAME.NAMESPACE.svc.cluster.local
k8sServiceDnsTimeout Integer Custom time for how long the DNS lookup is valid
k8sNamespace String Kubernetes namespace where Hazelcast is running
k8sServiceName String Service name used to scan only PODs connected to the given service; if not specified, then all PODs in the namespace are checked
k8sServicePort Integer Endpoint port of the service; if specified with a value greater than 0, it overrides the default; 0 by default

Miscellaneous properties

Property Type Description
smartRouting Boolean If false, the client will have only one connection to the cluster. Useful, if there are many clients, and we want to avoid each of them connecting to each member. true by default.
resubmissionMode String Strategy to retry failed queries. Valid values are: NEVER (the default), RETRY_SELECTS, RETRY_SELECTS_ALLOW_DUPLICATES and RETRY_ALL.

Hazelcast Cloud Configuration

For connecting to the Hazelcast cloud you only need to specify discoveryToken property and use the cluster-id as a host in the URL: jdbc:hazelcast://<cluster-id>/?discoveryToken=<yourDiscoveryToken>.

If you're connecting to a Viridian Serverless cluster, also specify cloudUrl=https://api.viridian.hazelcast.com and sslEnabled=true: jdbc:hazelcast://<cluster-id>/?discoveryToken=<yourDiscoveryToken>&cloudUrl=https://api.viridian.hazelcast.com&sslEnabled=true.

TLS is mandatory for Viridian Serverless cluster connections so pass TLS properties such as truststore, keystore etc along to DriverManager.getConnection().

Additional Configuration

Besides URL, it is possible to use Configuration Files and Overriding Configuration to configure the Hazelcast Java Client

SQL

SQL support is in active development. Have a look at SQL docs.

hazelcast-jdbc's People

Contributors

avtarraikmo avatar dependabot[bot] avatar eugeneabramchuk avatar fly-style avatar ivanthescientist avatar jackpgreen avatar krzysztofslusarski avatar kwart avatar neilstevenson avatar nishaatr avatar orcunc avatar pavlobida avatar seriybg avatar sumnerib avatar viliam-durina avatar

Stargazers

avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Forkers

viliam-durina fly-style holmistr tomaszgaweda seriybg ivanthescientist isabella232 krzysztofslusarski frant-hartm sumnerib kwart eugeneabramchuk winjaychan orcunc ldziedziul jackpgreen nishaatr

hazelcast-jdbc's Issues

PreparedStatement.setNull fails, missing tests for all types

The following test:

    @Test
    void testNull() throws SQLException {
        Connection connection = DriverManager.getConnection(JDBC_HAZELCAST_LOCALHOST);
        PreparedStatement statement = connection.prepareStatement("SELECT * FROM person WHERE name=?");
        statement.setNull(1, Types.VARCHAR);
        ResultSet rs = statement.executeQuery();
        assertFalse(rs.next());
    }

fails with:

java.sql.SQLException
	at com.hazelcast.jdbc.JdbcStatement.doExecute(JdbcStatement.java:409)
	at com.hazelcast.jdbc.JdbcPreparedStatement.executeQuery(JdbcPreparedStatement.java:56)
	at com.hazelcast.jdbc.JdbcPreparedStatementTest.testNull(JdbcPreparedStatementTest.java:90)
        ...
Caused by: com.hazelcast.sql.HazelcastSqlException
	at com.hazelcast.sql.impl.QueryUtils.toPublicException(QueryUtils.java:78)
	at com.hazelcast.sql.impl.client.SqlClientService.rethrow(SqlClientService.java:287)
	at com.hazelcast.sql.impl.client.SqlClientService.rethrow(SqlClientService.java:278)
	at com.hazelcast.sql.impl.client.SqlClientService.execute(SqlClientService.java:115)
	at com.hazelcast.jdbc.HazelcastSqlClient.execute(HazelcastSqlClient.java:37)
	at com.hazelcast.jdbc.JdbcStatement.doExecute(JdbcStatement.java:400)
	... 67 more
Caused by: java.lang.NullPointerException
	at com.hazelcast.client.impl.protocol.codec.builtin.DataCodec.encode(DataCodec.java:32)
	at com.hazelcast.client.impl.protocol.codec.builtin.ListMultiFrameCodec.encode(ListMultiFrameCodec.java:42)
	at com.hazelcast.client.impl.protocol.codec.SqlExecuteCodec.encodeRequest(SqlExecuteCodec.java:109)
	at com.hazelcast.sql.impl.client.SqlClientService.execute(SqlClientService.java:91)
	... 69 more```

We also miss a test for all supported types in JdbcPreparedStatement (related to #8)

Implement schema-related methods for DatabaseMetadata

Implement the following methods:

getCatalogs
getSchemas (2 variations)
getTables
getTableTypes
getColumns
getPrimaryKeys
getBestRowIdentifier
getIndexInfo

The implementation is dependant on INFORMATION_SCHEMA in the SQL-module on the server-side

Remove schema from the URL

The default schema is part of our URL. However, the schema isn't a connection property, let alone a mandatory one, it can be changed after connecting by using Connection.setSchema(). Other vendors use database name in the URL, but we don't have that concept.

Reconsider the URL format

The current URL format for non-cloud clusters is:

jdbc:hazelcast://host:port,host2:port2/public?clusterName=fooCluster

and for cloud clusters:

jdbc:hazelcast://fooCluster/public?discoveryToken=fooToken

That is, the part that resembles the host name in a typical http URL is either multiple hosts, or the cloud cluster name, depending on the presence of the discoveryToken. I find this perplexing.

The JDBC standard doesn't prescribe the URL format. My proposal is this:
Non-cloud:

jdbc:hazelcast:username/password@host:port,host2:port2?clusterName=fooCluster&optionN=valueN&...

Cloud:

jdbc:hazelcast:cloud:clusterName/discoveryToken?optionN=valueN&...

The aim is to break away from the look&feel of web URLs to avoid confusion, because the host is never just a host.

In the non-cloud URL the username/password is optional:

jdbc:hazelcast:@host:port,host2:port2?clusterName=fooCluster&optionN=valueN&...

In fact, this proposal already addresses #60 because it removes the word public from the URL.

JDBC connection and a Hazelcast client interoperability

The JdbcConnection encapsulates a HZ client instance. If the user wants to use a HZ feature not available through JDBC, he has to open a new HazelcastInstance. There should be a way to do that with a single HazelcastInstance.

We should consider adding one or both of these:

  • public API to get HazelcastInstance from a JdbcConnection
  • add ability to create a JDBC Connection by providing a HazelcastInstance.

The latter is even preferable to handle special cases when we don't expose some setting, so that the user can create a HazelcastInstance himself and then create a JDBC connection by using it.

Minor: unnecessary synchronization in Driver

I think the synchronization isn't necessary, it's called only from the static initializer and the JVM guarantees that it's not concurrent and that's its called only once, so we also don't need the registered field.

hazelcast-jdbc/src/main/java/com/hazelcast/jdbc/Driver.java

Lines 86 to 95 in 0e5037d

private static synchronized void load() {
try {
if (!registered) {
DriverManager.registerDriver(INSTANCE);
registered = true;
}
} catch (SQLException exception) {
throw new ExceptionInInitializerError(exception);
}
}

See mysql driver: https://github.com/mysql/mysql-connector-j/blob/18bbd5e68195d0da083cbd5bd0d05d76320df7cd/src/main/user-impl/java/com/mysql/cj/jdbc/Driver.java#L53-L59

Empty login AND password combination is not treated as no-authentication mode

Some DB tools always send empty string in place of Username and Password (instead of nulls). If the cluster operates in no authentication mode, it will throw an exception upon receiving non-null Login/Password, therefore making it impossible to connect to the cluster from a tool that always sends non-null Login/Password.

A workaround for this is to treat empty string in Login AND Password as a no-auth mode and just send nulls instead.

Allow multiple addresses in the URL

We allow at most one member address to be supplied. In case of a cluster it's the address of one member. However, if that very member isn't available, the driver can't connect. We should be able to supply multiple addresses.

Add tests for all type conversions in ResultSet

Each type in com.hazelcast.sql.SqlColumnType should be converted to each supported type in ResultSet and asserted whether a proper exception is thrown or the value is correctly converted.

Parameter parsing for PreparedStatement

For now any ? is considered as a parameter that is not a correct behavior as ? might be present in the string literal (SELECT * FROM foo WHERE bar='?' AND baz= ?).
Also, we need to validate the correctness of the parameter location, e.g. SELECT * FROM ? WHERE foo='bar' is not valid.

Incorrect return type for ResultSet.getObject()

According to ResultSet.getObject() javadoc:

The type of the Java object will be the default Java object type corresponding to the column's SQL type, following the mapping for built-in types specified in the JDBC specification.

We currently return our default type, e.g. in case of DATE we return LocalDate, not java.sql.Date. We must return the correct type.

Add tests for streaming queries with Jet

We need to test:

  • cancellation of a streaming query
  • results delivered with low latency to the client (e.g. a stream of one item per second should not wait until 4096 items are generated before returning anything to the client)
  • multiple streaming queries over one connection

I assume the above will just work, but we need a test for that.

Spring @Bean is not used if present

If a config file is missing, the cluster dev is assumed

If a Spring @bean ClientConfig or HazelcastInstance of the correct type exists, this should be used instead

JDBC configuration for cloud failure

Cloud configuration (https://github.com/hazelcast/hazelcast-jdbc#hazelcast-cloud-configuration)
says the URL can look like:

jdbc:hazelcast://<cluster-name>/?discoveryToken=<yourDiscoveryToken>

If I set this,

	static {
		System.setProperty("spring.datasource.url", "jdbc:hazelcast://xxx/?discoveryToken=yyy");
	}

I get

16:46:24.073 [main] WARN  com.hazelcast.client.impl.connection.ClientConnectionManager - hz.client_1 [xxx] [5.0] Exception from AddressProvider: com.hazelcast.client.impl.spi.impl.discovery.RemoteAddressProvider@6535117e
 com.hazelcast.core.HazelcastException: java.io.IOException: {"application":"orchestrator-service","message":"Cluster token not found for token.","status":404,"error":"Not Found","errorCode":"ClusterTokenNotFound","timestamp":1634312784009,"path":"http://coordinator.hazelcast.cloud/cluster/discovery","method":"GET"}
	at com.hazelcast.internal.util.ExceptionUtil.lambda$static$0(ExceptionUtil.java:56) ~[hazelcast-jdbc-5.0.jar!/:?]
	at com.hazelcast.internal.util.ExceptionUtil.peel(ExceptionUtil.java:134) ~[hazelcast-jdbc-5.0.jar!/:?]
	at com.hazelcast.internal.util.ExceptionUtil.peel(ExceptionUtil.java:77) ~[hazelcast-jdbc-5.0.jar!/:?]
	at com.hazelcast.internal.util.ExceptionUtil.rethrow(ExceptionUtil.java:139) ~[hazelcast-jdbc-5.0.jar!/:?]
	at com.hazelcast.client.impl.spi.impl.discovery.HazelcastCloudDiscovery.discoverNodes(HazelcastCloudDiscovery.java:72) ~[hazelcast-jdbc-5.0.jar!/:?]
	at com.hazelcast.client.impl.spi.impl.discovery.RemoteAddressProvider.loadAddresses(RemoteAddressProvider.java:40) ~[hazelcast-jdbc-5.0.jar!/:?]
	at com.hazelcast.client.impl.connection.tcp.TcpClientConnectionManager.getPossibleMemberAddresses(TcpClientConnectionManager.java:558) ~[hazelcast-jdbc-5.0.jar!/:?]
	at com.hazelcast.client.impl.connection.tcp.TcpClientConnectionManager.doConnectToCandidateCluster(TcpClientConnectionManager.java:501) ~[hazelcast-jdbc-5.0.jar!/:?]
	at com.hazelcast.client.impl.connection.tcp.TcpClientConnectionManager.doConnectToCluster(TcpClientConnectionManager.java:414) ~[hazelcast-jdbc-5.0.jar!/:?]
	at com.hazelcast.client.impl.connection.tcp.TcpClientConnectionManager.connectToCluster(TcpClientConnectionManager.java:375) ~[hazelcast-jdbc-5.0.jar!/:?]
	at com.hazelcast.client.impl.connection.tcp.TcpClientConnectionManager.start(TcpClientConnectionManager.java:324) ~[hazelcast-jdbc-5.0.jar!/:?]
	at com.hazelcast.client.impl.clientside.HazelcastClientInstanceImpl.start(HazelcastClientInstanceImpl.java:373) ~[hazelcast-jdbc-5.0.jar!/:?]
	at com.hazelcast.client.HazelcastClient.constructHazelcastClient(HazelcastClient.java:460) ~[hazelcast-jdbc-5.0.jar!/:?]
	at com.hazelcast.client.HazelcastClient.newHazelcastClientInternal(HazelcastClient.java:416) ~[hazelcast-jdbc-5.0.jar!/:?]
	at com.hazelcast.client.HazelcastClient.newHazelcastClient(HazelcastClient.java:136) ~[hazelcast-jdbc-5.0.jar!/:?]
	at com.hazelcast.jdbc.HazelcastSqlClient.<init>(HazelcastSqlClient.java:33) ~[hazelcast-jdbc-5.0.jar!/:?]
	at com.hazelcast.jdbc.Driver.connect(Driver.java:53) ~[hazelcast-jdbc-5.0.jar!/:?]
	at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:121) ~[HikariCP-4.0.3.jar!/:?]
	at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:364) ~[HikariCP-4.0.3.jar!/:?]
	at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:206) ~[HikariCP-4.0.3.jar!/:?]
	at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:476) ~[HikariCP-4.0.3.jar!/:?]
	at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:561) ~[HikariCP-4.0.3.jar!/:?]
	at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:115) ~[HikariCP-4.0.3.jar!/:?]
	at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:112) ~[HikariCP-4.0.3.jar!/:?]
	at org.springframework.jdbc.datasource.DataSourceUtils.fetchConnection(DataSourceUtils.java:158) ~[spring-jdbc-5.3.10.jar!/:5.3.10]
	at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:116) ~[spring-jdbc-5.3.10.jar!/:5.3.10]
	at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:79) ~[spring-jdbc-5.3.10.jar!/:5.3.10]
	at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:330) ~[spring-jdbc-5.3.10.jar!/:5.3.10]

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.