Code Monkey home page Code Monkey logo

dotnet-spanner-nhibernate's Introduction

Google APIs

This repository contains the original interface definitions of public Google APIs that support both REST and gRPC protocols. Reading the original interface definitions can provide a better understanding of Google APIs and help you to utilize them more efficiently. You can also use these definitions with open source tools to generate client libraries, documentation, and other artifacts.

Building

Bazel

The recommended way to build the API client libraries is through Bazel >= 4.2.2.

First, install bazel.

To build all libraries:

bazel build //...

To test all libraries:

bazel test //...

To build one library in all languages:

bazel build //google/example/library/v1/...

To build the Java package for one library:

bazel build //google/example/library/v1:google-cloud-example-library-v1-java

Bazel packages exist in all the libraries for Java, Go, Python, Ruby, Node.js, PHP and C#.

Overview

Google APIs are typically deployed as API services that are hosted under different DNS names. One API service may implement multiple APIs and multiple versions of the same API.

Google APIs use Protocol Buffers version 3 (proto3) as their Interface Definition Language (IDL) to define the API interface and the structure of the payload messages. The same interface definition is used for both REST and RPC versions of the API, which can be accessed over different wire protocols.

There are several ways of accessing Google APIs:

  1. JSON over HTTP: You can access all Google APIs directly using JSON over HTTP, using Google API client library or third-party API client libraries.

  2. Protocol Buffers over gRPC: You can access Google APIs published in this repository through GRPC, which is a high-performance binary RPC protocol over HTTP/2. It offers many useful features, including request/response multiplex and full-duplex streaming.

  3. Google Cloud Client Libraries: You can use these libraries to access Google Cloud APIs. They are based on gRPC for better performance and provide idiomatic client surface for better developer experience.

Discussions

This repo contains copies of Google API definitions and related files. For discussions or to raise issues about Google API client libraries, GRPC or Google Cloud Client Libraries please refer to the repos associated with each area.

Repository Structure

This repository uses a directory hierarchy that reflects the Google API product structure. In general, every API has its own root directory, and each major version of the API has its own subdirectory. The proto package names exactly match the directory: this makes it easy to locate the proto definitions and ensures that the generated client libraries have idiomatic namespaces in most programming languages. Alongside the API directories live the configuration files for the GAPIC toolkit.

NOTE: The major version of an API is used to indicate breaking change to the API.

Generate gRPC Source Code

To generate gRPC source code for Google APIs in this repository, you first need to install both Protocol Buffers and gRPC on your local machine, then you can run make LANGUAGE=xxx all to generate the source code. You need to integrate the generated source code into your application build system.

NOTE: The Makefile is only intended to generate source code for the entire repository. It is not for generating linkable client library for a specific API. Please see other repositories under https://github.com/googleapis for generating linkable client libraries.

Go gRPC Source Code

It is difficult to generate Go gRPC source code from this repository, since Go has different directory structure. Please use this repository instead.

dotnet-spanner-nhibernate's People

Contributors

alexandrtrf avatar amanda-tarafa avatar chingor13 avatar dependabot[bot] avatar google-cloud-policy-bot[bot] avatar jskeet avatar olavloite avatar release-please[bot] avatar renovate-bot avatar skuruppu avatar surbhigarg92 avatar

Stargazers

 avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

dotnet-spanner-nhibernate's Issues

Flaky test failure

[xUnit.net 00:00:02.32]     Google.Cloud.Spanner.Connection.MockServer.MockSpannerServerTests.WriteMutations [FAIL]
[xUnit.net 00:00:02.32]       System.InvalidCastException : Unable to cast object of type 'Google.Cloud.Spanner.V1.BeginTransactionRequest' to type 'Google.Cloud.Spanner.V1.CommitRequest'.
[xUnit.net 00:00:02.32]       Stack Trace:
[xUnit.net 00:00:02.32]         D:\a\dotnet-spanner-nhibernate\dotnet-spanner-nhibernate\Google.Cloud.Spanner.Connection.Tests\MockServer\MockSpannerServerTests.cs(104,0): at Google.Cloud.Spanner.Connection.MockServer.MockSpannerServerTests.WriteMutations()
[xUnit.net 00:00:02.32]         --- End of stack trace from previous location where exception was thrown ---
  X Google.Cloud.Spanner.Connection.MockServer.MockSpannerServerTests.WriteMutations [67ms]
  Error Message:
   System.InvalidCastException : Unable to cast object of type 'Google.Cloud.Spanner.V1.BeginTransactionRequest' to type 'Google.Cloud.Spanner.V1.CommitRequest'.
  Stack Trace:
     at Google.Cloud.Spanner.Connection.MockServer.MockSpannerServerTests.WriteMutations() in D:\a\dotnet-spanner-nhibernate\dotnet-spanner-nhibernate\Google.Cloud.Spanner.Connection.Tests\MockServer\MockSpannerServerTests.cs:line 104
--- End of stack trace from previous location where exception was thrown ---

feat: Add support for server-side DEFAULT column values

The Cloud Spanner NHibernate dialect currently supports DEFAULT column values using a client side implementation. Cloud Spanner now however supports DEFAULT column values, and the NHibernate dialect should use this instead when possible. A column with a default value can't be a commit timestamp column. The implementation for this must therefore remain client side in the dialect implementation.

See also https://cloud.google.com/spanner/docs/reference/standard-sql/data-definition-language#description_2

Implement optimistic concurrency control when using mutations

Optimistic concurrency control is normally done by NHibernate by adding a WHERE Id=:id AND Version=:version to a DML statement. It is not possible to add such a where clause when using mutations for updating/deleting a record. Instead, NHibernate should in that case execute a SELECT 1 FROM <table> WHERE Id=:id AND Version=:version in the same read/write transaction as where the mutation will be applied. This will ensure that:

  1. If the query actually returns a row, we know for sure that the condition holds (i.e. the version has not changed).
  2. The row will be locked during the read/write transaction, which again means that we know for sure that the condition holds true until the transaction has committed.

The tests that verify the same behavior for Entity Framework can be found here:

  1. https://github.com/googleapis/dotnet-spanner-entity-framework/blob/fdc22c1566be535ad7f6a167fe29ca2bc805a76b/Google.Cloud.EntityFrameworkCore.Spanner.Tests/EntityFrameworkUsingMutationsMockServerTests.cs#L304
  2. https://github.com/googleapis/dotnet-spanner-entity-framework/blob/fdc22c1566be535ad7f6a167fe29ca2bc805a76b/Google.Cloud.EntityFrameworkCore.Spanner.Tests/EntityFrameworkUsingMutationsMockServerTests.cs#L392
  3. https://github.com/googleapis/dotnet-spanner-entity-framework/blob/fdc22c1566be535ad7f6a167fe29ca2bc805a76b/Google.Cloud.EntityFrameworkCore.Spanner.IntegrationTests/VersioningTests/VersioningTests.cs#L23

The last point references integration tests that will automatically use mutations instead of DML. That is different in Entity Framework and NHibernate:

  1. Entity Framework will automatically use mutations when no explicit transaction is started. This is the case for the above mentioned integration tests.
  2. NHibernate never uses mutations unless it is explicitly told to do so. You can instruct NHibernate to use mutations for a transaction by calling BeginTransaction(MutationUsage.Always). See this test case for an example.

Test that single (Batch) DML statements are retried if aborted

The Connection API retries explicit transactions using a checksum strategy if these are aborted by Cloud Spanner. For single (Batch) DML statements the checksum strategy is overkill, and instead the underlying transactions should use the Spanner data client SpannerRetriableTransaction version.

Merge the two SpannerEntityPersister classes

The Spanner NHibernate driver contains two different SpannerEntityPersister instances. This is probably confusing for users, and it is better to merge the two into one and instruct all users to always use the only SpannerEntityPersister.

config: Consider setting a number of defaults for the dialect

The following configuration should be considered for inclusion in the default settings for the dialect:

  • BatchVersionedData: Setting this to default true will ensure that updates/deletes that include a version check will also be batched (i.e. use BatchDML or Mutations)
  • UseComments: Setting this to default true will make it less likely that anyone makes a misconfiguration when they want to use query hints. Query hints are included in a query by first adding them as a comment, and not having comments enabled will cause hints not to be generated.
  • WrapResultSets: This will ensure calls to ResultSet.GetOrdinal will be cached.

EDIT:

  • UseComments: This should not be enabled by default, as comments are only necessary for query hints. Query hints also require a query interceptor to be added to the configuration (and interceptors cannot be added to the default configuration), which means that users will always need to take action to use query hints. Requiring users to add two items to the configuration instead of only one does not make a very big difference, while setting UseComments to true by default would mean that the dialect would always generate additional SQL comments for each statement that would not be needed in most cases.
  • WrapResultSets: This option has been tested with the benchmarks, and did not make any measurable difference. Adding it to the default configuration is therefore not necessary.

Support JOIN hints

JOIN hints can (probably) be supported in the same way as TABLE hints are currently supported.

Merge the Spanner entity persisters into one

The Spanner driver currently has two custom entity persisters:

  • One that supports generating mutations
  • One that supports fixed literal values, for example for PENDING_COMMIT_TIMESTAMP()

These entity persisters should be merged into one general SpannerSingleTableEntityPersister.

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Ignored or Blocked

These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.

Detected dependencies

github-actions
.github/workflows/build-and-test-on-windows.yml
  • actions/checkout v3
  • actions/setup-dotnet v2
.github/workflows/build-and-test.yml
  • actions/checkout v3
  • actions/setup-dotnet v2
.github/workflows/integration-tests-on-emulator.yml
  • actions/checkout v3
  • actions/setup-dotnet v2
.github/workflows/integration-tests-on-production.yml
  • actions/checkout v3
  • actions/setup-dotnet v2
nuget
Google.Cloud.Spanner.Connection.Tests/Google.Cloud.Spanner.Connection.Tests.csproj
  • coverlet.collector 3.1.2
  • xunit.runner.visualstudio 2.4.5
  • Xunit.Combinatorial 1.4.1
  • xunit 2.4.2
  • Microsoft.NET.Test.Sdk 17.3.0
Google.Cloud.Spanner.Connection/Google.Cloud.Spanner.Connection.csproj
  • Google.Cloud.Spanner.Data 3.15.1
Google.Cloud.Spanner.NHibernate.Benchmarks/Google.Cloud.Spanner.NHibernate.Benchmarks.csproj
  • BenchmarkDotNet 0.13.1
Google.Cloud.Spanner.NHibernate.IntegrationTests/Google.Cloud.Spanner.NHibernate.IntegrationTests.csproj
  • Xunit.SkippableFact 1.4.13
  • coverlet.collector 3.1.2
  • xunit.runner.visualstudio 2.4.5
  • Xunit.Combinatorial 1.4.1
  • xunit 2.4.2
  • Microsoft.NET.Test.Sdk 17.3.0
Google.Cloud.Spanner.NHibernate.Samples/Google.Cloud.Spanner.NHibernate.Samples.csproj
  • Docker.DotNet 3.125.10
Google.Cloud.Spanner.NHibernate.Tests/Google.Cloud.Spanner.NHibernate.Tests.csproj
  • coverlet.collector 3.1.2
  • xunit.runner.visualstudio 2.4.5
  • Xunit.Combinatorial 1.4.1
  • xunit 2.4.2
  • Microsoft.NET.Test.Sdk 17.3.0
Google.Cloud.Spanner.NHibernate/Google.Cloud.Spanner.NHibernate.csproj
  • NHibernate 5.3.12
pip_requirements
.kokoro/requirements.txt
  • attrs ==22.1.0
  • cachetools ==5.2.0
  • certifi ==2022.6.15.2
  • cffi ==1.15.1
  • charset-normalizer ==2.1.1
  • click ==8.0.4
  • colorama ==0.4.5
  • cryptography ==38.0.1
  • gcp-releasetool ==1.8.7
  • google-auth ==2.11.0
  • idna ==3.4
  • importlib-metadata ==4.12.0
  • jaraco-classes ==3.2.2
  • jinja2 ==3.1.2
  • keyring ==23.9.1
  • markupsafe ==2.1.1
  • more-itertools ==8.14.0
  • packaging ==21.3
  • protobuf ==3.20.2
  • pyasn1 ==0.4.8
  • pyasn1-modules ==0.2.8
  • pycparser ==2.21
  • pyjwt ==2.4.0
  • pyparsing ==3.0.9
  • pyperclip ==1.8.2
  • python-dateutil ==2.8.2
  • pywin32-ctypes ==0.2.0
  • requests ==2.28.1
  • rsa ==4.9
  • six ==1.16.0
  • typing-extensions ==4.3.0
  • urllib3 ==1.26.12
  • zipp ==3.8.1

  • Check this box to trigger a request for Renovate to run again on this repository

Create benchmarks

Benchmark NHibernate using the same scenarios as for EF Core.

System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
 ---> Google.Cloud.Spanner.Data.SpannerException: The operation was aborted.
 ---> Grpc.Core.RpcException: Status(StatusCode="Aborted", Detail="Transaction aborted. Database schema probably changed during transaction, retry may succeed.", DebugException="Grpc.Core.Internal.CoreErrorDetailException: {"created":"@1640177671.725563766","description":"Error received from peer ipv4:108.177.111.95:443","file":"/var/local/git/grpc/src/core/lib/surface/call.cc","file_line":1067,"grpc_message":"Transaction aborted. Database schema probably changed during transaction, retry may succeed.","grpc_status":10}")
   at Grpc.Gcp.GcpCallInvoker.<>c__DisplayClass27_0`2.<<AsyncUnaryCall>g__PostProcessPropagateResult|3>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Google.Api.Gax.Grpc.ApiCallRetryExtensions.<>c__DisplayClass0_0`2.<<WithRetry>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Google.Cloud.Spanner.V1.Internal.ExecuteHelper.WithSessionExpiryChecking[T](Task`1 task, Session session)
   at Google.Cloud.Spanner.V1.PooledSession.RecordSuccessAndExpiredSessions[T](Task`1 task)
   at Google.Cloud.Spanner.V1.PooledSession.CommitAsync(CommitRequest request, CallSettings callSettings)
   at Google.Cloud.Spanner.Data.SpannerTransaction.<>c__DisplayClass53_0.<<CommitAsync>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Google.Cloud.Spanner.Data.ExecuteHelper.WithErrorTranslationAndProfiling[T](Func`1 t, String name, Logger logger)
   --- End of inner exception stack trace ---
   at Google.Cloud.Spanner.Data.ExecuteHelper.WithErrorTranslationAndProfiling[T](Func`1 t, String name, Logger logger)
   at Google.Api.Gax.TaskExtensions.WaitWithUnwrappedExceptions(Task task)
   at Google.Cloud.Spanner.Data.SpannerTransaction.Commit(DateTime& timestamp)
   at Google.Cloud.Spanner.NHibernate.Benchmarks.SpannerNHibernateBenchmarks.SelectMultipleSingersInReadWriteTransactionSpanner() in /home/koloite/dotnet-spanner-nhibernate/Google.Cloud.Spanner.NHibernate.Benchmarks/SpannerNHibernateBenchmarks.cs:line 343
   at BenchmarkDotNet.Autogenerated.Runnable_12.WorkloadActionNoUnroll(Int64 invokeCount) in /home/koloite/dotnet-spanner-nhibernate/Google.Cloud.Spanner.NHibernate.Benchmarks/bin/Debug/netcoreapp3.1/3d391572-0dcf-44f0-b8ae-1de4d6327430/3d391572-0dcf-44f0-b8ae-1de4d6327430.notcs:line 9287
   at BenchmarkDotNet.Engines.Engine.RunIteration(IterationData data)
   at BenchmarkDotNet.Engines.EngineStage.RunIteration(IterationMode mode, IterationStage stage, Int32 index, Int64 invokeCount, Int32 unrollFactor)
   at BenchmarkDotNet.Engines.EngineActualStage.RunAuto(Int64 invokeCount, IterationMode iterationMode, Int32 unrollFactor)
   at BenchmarkDotNet.Engines.EngineActualStage.Run(Int64 invokeCount, IterationMode iterationMode, Boolean runAuto, Int32 unrollFactor, Boolean forceSpecific)
   at BenchmarkDotNet.Engines.EngineActualStage.RunWorkload(Int64 invokeCount, Int32 unrollFactor, Boolean forceSpecific)
   at BenchmarkDotNet.Engines.Engine.Run()
   at BenchmarkDotNet.Autogenerated.Runnable_12.Run(IHost host, String benchmarkName) in /home/koloite/dotnet-spanner-nhibernate/Google.Cloud.Spanner.NHibernate.Benchmarks/bin/Debug/netcoreapp3.1/3d391572-0dcf-44f0-b8ae-1de4d6327430/3d391572-0dcf-44f0-b8ae-1de4d6327430.notcs:line 8968
   --- End of inner exception stack trace ---
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor, Boolean wrapExceptions)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at System.Reflection.MethodBase.Invoke(Object obj, Object[] parameters)
   at BenchmarkDotNet.Autogenerated.UniqueProgramName.AfterAssemblyLoadingAttached(String[] args) in /home/koloite/dotnet-spanner-nhibernate/Google.Cloud.Spanner.NHibernate.Benchmarks/bin/Debug/netcoreapp3.1/3d391572-0dcf-44f0-b8ae-1de4d6327430/3d391572-0dcf-44f0-b8ae-1de4d6327430.notcs:line 66

Add sample / documentation for connection usage

Users must use the SpannerRetriableConnection and not SpannerConnection with the NHibernate driver. This is automatically done when the users allow the driver to create the connection, but when supplying a connection manually, the user is responsible for supplying the correct type of connection.

[Policy Bot] found one or more issues with this repository.

Policy Bot found one or more issues with this repository.

  • Default branch is 'main'
  • Branch protection is enabled
  • Renovate bot is enabled
  • Merge commits disabled
  • There is a CODEOWNERS file
  • There is a valid LICENSE.md
  • There is a CODE_OF_CONDUCT.md
  • There is a CONTRIBUTING.md
  • There is a SECURITY.md

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.