hyperledger / fabric-gateway Goto Github PK

View Code? Open in Web Editor NEW

134.0 10.0 86.0 5.21 MB

Go, Node and Java client API for Hyperledger Fabric v2.4+

Home Page: https://hyperledger.github.io/fabric-gateway/

License: Apache License 2.0

Makefile 0.68% Go 29.17% JavaScript 0.25% Gherkin 3.12% Shell 0.42% TypeScript 29.54% Java 36.82%

hacktoberfest blockchain go golang hyperledger hyperledger-fabric java typescript node node-js

fabric-gateway's People

Contributors

Stargazers

Watchers

Forkers

bestbeforetoday wanglg007 andrew-coleman jt-nti lesleyannjordan nadongjun kemi04 ndwadia yiwenlong ericyk r2ys void-software softallice despitedeath davidkel mbwhite itasks acrowise sapthasurendran ksanjaykumar1 deeptiraom tomcn0803 nanra cyanlink rhy000rhy adam-booth naonishijima sb2407 jasonceng rootchain-kr ggessner affiner drhanlondon kuangxianhua hans-van-velzen matega89 memduhb leicc520 melissa1412 hanoolchoi75 amosnewt chanachana dangrenren karthikkavina luoshaoxia pinkdiamond1 elenduuche luigieai vishwas031 mik-patient carma2002 fahmiosman aiguo186 johannww completist29 shenzhongshan bhaskarvilles bviswana101 nikirepmvby sathishvjd sberenson mady-camara dataspider-fan aruns05 hyplegfabricvot lhrbeijing developfast talhayasinkeskin asbaharoon java-cds-club shiva953 rajat-dlt qiita-mobileapps clouds1st nithin-pankaj pravici iyersk happyzyxy demorripv osemuaimiosior satota2 toni171 denyeart mbrandenburger ryotakuzuya tarun5139

fabric-gateway's Issues

Scenario tests for transaction error response

As an application developer
I want to receive error response messages returned by transaction functions
So that I can diagnose invocation errors

Additional details may be associated with the errors returned but the top-level error message should contain the error response message returned by the transaction function.

Unexpected error message when chaincode id doesn't exist

In both Go and Node clients If I specify a non existent chaincode then I get the following on evaluate

no endorsing peers found for channel: mychannel

Which isn't really telling me that I have not provided the correct chaincode id.

On a submit I get something a little more helpful

discovery service failed to build endorsement plan: No metadata was found for chaincode fixed-asset1 in channel mychannel

But still that error could be more helpful to say that there is no chaincode with this ID instantiated on that channel

Gateway system testing

The following categories of 'system test' will be prioritized:

adhoc test - small amount of using and poking to assess initial user experience, error handling, problem determination (Dave K)
functional error scenarios - e.g. endorsements don't match, various timeouts, etc - prioritize relative to other functional tests (Andy)
nodes becoming unavailable - gateway peer (without load balancer), endorsing peers (with and without sufficient remaining endorsers), and orderers (with and without consensus) become unavailable (Dave K)
gateway peer with K8s load balancer - gateway peer becomes unavailable, automatically fail over to another peer in the org (Matthew/Josh)
config changes - ensure discovery and gateway work well together (Saptha)

Migrate fabric-samples to use Gateway SDKs

The following samples should be extended with gateway app samples. We don't need to have a full matrix of all samples in a languages however. Items with a 'P' are initial priorities for this work item. Add your name next to an item if you intend to pick it up.

Higher priority

asset-transfer-basic (demonstrates transaction submit / evaluate, submit error capture)
- Typescript - P1 - Saptha
- Java - P2 - Deepti
- Go - P2 - Saptha
asset-transfer-events (demonstrates chaincode events)
- Typescript - P3 - Saptha
- Java - P3 - Deepti
- Go - P3 - Mark
asset-transfer-private-data (demonstrates auto-magic handling of endorsement with private data collections)
- Typescript - P4 - Saptha
- Java - P4
- Go - P4
asset-transfer-secured-agreement (demonstrates auto-magic handling of state-based endorsement)
- Typescript - P4 - Saptha
- Java
- Go
off_chain_data (demonstrates block eventing, checkpointing)
- TypeScript - Mark
- Java - Mark
- Go
full-stack-asset-transfer-guide trader application
- Go
- Java

Lower priority

asset-transfer-ledger-queries
- Typescript - P5
- Java
- Go
asset-transfer-sbe
- Typescript - P5
- Java
- Go

What about other samples? For example the old fabcar samples and commercial paper?

Timeouts for service invocations

As an application developer
I want to specify timeouts for service invocations such as endorse and submit
So that my application will not wait indefinitely or timeout too quickly for my particular usage scenario

Currently some language implements have an arbitrary timeout hard-coded, which cannot be modified, while some have no timeout set so wait indefinitely in cases such as packets being black-holed.

It should be possible to specify:

timeouts on a per-invocation basis
timeout defaults when connecting the Gateway

Suggest exposing elements of the gRPC API as invocation options rather than providing an abstraction that duplicates the capabilities provided by gRPC and then needs to be maintained and supported. For example, in the Node client we could accept a gRPC CallOption as an option to service invocation, which allows a deadline for a particular call to be specified. Go accepts a Context that can include a deadline or timeout, or can be used to explicitly cancel the call (in addition to CallOption to configure other call behaviour).

See gRPC blog post on deadlines.

Tasks:

Java per-call timeouts
Java default timeouts
Update Java sample to use default timeouts
Node per-call timeouts
Node default
Update Node sample to use default timeouts
Go per-call timeouts
Go default timeouts
Update Go sample to use (or demonstrate) default timeouts

Support for Java 17 LTS

LTS start date for Java 17 is 2021-09-14.

Add test runs using Java 17 to build
Update supported versions documentation
Build Javadoc with Java 17 to get latest look & feel

Run vulnerability scan in nightly build

As a maintainer
I want a vulnerability scan run as part of a nightly build
So that I can get notified about and fix security vulnerabilities in dependencies

The vulnerability scan should be a separate build stage from tests to give clear feedback on which aspects are passing and failing.

The Maven Java build has an owasp profile than can be used to run a vulnerability scan, for example: mvn verify -D skipTests -P owasp

The Node build could use npm audit.

The Go build could use gosec. This is really a linter looking for potential bad security practices in code, rather than scanning for known vulnerabilities in dependencies, so probably should run as part of the regular linting in all builds.

For vulnerability scan of Go module dependencies, nancy could be used.

Additionally, we can make use of GitHub's dependabot security vulnerability alerts. These need to be enabled for the repository.

good practice for gateway peer loadbalancing

We should provide guidance on this as part of samples/documentation some considerations are

load balancers in a K8s environment
client handling the balancing itself

Please feel free to add more thoughts, info for capturing plus where appropriate artifacts demonstrating this should be created

eg samples, docs

In the peer logs the gateway logs it's sending to an orderer but it doesn't list which orderer

[gateway] broadcast -> Submitting txn to orderer

this would be really handy to have the orderer it was actually trying to send to

Commit status API improvements

Particularly in Go, the flow for interrogating transaction commit status is quite cumbersome:

if successful, err := commit.Successful(); err != nil {
    panic(fmt.Errorf("failed to get commit status: %w", err))
} else if !successful {
    status, err := commit.Status()
    if err != nil {
        panic(err)
    }
    panic(fmt.Errorf("transaction failed to commit with status: %d", int32(status)))
}

blockNumber, err := commit.BlockNumber()
if err != nil {
    panic(err)
}

This is because the implementation invokes the TransactionStatus service on whichever function call is first made on the Commit struct so, while the results are cached so any calls after the first successful call will succeed and not return an error, every function call is potentially a remote service invocation. This adds a lot of unnecessary error handling overhead to the API.

This should be refactored so that the Commit only provides a Status() function, which does the remote service invocation and returns a struct containing all the resulting information. This means that the Status() function may return an error, but no error handling is required to access any of the resulting information. The equivalent usage example would then look like:

status, err := commit.Status()
if err != nil {
    panic(fmt.Errorf("failed to get commit result: %w", err))
}

if !status.Successful {
    panic(fmt.Errorf("transaction failed to commit with status: %d", int32(status.Code)))
}

blockNumber := status.BlockNumber

For consistency and to promote the use of immutable objects in the implementation, both Node and Java APIs should be refactored to match this pattern.

Better error responses from Go client compared to Node Client

If I pass in an invalid string for a Certificate or Private Key, in Go I get

failed to parse certificate PEM
failed to parse private key PEM

However if I use node then I get for a certificate

failed to endorse transaction: error validating proposal: access denied: channel [mychannel] creator org unknown, creator is malformed

Which is a response from fabric

and for a private key

Error: error:0909006C:PEM routines:get_name:no start line

because of calling crypto.createPrivateKey

So in the case of Go it's error messages are more helpful using the thin client identity facilities, but in node we are subject to a fabric response or a node.js response, both of which are not the most helpful.

This issue is just to highlight a minor inconsistency between sdks

Intermitted private data scenario test failure

The following error is fairly common in builds:

Scenario: Org1 writes private data to SharedCollection, endorsed by Org3, and can read it back via Org1 peer # /home/vsts/work/1/s/scenario/features/privatedata.feature:78
  Then the response should be "value-105" # /home/vsts/work/1/s/scenario/features/privatedata.feature:91
    Error: transaction response "" does not match expected value "value-105"

This instance of the error occurred in the Go scenario tests, but I think they have happened intermittently in other language scenario tests.

Async submit example in Java API documentation

The Javadoc for the Contract class should include a code example for async submit.

Return gRPC errors directly to application in Go client

As an application developer
I want to have direct access to gRPC errors
So that I can convert them to gRPC status and interrogate their code and additional details associated with the error

Avoid wrapping any errors from gRPC calls. Rely on the Gateway to provide an appropriate error message.

Need best practices for constructing GRPC clients documented or in samples

Using just node as a client for now I messed around with changing the parameters to GrpcClient creation to see how helpful it was when something was not done right

Incorrect TLS Cert for peer in org2 when communicating to peer in org1

Error: 14 UNAVAILABLE: No connection established
    at Object.callErrorFromStatus (/home/dave/github-mine/fabric-gateway/samples/node/node_modules/@grpc/grpc-js/build/src/call.js:31:26)
    at Object.onReceiveStatus (/home/dave/github-mine/fabric-gateway/samples/node/node_modules/@grpc/grpc-js/build/src/client.js:179:52)
    at Object.onReceiveStatus (/home/dave/github-mine/fabric-gateway/samples/node/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:336:141)
    at Object.onReceiveStatus (/home/dave/github-mine/fabric-gateway/samples/node/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:299:181)
    at /home/dave/github-mine/fabric-gateway/samples/node/node_modules/@grpc/grpc-js/build/src/call-stream.js:145:78
    at processTicksAndRejections (internal/process/task_queues.js:79:11) {
  code: 14,
  details: 'No connection established',
  metadata: Metadata { internalRepr: Map {}, options: {} }

invalid hostname

Error: 14 UNAVAILABLE: Name resolution failed for target dns:localhost2:7051
    at Object.callErrorFromStatus (/home/dave/github-mine/fabric-gateway/samples/node/node_modules/@grpc/grpc-js/build/src/call.js:31:26)
    at Object.onReceiveStatus (/home/dave/github-mine/fabric-gateway/samples/node/node_modules/@grpc/grpc-js/build/src/client.js:179:52)
    at Object.onReceiveStatus (/home/dave/github-mine/fabric-gateway/samples/node/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:336:141)
    at Object.onReceiveStatus (/home/dave/github-mine/fabric-gateway/samples/node/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:299:181)
    at /home/dave/github-mine/fabric-gateway/samples/node/node_modules/@grpc/grpc-js/build/src/call-stream.js:145:78
    at processTicksAndRejections (internal/process/task_queues.js:79:11) {
  code: 14,
  details: 'Name resolution failed for target dns:localhost2:7051',
  metadata: Metadata { internalRepr: Map {}, options: {} }

invalid port

Error: 14 UNAVAILABLE: No connection established
    at Object.callErrorFromStatus (/home/dave/github-mine/fabric-gateway/samples/node/node_modules/@grpc/grpc-js/build/src/call.js:31:26)
    at Object.onReceiveStatus (/home/dave/github-mine/fabric-gateway/samples/node/node_modules/@grpc/grpc-js/build/src/client.js:179:52)
    at Object.onReceiveStatus (/home/dave/github-mine/fabric-gateway/samples/node/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:336:141)
    at Object.onReceiveStatus (/home/dave/github-mine/fabric-gateway/samples/node/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:299:181)
    at /home/dave/github-mine/fabric-gateway/samples/node/node_modules/@grpc/grpc-js/build/src/call-stream.js:145:78
    at processTicksAndRejections (internal/process/task_queues.js:79:11) {
  code: 14,
  details: 'No connection established',
  metadata: Metadata { internalRepr: Map {}, options: {} }

incorrect grpc override

Error: 14 UNAVAILABLE: No connection established
    at Object.callErrorFromStatus (/home/dave/github-mine/fabric-gateway/samples/node/node_modules/@grpc/grpc-js/build/src/call.js:31:26)
    at Object.onReceiveStatus (/home/dave/github-mine/fabric-gateway/samples/node/node_modules/@grpc/grpc-js/build/src/client.js:179:52)
    at Object.onReceiveStatus (/home/dave/github-mine/fabric-gateway/samples/node/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:336:141)
    at Object.onReceiveStatus (/home/dave/github-mine/fabric-gateway/samples/node/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:299:181)
    at /home/dave/github-mine/fabric-gateway/samples/node/node_modules/@grpc/grpc-js/build/src/call-stream.js:145:78
    at processTicksAndRejections (internal/process/task_queues.js:79:11) {
  code: 14,
  details: 'No connection established',
  metadata: Metadata { internalRepr: Map {}, options: {} }

peer endpoint set to 'www.microsoft.com:7051'
Application just hangs

It may or may not be possible to improve some of these responses as this is purely down to Grpc for node, but I think at least we should have a timeout for a grpc connection via the waitForReady api ?

Upgrade to Go 1.16

Migrate the Go client module, scenario tests and samples to use Go 1.16.

Stop receiving chaincode events

As an application developer
I want to stop receiving chaincode events
So that I can stop event processing at the end of a batch window, and restart again later

Explicit capability for stopping event listening is currently only exposed in the Go client, and is achieved by cancelling the context provided when when requesting events. This cancels the server stream and therefore stops event delivery.

The Node client has internal capability to cancel the server stream but this is not exposed by the client API.

The Java client does obviously have an internal mechanism to cancel the server stream but this needs further investigation. Perhaps Possibly using a different client stub implementation may make this capability accessible, but care needs to be taken to maintain client control over the rate of delivery of events.

A workaround to allow even listening to be stopped in Node and Java client APIs is to shutdown the gRPC client / channel used by the Gateway connection. Provided the application uses separate gRPC channels (and Gateway connection built on those channels) for event listening and other transactional, this would allow event listening to be cancelled while allowing other transactional work to continue uninterrupted.

This behaviour is tied to the ChaincodeEvents gRPC service implementation, which is a server streaming service, so the only way for the client to end event delivery is to send the number of events to deliver as part of the initial request (which we do not do), or to cancel the server stream. An alternative implementation approach would be to use bi-directional streaming so the client could send a cancel message to the server after event delivery has been initiated. This seems to be a pattern favoured by gRPC implementors but is a more complex interaction pattern and would require refactoring of the gRPC service definition and server-side implementation in addition to client changes.

Blank error received when using using an identity from unknown authority

For the node client I use a valid identity but created by a CA that isn't part of the channel or consortium, in effect it is an unknown authority. I see the following in the peer logs

2021-09-21 15:32:21.736 UTC 00bb WARN [endorser] Validate -> access denied: channel the supplied identity is not valid: x509: certificate signed by unknown authority channel=mychannel txID=4a39ce0f
2021-09-21 15:32:21.736 UTC 00bc INFO [comm.grpc.server] 1 -> unary call completed grpc.service=gateway.Gateway grpc.method=Evaluate grpc.peer_address=172.21.0.1:37164 error="rpc error: code = Unknown desc = failed to evaluate transaction: error validating proposal: access denied: channel [mychannel] creator org unknown, creator is malformed" grpc.code=Unknown grpc.call_duration=396.8µs

An error is thrown, but there is nothing in the error message.

This happens for a submit or evaluate and I used the node sdk thin client.

Gateway error scenarios

Add extra scenario tests to the suite that specifically test the error messages returned from Fabric Gateway.
Ensure relevant messages are easily accessible and contain appropriate diagnostic information.
Ensure consistency across the three SDKs.

Unable to submit txn when using the default endorsement policy

I have a 3 org, 2 peer network and deploy a chaincode without an endorsement policy (so should default to majority I believe). However every submission fails with the error

Error: 10 ABORTED: failed to endorse transaction: [{address:"peer0.org2.example.com:9051" msp_id:"Org2MSP" message:"error 500, error in simulation: failed to execute transaction  45987671840eb72f8c85e26d3b745d17272c7c94d723c2916b897b84b4e1db26: error sending: txid:  45987671840eb72f8c85e26d3b745d17272c7c94d723c2916b897b84b4e1db26(mychannel) exists" }]

And I see in the peer logs

 callChaincode -> finished chaincode: basic duration: 3ms channel=mychannel txID=45987671
2021-10-12 08:54:09.761 UTC 00ec INFO [endorser] buildChaincodeInterest -> ccInterest chaincodes:<name:"basic" >
2021-10-12 08:54:09.761 UTC 00ed INFO [gateway] Endorse -> Seeking extra endorsements from: Org3MSP=peer1.org3.example.com:11151 Org2MSP=peer1.org2.example.com:9151 Org2MSP=peer1.org2.example.com:9151
2021-10-12 08:54:09.767 UTC 00ee INFO [comm.grpc.server] 1 -> unary call completed grpc.service=gateway.Gateway grpc.method=Endorse grpc.peer_address=192.168.0.1:48138 error="rpc error: code = Aborted desc = failed to endorse transaction: [{address:\"peer1.org2.example.com:9151\" msp_id:\"Org2MSP\" message:\"error 500, error in simulation: failed to execute transaction 45987671840eb72f8c85e26d3b745d17272c7c94d723c2916b897b84b4e1db26: error sending: txid: 45987671840eb72f8c85e26d3b745d17272c7c94d723c2916b897b84b4e1db26(mychannel) exists\" }]" grpc.code=Aborted grpc.call_duration=9.9132ms

I've tried the following explicit endorsement policies but I am unable to recreate it currently using an explicit endorsement policy

AND('Org1MSP.member','Org2MSP.member','Org3MSP.member')

AND('Org1MSP.member','Org2MSP.member')

OR(AND('Org1MSP.member','Org2MSP.member'),AND('Org1MSP.member','Org3MSP.member'),AND('Org3MSP.member','Org2MSP.member'))

Gateway peer logs flooded with errors

Testing the new endorsement logic, while my client is running all seems fine but as soon as I stop the client the peer logs are flooded with I stopped the client at 10:35:33 and there are no errors in the log prior to that

2021-10-27 10:35:33.425 UTC 0687 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org2.example.com:9151 mspid=Org2MSP
2021-10-27 10:35:33.425 UTC 0688 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org2.example.com:9151 mspid=Org2MSP
2021-10-27 10:35:33.425 UTC 0689 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org2.example.com:9151 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.425 UTC 068a INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org2.example.com:9051 mspid=Org2MSP
2021-10-27 10:35:33.425 UTC 068b INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org3.example.com:11151 mspid=Org3MSP
2021-10-27 10:35:33.425 UTC 068c INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org3.example.com:11051 mspid=Org3MSP
2021-10-27 10:35:33.425 UTC 068d INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org2.example.com:9151 mspid=Org2MSP
2021-10-27 10:35:33.425 UTC 068f ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org2.example.com:9151 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.425 UTC 068e INFO [comm.grpc.server] 1 -> unary call completed grpc.service=gateway.Gateway grpc.method=Endorse grpc.peer_address=172.20.0.1:39872 error="rpc error: code = Aborted desc = failed to endorse transaction, see attached details for more info" grpc.code=Aborted grpc.call_duration=2.3953209s
2021-10-27 10:35:33.425 UTC 0690 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org2.example.com:9051 mspid=Org2MSP
2021-10-27 10:35:33.425 UTC 0691 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org2.example.com:9051 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.426 UTC 0692 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org2.example.com:9151 mspid=Org2MSP
2021-10-27 10:35:33.426 UTC 0693 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org2.example.com:9151 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.426 UTC 0694 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org2.example.com:9051 mspid=Org2MSP
2021-10-27 10:35:33.426 UTC 0695 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org2.example.com:9051 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.426 UTC 0696 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org2.example.com:9151 mspid=Org2MSP
2021-10-27 10:35:33.426 UTC 0697 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org2.example.com:9151 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.426 UTC 0698 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org2.example.com:9151 mspid=Org2MSP
2021-10-27 10:35:33.426 UTC 0699 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org2.example.com:9151 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.426 UTC 069a INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org2.example.com:9151 mspid=Org2MSP
2021-10-27 10:35:33.426 UTC 069b ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org2.example.com:9151 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.426 UTC 069c INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org2.example.com:9151 mspid=Org2MSP
2021-10-27 10:35:33.426 UTC 069d ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org2.example.com:9151 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.426 UTC 069e INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org2.example.com:9151 mspid=Org2MSP
2021-10-27 10:35:33.427 UTC 069f ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org2.example.com:9151 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.427 UTC 06a0 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org2.example.com:9151 mspid=Org2MSP
2021-10-27 10:35:33.427 UTC 06a1 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org2.example.com:9151 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.427 UTC 06a2 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org2.example.com:9151 mspid=Org2MSP
2021-10-27 10:35:33.427 UTC 06a3 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org2.example.com:9151 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.427 UTC 06a4 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org2.example.com:9151 mspid=Org2MSP
2021-10-27 10:35:33.427 UTC 06a5 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org2.example.com:9151 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.427 UTC 06a6 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org2.example.com:9151 mspid=Org2MSP
2021-10-27 10:35:33.427 UTC 06a7 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org2.example.com:9151 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.427 UTC 06a8 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org2.example.com:9151 mspid=Org2MSP
2021-10-27 10:35:33.427 UTC 06a9 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org2.example.com:9151 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.427 UTC 06aa INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org2.example.com:9151 mspid=Org2MSP
2021-10-27 10:35:33.427 UTC 06ab ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org2.example.com:9151 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.427 UTC 06ac INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org2.example.com:9151 mspid=Org2MSP
2021-10-27 10:35:33.428 UTC 06ad ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org2.example.com:9151 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.428 UTC 06ae INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org2.example.com:9151 mspid=Org2MSP
2021-10-27 10:35:33.428 UTC 06af ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org2.example.com:9151 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.428 UTC 06b0 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org2.example.com:9151 mspid=Org2MSP
2021-10-27 10:35:33.428 UTC 06b1 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org2.example.com:9151 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.428 UTC 06b2 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org2.example.com:9151 mspid=Org2MSP
2021-10-27 10:35:33.428 UTC 06b3 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org2.example.com:9151 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.428 UTC 06b4 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org2.example.com:9051 mspid=Org2MSP
2021-10-27 10:35:33.428 UTC 06b5 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org2.example.com:9051 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.428 UTC 06b6 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org2.example.com:9151 mspid=Org2MSP
2021-10-27 10:35:33.428 UTC 06b7 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org2.example.com:9151 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.428 UTC 06b8 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org2.example.com:9151 mspid=Org2MSP
2021-10-27 10:35:33.428 UTC 06b9 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org2.example.com:9151 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.429 UTC 06ba INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org2.example.com:9151 mspid=Org2MSP
2021-10-27 10:35:33.429 UTC 06bb ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org2.example.com:9151 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.429 UTC 06bc INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org2.example.com:9151 mspid=Org2MSP
2021-10-27 10:35:33.429 UTC 06bd ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org2.example.com:9151 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.429 UTC 06be INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org2.example.com:9151 mspid=Org2MSP
2021-10-27 10:35:33.429 UTC 06bf ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org2.example.com:9151 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.429 UTC 06c0 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org2.example.com:9051 mspid=Org2MSP
2021-10-27 10:35:33.429 UTC 06c1 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org2.example.com:9051 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.429 UTC 06c2 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org2.example.com:9151 mspid=Org2MSP
2021-10-27 10:35:33.429 UTC 06c3 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org2.example.com:9151 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.429 UTC 06c4 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org3.example.com:11151 mspid=Org3MSP
2021-10-27 10:35:33.429 UTC 06c5 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org3.example.com:11151 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.429 UTC 06c6 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org3.example.com:11151 mspid=Org3MSP
2021-10-27 10:35:33.429 UTC 06c7 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org3.example.com:11151 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.429 UTC 06c8 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org2.example.com:9051 mspid=Org2MSP
2021-10-27 10:35:33.429 UTC 06c9 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org2.example.com:9051 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.429 UTC 06ca INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org2.example.com:9051 mspid=Org2MSP
2021-10-27 10:35:33.429 UTC 06cb ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org2.example.com:9051 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.429 UTC 06cc INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org2.example.com:9051 mspid=Org2MSP
2021-10-27 10:35:33.429 UTC 06cd ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org2.example.com:9051 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.429 UTC 06ce INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org2.example.com:9051 mspid=Org2MSP
2021-10-27 10:35:33.429 UTC 06cf ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org2.example.com:9051 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.429 UTC 06d0 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org2.example.com:9051 mspid=Org2MSP
2021-10-27 10:35:33.429 UTC 06d1 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org2.example.com:9051 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.429 UTC 06d2 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org2.example.com:9051 mspid=Org2MSP
2021-10-27 10:35:33.429 UTC 06d3 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org2.example.com:9051 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.429 UTC 06d4 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org2.example.com:9051 mspid=Org2MSP
2021-10-27 10:35:33.429 UTC 06d5 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org2.example.com:9051 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.430 UTC 06d6 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org2.example.com:9051 mspid=Org2MSP
2021-10-27 10:35:33.430 UTC 06d7 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org2.example.com:9051 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.430 UTC 06d8 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org2.example.com:9051 mspid=Org2MSP
2021-10-27 10:35:33.430 UTC 06d9 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org2.example.com:9051 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.430 UTC 06da INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org2.example.com:9051 mspid=Org2MSP
2021-10-27 10:35:33.430 UTC 06db ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org2.example.com:9051 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.430 UTC 06dc INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org2.example.com:9051 mspid=Org2MSP
2021-10-27 10:35:33.430 UTC 06dd ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org2.example.com:9051 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.430 UTC 06de INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org2.example.com:9051 mspid=Org2MSP
2021-10-27 10:35:33.430 UTC 06df ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org2.example.com:9051 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.430 UTC 06e0 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org3.example.com:11151 mspid=Org3MSP
2021-10-27 10:35:33.430 UTC 06e1 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org3.example.com:11151 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.430 UTC 06e2 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org3.example.com:11051 mspid=Org3MSP
2021-10-27 10:35:33.430 UTC 06e3 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org3.example.com:11051 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.430 UTC 06e4 INFO [comm.grpc.server] 1 -> unary call completed grpc.service=gateway.Gateway grpc.method=Endorse grpc.peer_address=172.20.0.1:39872 error="rpc error: code = Aborted desc = failed to endorse transaction, see attached details for more info" grpc.code=Aborted grpc.call_duration=2.4012251s
2021-10-27 10:35:33.430 UTC 06e5 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org2.example.com:9051 mspid=Org2MSP
2021-10-27 10:35:33.430 UTC 06e6 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org2.example.com:9051 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.430 UTC 06e7 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org3.example.com:11151 mspid=Org3MSP
2021-10-27 10:35:33.430 UTC 06e8 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org3.example.com:11151 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.431 UTC 06e9 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org3.example.com:11051 mspid=Org3MSP
2021-10-27 10:35:33.431 UTC 06ea ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org3.example.com:11051 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.431 UTC 06eb INFO [comm.grpc.server] 1 -> unary call completed grpc.service=gateway.Gateway grpc.method=Endorse grpc.peer_address=172.20.0.1:39872 error="rpc error: code = Aborted desc = failed to endorse transaction, see attached details for more info" grpc.code=Aborted grpc.call_duration=11.6965765s
2021-10-27 10:35:33.431 UTC 06ec INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org2.example.com:9051 mspid=Org2MSP
2021-10-27 10:35:33.431 UTC 06ed ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org2.example.com:9051 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.431 UTC 06ee INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org3.example.com:11151 mspid=Org3MSP
2021-10-27 10:35:33.431 UTC 06ef ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org3.example.com:11151 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.431 UTC 06f0 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org3.example.com:11051 mspid=Org3MSP
2021-10-27 10:35:33.431 UTC 06f1 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org3.example.com:11051 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.431 UTC 06f2 INFO [comm.grpc.server] 1 -> unary call completed grpc.service=gateway.Gateway grpc.method=Endorse grpc.peer_address=172.20.0.1:39872 error="rpc error: code = Aborted desc = failed to endorse transaction, see attached details for more info" grpc.code=Aborted grpc.call_duration=2.4020977s
2021-10-27 10:35:33.431 UTC 06f3 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org2.example.com:9051 mspid=Org2MSP
2021-10-27 10:35:33.431 UTC 06f4 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org2.example.com:9051 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.431 UTC 06f5 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org3.example.com:11151 mspid=Org3MSP
2021-10-27 10:35:33.431 UTC 06f6 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org3.example.com:11151 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.431 UTC 06f7 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org3.example.com:11051 mspid=Org3MSP
2021-10-27 10:35:33.431 UTC 06f8 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org3.example.com:11051 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.431 UTC 06f9 INFO [comm.grpc.server] 1 -> unary call completed grpc.service=gateway.Gateway grpc.method=Endorse grpc.peer_address=172.20.0.1:39872 error="rpc error: code = Aborted desc = failed to endorse transaction, see attached details for more info" grpc.code=Aborted grpc.call_duration=7.504642s
2021-10-27 10:35:33.431 UTC 06fa INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org2.example.com:9151 mspid=Org2MSP
2021-10-27 10:35:33.431 UTC 06fb ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org2.example.com:9151 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.432 UTC 06fc INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org3.example.com:11151 mspid=Org3MSP
2021-10-27 10:35:33.432 UTC 06fd ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org3.example.com:11151 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.432 UTC 06fe INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org3.example.com:11051 mspid=Org3MSP
2021-10-27 10:35:33.432 UTC 06ff ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org3.example.com:11051 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.432 UTC 0700 INFO [comm.grpc.server] 1 -> unary call completed grpc.service=gateway.Gateway grpc.method=Endorse grpc.peer_address=172.20.0.1:39872 error="rpc error: code = Aborted desc = failed to endorse transaction, see attached details for more info" grpc.code=Aborted grpc.call_duration=2.4009105s
2021-10-27 10:35:33.432 UTC 0701 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org2.example.com:9051 mspid=Org2MSP
2021-10-27 10:35:33.432 UTC 0702 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org2.example.com:9051 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.432 UTC 0703 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org3.example.com:11151 mspid=Org3MSP
2021-10-27 10:35:33.432 UTC 0704 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org3.example.com:11151 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.432 UTC 0705 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org3.example.com:11051 mspid=Org3MSP
2021-10-27 10:35:33.432 UTC 0706 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org3.example.com:11051 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.432 UTC 0707 INFO [comm.grpc.server] 1 -> unary call completed grpc.service=gateway.Gateway grpc.method=Endorse grpc.peer_address=172.20.0.1:39872 error="rpc error: code = Aborted desc = failed to endorse transaction, see attached details for more info" grpc.code=Aborted grpc.call_duration=5.1740335s
2021-10-27 10:35:33.432 UTC 0708 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org2.example.com:9051 mspid=Org2MSP
2021-10-27 10:35:33.432 UTC 0709 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org2.example.com:9051 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.432 UTC 070a INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org3.example.com:11151 mspid=Org3MSP
2021-10-27 10:35:33.433 UTC 070b ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org3.example.com:11151 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.433 UTC 070c INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org3.example.com:11051 mspid=Org3MSP
2021-10-27 10:35:33.433 UTC 070d ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org3.example.com:11051 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.433 UTC 070e INFO [comm.grpc.server] 1 -> unary call completed grpc.service=gateway.Gateway grpc.method=Endorse grpc.peer_address=172.20.0.1:39872 error="rpc error: code = Aborted desc = failed to endorse transaction, see attached details for more info" grpc.code=Aborted grpc.call_duration=11.6989772s
2021-10-27 10:35:33.433 UTC 070f INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org2.example.com:9051 mspid=Org2MSP
2021-10-27 10:35:33.433 UTC 0710 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org2.example.com:9051 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.433 UTC 0711 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org3.example.com:11151 mspid=Org3MSP
2021-10-27 10:35:33.434 UTC 0712 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org3.example.com:11151 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.434 UTC 0713 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org3.example.com:11051 mspid=Org3MSP
2021-10-27 10:35:33.434 UTC 0714 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org3.example.com:11051 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.434 UTC 0715 INFO [comm.grpc.server] 1 -> unary call completed grpc.service=gateway.Gateway grpc.method=Endorse grpc.peer_address=172.20.0.1:39872 error="rpc error: code = Aborted desc = failed to endorse transaction, see attached details for more info" grpc.code=Aborted grpc.call_duration=11.6999118s
2021-10-27 10:35:33.434 UTC 0716 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org3.example.com:11151 mspid=Org3MSP
2021-10-27 10:35:33.434 UTC 0717 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org3.example.com:11151 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.434 UTC 0718 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org3.example.com:11051 mspid=Org3MSP
2021-10-27 10:35:33.434 UTC 0719 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org3.example.com:11051 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.434 UTC 071a INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org2.example.com:9051 mspid=Org2MSP
2021-10-27 10:35:33.434 UTC 071b ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org2.example.com:9051 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.434 UTC 071c INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org3.example.com:11151 mspid=Org3MSP
2021-10-27 10:35:33.435 UTC 071d INFO [comm.grpc.server] 1 -> unary call completed grpc.service=gateway.Gateway grpc.method=Endorse grpc.peer_address=172.20.0.1:39872 error="rpc error: code = Aborted desc = failed to endorse transaction, see attached details for more info" grpc.code=Aborted grpc.call_duration=11.7006881s
2021-10-27 10:35:33.435 UTC 071e ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org3.example.com:11151 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.435 UTC 071f INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org3.example.com:11051 mspid=Org3MSP
2021-10-27 10:35:33.435 UTC 0720 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org3.example.com:11051 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.435 UTC 0721 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org2.example.com:9051 mspid=Org2MSP
2021-10-27 10:35:33.435 UTC 0722 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org2.example.com:9051 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.435 UTC 0723 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org3.example.com:11151 mspid=Org3MSP
2021-10-27 10:35:33.435 UTC 0724 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org3.example.com:11151 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.435 UTC 0725 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org2.example.com:9051 mspid=Org2MSP
2021-10-27 10:35:33.435 UTC 0726 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org2.example.com:9051 mspid=Org2MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.435 UTC 0727 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org3.example.com:11051 mspid=Org3MSP
2021-10-27 10:35:33.435 UTC 0728 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org3.example.com:11051 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.435 UTC 0729 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org3.example.com:11051 mspid=Org3MSP
2021-10-27 10:35:33.435 UTC 072a ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org3.example.com:11051 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.435 UTC 072b INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org3.example.com:11151 mspid=Org3MSP
2021-10-27 10:35:33.435 UTC 072c ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org3.example.com:11151 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.435 UTC 072d INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org3.example.com:11151 mspid=Org3MSP
2021-10-27 10:35:33.435 UTC 072e ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org3.example.com:11151 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.435 UTC 072f INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org3.example.com:11051 mspid=Org3MSP
2021-10-27 10:35:33.435 UTC 0730 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org3.example.com:11051 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.435 UTC 0731 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org3.example.com:11151 mspid=Org3MSP
2021-10-27 10:35:33.436 UTC 0732 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org3.example.com:11151 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.436 UTC 0733 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org3.example.com:11151 mspid=Org3MSP
2021-10-27 10:35:33.436 UTC 0734 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org3.example.com:11151 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.436 UTC 0735 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org3.example.com:11151 mspid=Org3MSP
2021-10-27 10:35:33.436 UTC 0736 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org3.example.com:11151 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.436 UTC 0737 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org3.example.com:11151 mspid=Org3MSP
2021-10-27 10:35:33.436 UTC 0738 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org3.example.com:11151 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.436 UTC 0739 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org3.example.com:11051 mspid=Org3MSP
2021-10-27 10:35:33.436 UTC 073a ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org3.example.com:11051 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.436 UTC 073b INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org3.example.com:11151 mspid=Org3MSP
2021-10-27 10:35:33.436 UTC 073c ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org3.example.com:11151 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.436 UTC 073d INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org3.example.com:11051 mspid=Org3MSP
2021-10-27 10:35:33.436 UTC 073e ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org3.example.com:11051 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.436 UTC 073f INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org3.example.com:11151 mspid=Org3MSP
2021-10-27 10:35:33.436 UTC 0740 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org3.example.com:11151 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.436 UTC 0741 INFO [comm.grpc.server] 1 -> unary call completed grpc.service=gateway.Gateway grpc.method=Endorse grpc.peer_address=172.20.0.1:39872 error="rpc error: code = Aborted desc = failed to endorse transaction, see attached details for more info" grpc.code=Aborted grpc.call_duration=2.407182s
2021-10-27 10:35:33.436 UTC 0742 INFO [comm.grpc.server] 1 -> unary call completed grpc.service=gateway.Gateway grpc.method=Endorse grpc.peer_address=172.20.0.1:39872 error="rpc error: code = Aborted desc = failed to endorse transaction, see attached details for more info" grpc.code=Aborted grpc.call_duration=11.7022697s
2021-10-27 10:35:33.436 UTC 0743 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org3.example.com:11151 mspid=Org3MSP
2021-10-27 10:35:33.436 UTC 0744 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org3.example.com:11151 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.436 UTC 0745 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org3.example.com:11051 mspid=Org3MSP
2021-10-27 10:35:33.437 UTC 0746 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org3.example.com:11051 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.437 UTC 0747 INFO [comm.grpc.server] 1 -> unary call completed grpc.service=gateway.Gateway grpc.method=Endorse grpc.peer_address=172.20.0.1:39872 error="rpc error: code = Aborted desc = failed to endorse transaction, see attached details for more info" grpc.code=Aborted grpc.call_duration=11.702624s
2021-10-27 10:35:33.437 UTC 0748 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org3.example.com:11051 mspid=Org3MSP
2021-10-27 10:35:33.437 UTC 0749 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org3.example.com:11051 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.437 UTC 074a INFO [comm.grpc.server] 1 -> unary call completed grpc.service=gateway.Gateway grpc.method=Endorse grpc.peer_address=172.20.0.1:39872 error="rpc error: code = Aborted desc = failed to endorse transaction, see attached details for more info" grpc.code=Aborted grpc.call_duration=7.5099229s
2021-10-27 10:35:33.437 UTC 074b INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org3.example.com:11151 mspid=Org3MSP
2021-10-27 10:35:33.437 UTC 074c ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org3.example.com:11151 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.437 UTC 074d INFO [comm.grpc.server] 1 -> unary call completed grpc.service=gateway.Gateway grpc.method=Endorse grpc.peer_address=172.20.0.1:39872 error="rpc error: code = Aborted desc = failed to endorse transaction, see attached details for more info" grpc.code=Aborted grpc.call_duration=2.4072448s
2021-10-27 10:35:33.437 UTC 074e INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org3.example.com:11051 mspid=Org3MSP
2021-10-27 10:35:33.437 UTC 074f ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org3.example.com:11051 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.437 UTC 0750 INFO [comm.grpc.server] 1 -> unary call completed grpc.service=gateway.Gateway grpc.method=Endorse grpc.peer_address=172.20.0.1:39872 error="rpc error: code = Aborted desc = failed to endorse transaction, see attached details for more info" grpc.code=Aborted grpc.call_duration=11.7034295s
2021-10-27 10:35:33.437 UTC 0751 INFO [comm.grpc.server] 1 -> unary call completed grpc.service=gateway.Gateway grpc.method=Endorse grpc.peer_address=172.20.0.1:39872 error="rpc error: code = Aborted desc = failed to endorse transaction, see attached details for more info" grpc.code=Aborted grpc.call_duration=2.4072529s
2021-10-27 10:35:33.437 UTC 0752 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org3.example.com:11051 mspid=Org3MSP
2021-10-27 10:35:33.437 UTC 0753 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org3.example.com:11051 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.437 UTC 0754 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org3.example.com:11051 mspid=Org3MSP
2021-10-27 10:35:33.437 UTC 0756 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org3.example.com:11051 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.437 UTC 0757 INFO [comm.grpc.server] 1 -> unary call completed grpc.service=gateway.Gateway grpc.method=Endorse grpc.peer_address=172.20.0.1:39872 error="rpc error: code = Aborted desc = failed to endorse transaction, see attached details for more info" grpc.code=Aborted grpc.call_duration=11.7035735s
2021-10-27 10:35:33.437 UTC 0755 INFO [comm.grpc.server] 1 -> unary call completed grpc.service=gateway.Gateway grpc.method=Endorse grpc.peer_address=172.20.0.1:39872 error="rpc error: code = Aborted desc = failed to endorse transaction, see attached details for more info" grpc.code=Aborted grpc.call_duration=11.7032447s
2021-10-27 10:35:33.438 UTC 0758 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org3.example.com:11051 mspid=Org3MSP
2021-10-27 10:35:33.438 UTC 0759 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org3.example.com:11051 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.438 UTC 075a INFO [comm.grpc.server] 1 -> unary call completed grpc.service=gateway.Gateway grpc.method=Endorse grpc.peer_address=172.20.0.1:39872 error="rpc error: code = Aborted desc = failed to endorse transaction, see attached details for more info" grpc.code=Aborted grpc.call_duration=7.5109255s
2021-10-27 10:35:33.438 UTC 075b INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org3.example.com:11051 mspid=Org3MSP
2021-10-27 10:35:33.438 UTC 075c ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org3.example.com:11051 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.438 UTC 075d INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org3.example.com:11051 mspid=Org3MSP
2021-10-27 10:35:33.438 UTC 075e INFO [comm.grpc.server] 1 -> unary call completed grpc.service=gateway.Gateway grpc.method=Endorse grpc.peer_address=172.20.0.1:39872 error="rpc error: code = Aborted desc = failed to endorse transaction, see attached details for more info" grpc.code=Aborted grpc.call_duration=5.1797234s
2021-10-27 10:35:33.438 UTC 075f ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org3.example.com:11051 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.438 UTC 0760 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org3.example.com:11051 mspid=Org3MSP
2021-10-27 10:35:33.438 UTC 0761 INFO [comm.grpc.server] 1 -> unary call completed grpc.service=gateway.Gateway grpc.method=Endorse grpc.peer_address=172.20.0.1:39872 error="rpc error: code = Aborted desc = failed to endorse transaction, see attached details for more info" grpc.code=Aborted grpc.call_duration=7.5109545s
2021-10-27 10:35:33.438 UTC 0762 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org3.example.com:11051 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.438 UTC 0763 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org3.example.com:11151 mspid=Org3MSP
2021-10-27 10:35:33.438 UTC 0764 INFO [comm.grpc.server] 1 -> unary call completed grpc.service=gateway.Gateway grpc.method=Endorse grpc.peer_address=172.20.0.1:39872 error="rpc error: code = Aborted desc = failed to endorse transaction, see attached details for more info" grpc.code=Aborted grpc.call_duration=9.6956889s
2021-10-27 10:35:33.438 UTC 0765 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org3.example.com:11151 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.438 UTC 0767 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer1.org3.example.com:11151 mspid=Org3MSP
2021-10-27 10:35:33.438 UTC 0766 INFO [comm.grpc.server] 1 -> unary call completed grpc.service=gateway.Gateway grpc.method=Endorse grpc.peer_address=172.20.0.1:39872 error="rpc error: code = Aborted desc = failed to endorse transaction, see attached details for more info" grpc.code=Aborted grpc.call_duration=5.179908s
2021-10-27 10:35:33.438 UTC 0768 ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer1.org3.example.com:11151 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.438 UTC 0769 INFO [gateway] removeEndorser -> Closing connection to remote endorser address=peer0.org3.example.com:11051 mspid=Org3MSP
2021-10-27 10:35:33.438 UTC 076a INFO [comm.grpc.server] 1 -> unary call completed grpc.service=gateway.Gateway grpc.method=Endorse grpc.peer_address=172.20.0.1:39872 error="rpc error: code = Aborted desc = failed to endorse transaction, see attached details for more info" grpc.code=Aborted grpc.call_duration=9.6957617s
2021-10-27 10:35:33.438 UTC 076b ERRO [gateway] removeEndorser -> Failed to close connection to endorser address=peer0.org3.example.com:11051 mspid=Org3MSP err="rpc error: code = Canceled desc = grpc: the client connection is closing"
2021-10-27 10:35:33.438 UTC 076c INFO [comm.grpc.server] 1 -> unary call completed grpc.service=gateway.Gateway grpc.method=Endorse grpc.peer_address=172.20.0.1:39872 error="rpc error: code = Aborted desc = failed to endorse transaction, see attached details for more info" grpc.code=Aborted grpc.call_duration=9.6960807s

And then they stop

Basic transaction scenario tests with Node chaincode

As a contributor
I want to run basic transaction scenario tests with Node and Java chaincode
So that I can ensure behaviour is consistent regardless of chaincode implementation language

Handling of transaction error responses appears to be different depending on the chaincode implementation language. Scenario tests are needed to flush out behavioural differences and direct error handling design.

Suggest implementing an equivalent of the tests in scenario/features/transactions.feature but using a Node chaincode equivalent to the Go chaincode in scenario/fixtures/chaincode/golang/basic/.

duplicate error message makes it difficult to determine code location

There are 2 identical error messages in the code making it difficult to work out which code path occurred.

		if firstEndorser == nil || firstResponse == nil {
			return nil, rpcError(codes.Aborted, "failed to endorse transaction, see attached details for more info", errDetails...)
		}

	if endorsements == nil {
		return nil, rpcError(codes.Aborted, "failed to endorse transaction, see attached details for more info", errorDetails...)
	}

in api.go

Would be good to make these distinguishable or have error log messages in the peer so that you have something in the peer logs to indicate the code path

pkcs11 is mandatory for node thin client

package built and reports a problem building the pkcs11 library which is fine as it's an optional dependency, but client applications can't run without it

Error: Cannot find module 'pkcs11js'
Require stack:
- /node_modules/fabric-gateway/dist/identity/hsmsigner.js
- /node_modules/fabric-gateway/dist/identity/signers.js
- /node_modules/fabric-gateway/dist/index.js
- /dist/transaction-driver.js
- /dist/start.js

Doc updates for submit/evaluate

As an application developer
I want examples of how the single call submit/evaluate methods correspond to finer grained API calls
So that I can better understand the transaction flow and relationships between the different API methods

Peer log message a bit misleading about required endorsements

[        org1peer] 2021-09-22 13:21:43.793 UTC 00de INFO [gateway] Endorse -> Endorsement required from:

Looks odd with a blank entry.

From Andy

just looked at the code - yes that message is a bit misleading - it’s supposed to say which extra endorsers are required having already got an endorsement from the local org.  In this case no more are required.  Message could definitely be better!

Tagging of packages published to npm

There is currently a very old fabric-gateway package tagged as latest in the npm registry. All subsequent packages have been published tagged as unstable. Publishing development builds tagged as unstable is fine, but this does mean that, right now, anyone installing fabric-gateway with npm install fabric-gateway will get a very old version. This is not what we want.

Suggest that we:

Publish all development builds as latest until we get to a first release (or pre-release) version that should stay as the latest version; then
Switch back to publishing development builds as unstable, with only release builds then being tagged as latest.

[Java SDK] Re-endorsing an existing proposal fails but with no indication of why

Java GatewaySDK, calling proposal.endorse() on an existing proposal fails with

 io.grpc.StatusRuntimeException: INTERNAL: RST_STREAM closed stream. HTTP/2 error code: INTERNAL_ERROR

a) took a while to find as this is an unchecked exception so it got 'lost'
b) there's no linked exception or any other indication of what has failed. The peer log gives more insight

failed to endorse transaction:  [xxxxx]

A checked (for preference) Exception with the peer error would be more useful.

Something for consideration under #178

Chaincode event example in Go API documentation

The Go API documentation for the Network struct should include an example of chaincode event listening.

Document supported versions

As an application developer
I want to have documentation of supported versions
So that I know which runtimes and platforms I can use to develop and deploy Fabric Gateway client applications

This should include at least:

Language runtime versions (such as Golang, Node.js and Java versions)
Platforms (such as Ubuntu 20.04)
Fabric server versions

When gateway peer starts or restarts it could be unusable

As of driver downloaded today 22nd Oct 2021 (built 3 days ago)
hyperledger-fabric.jfrog.io/fabric-peer cd4ffa294471 3 days ago 54.7MB

While testing bringing the gateway peer down and up I've found it to be unreliable when it comes back up, ie unable to submit transactions with 1 of 2 errors

14 UNAVAILABLE: failed to select a set of endorsers that satisfy the endorsement policy

and

14 UNAVAILABLE: no combination of peers can be derived which satisfy the endorsement policy: no peer combination can satisfy the endorsement policy

The gateway peer remains in this state (stopping and starting the client makes no difference). It requires another restart
One thing that appears to affect recovery is how much pressure the client is putting on the gateway peer during the time it comes up. If I reduce the pressure the gateway peer seems to restart more reliably .

Async submit example in Go API documentation

The Go API documentation for the Network struct should include a code example of chaincode event listening.

Expectations if a gateway peer is restarted for submit transactions

A client does not have to recreate a grpc connection or gateway to cater for a gateway peer terminating and restarting (note this does not take into account talking to a peer through an ALB which may create a different connectivity behaviour), nor should a well designed client applicaton require restarting however the client application should expect certain errors when the peer goes down.
In our testing we see

"Failed","message":"14 UNAVAILABLE: Connection dropped"

and

"Failed","message":"1 CANCELLED: Call cancelled"}

occuring when the gateway peer terminates

When the gateway peer is restarted, even though it will accept requests to endorse you could see the following error

"14 UNAVAILABLE: no combination of peers can be derived which satisfy the endorsement policy: no peer combination can satisfy the endorsement policy"

These stop after about 1 second in our tests as the gateway peer rediscovers the available peers. This implies that a gateway peer isn't instantly available to process requests even though it accepts requests (potential for enhancement ????)
This doesn't happen all the time due to delays in the client polling for a ready connection and possibly how fast the peer creates it's knowledge of endorsing peers

We could recommend a client make use of the waitForReady api in their grpc connection. This provides a simple api to determine if there is an actual connection to the gateway peer and can be used in a polling fashion to wait until the gateway peer has started and the connection is restored. This of course is no indication that the peer is ready to accept requests and if you don't do this then you will get an error trying to submit the request (as expected as the peer is down). In our testing this proved to be a nice way to wait before sending submit requests but of course doesn't help with the gateway peer not being ready to process the request due to discovering the network

Idemix signer for Gateway SDK Go client

As a Golang application developer
I want to use an Idemix signer implementation
So that client permission to transact with the network can be verified while maintaining anonymity

Golang Idemix implementation supported by current Fabric nodes here:

https://github.com/IBM/idemix

See this comment for pointers on integration:

hyperledger/fabric#2955 (comment)

[Java SDK] What is a new proposal?

To create a proposal in Java

 Proposal.Builder pb = contract.newProposal("create").addArguments(json);
 Proposal propAlpha = pb.build()
 Proposal propBeta = pb.build()

Though they are different objects, propAlpha, and propBeta are the same logical proposal (same tx id).

Is this correct? (either way - would be worth documenting this in the API docs)

Support for Node 16 LTS

Target LTS start date for Node 16 is 2021-10-26:

https://nodejs.org/en/about/releases/

Add first LTS version to engines property in package.json
Add test runs using Node 16 to build
Update supported versions documentation

Create a Commit object for a given transaction ID

As an application developer
I want to create a Commit object directly from a specified transaction ID
So that I can obtain the validation code for an arbitrary transaction

Currently Commit objects are only obtained by submitting a transaction. It is convenient to be able to check the commit status of a given transaction, which may not have been previously submitted by the current client application process.

Suggested implementation is to provide a newCommit() method on Network that takes a transaction ID as a parameter. Gateway already has a newSignedCommit() method for use in off-line signing, and this would mirror the pattern for Proposal, which has both newProposal on Contract and newSignedProposal on Gateway.

Tasks:

Go implementation of NewCommit() receiver function for Network, with unit tests and Godoc.
Node implementation of newCommit() method on Network, with unit tests and Typedoc.
Java implementation of newCommit() method on Network, with unit tests and Javadoc.

Release process

A release process (ideally automated in the CI pipeline) needs to be implemented and documented for all of the client language APIs. This includes:

Selecting specific builds (or Git commits) as releases and specifying their release version.
Publishing of release builds (to NPM registry, Maven Central etc.)
Creating GitHub releases and associated Git tags.

Gateway Peer appears to not establish connections to some peers

I have a scenario test where I

bring down all the non-gateway peers
sleep for 20 seconds
bring up all the non-gateway peers
sleep for 10 seconds
bring down the gateway peer
sleep for 10 seconds
bring up the gateway peer

after this I cannot use the gateway peer because the client reports

14 UNAVAILABLE: failed to select a set of endorsers that satisfy the endorsement policy

and in the gateway peer logs you see lots of

2021-10-13 07:44:12.532 UTC 2710 WARN [gateway] endorsers -> Failed to find endorser at peer1.org1.example.com:8151
2021-10-13 07:44:12.532 UTC 2711 WARN [gateway] endorsers -> Failed to find endorser at peer0.org2.example.com:9051
2021-10-13 07:44:12.532 UTC 2712 WARN [gateway] endorsers -> Failed to find endorser at peer1.org2.example.com:9151
2021-10-13 07:44:12.532 UTC 2713 WARN [gateway] endorsers -> Failed to find endorser at peer1.org1.example.com:8151
2021-10-13 07:44:12.532 UTC 2714 WARN [gateway] endorsers -> Failed to find endorser at peer1.org3.example.com:11151
2021-10-13 07:44:12.532 UTC 2715 WARN [gateway] endorsers -> Failed to find endorser at peer0.org3.example.com:11051
2021-10-13 07:44:12.532 UTC 2716 WARN [gateway] endorsers -> Failed to find endorser at peer1.org3.example.com:11151
2021-10-13 07:44:12.532 UTC 2717 WARN [gateway] endorsers -> Failed to find endorser at peer0.org3.example.com:11051
2021-10-13 07:44:12.532 UTC 2718 INFO [comm.grpc.server] 1 -> unary call completed grpc.service=gateway.Gateway grpc.method=Endorse grpc.peer_address=192.168.16.1:43500 error="rpc error: code = Unavailable desc = failed to select a set of endorsers that satisfy the endorsement policy" grpc.code=Unavailable grpc.call_duration=1.2687763s

The timing of this has influence. If I sleep longer between bringing the non-gateway peers up and bringing the gateway peer down (say 30 seconds) then it all seems to be ok
The peers are back up and ok, but the gateway doesn't think so. restarting the gateway peer solves the problem. This is an extreme example, but it seems that if the gateway peer is restarted it can only interact with peers it thought were available at time of starting (note that my test client is constantly trying to send submit requests).

The peer image was pulled on 12/10/2021 and node client version is [email protected]

contract evaluateTransaction doesn't throw an error if the chaincode responds with an error

Using node.js chaincode and node.js thin client I execute a submit transaction where the transaction name doesn't exist and the contract api returns an error informing you of the fact. the submit throws an error and the error provides appropriate detail

Error: 10 ABORTED: failed to endorse transaction: [{address:"peer0.org1.example.com:7051" msp_id:"Org1MSP" message:"error 500, error in simulation: transaction returned with failure: Error: You've asked to invoke a function that does not exist: put" }]

Doing the same using evaluate transaction on a contract results in an empty response, no error is thrown and the contract definitely reports that the function name doesn't exist

2021-09-14 16:04:06.481 UTC 00ec ERRO [endorser] simulateProposal -> failed to invoke chaincode basic, error: transaction returned with failure: Error: You've asked to invoke a function that does not exist: ReadAsset1
github.com/hyperledger/fabric/core/chaincode.processChaincodeExecutionResult
        /go/src/github.com/hyperledger/fabric/core/chaincode/chaincode_support.go:188
github.com/hyperledger/fabric/core/chaincode.(*ChaincodeSupport).Execute
        /go/src/github.com/hyperledger/fabric/core/chaincode/chaincode_support.go:162
github.com/hyperledger/fabric/core/endorser.(*SupportImpl).Execute
        /go/src/github.com/hyperledger/fabric/core/endorser/support.go:126
github.com/hyperledger/fabric/core/endorser.(*Endorser).callChaincode
        /go/src/github.com/hyperledger/fabric/core/endorser/endorser.go:119
github.com/hyperledger/fabric/core/endorser.(*Endorser).simulateProposal
        /go/src/github.com/hyperledger/fabric/core/endorser/endorser.go:186
github.com/hyperledger/fabric/core/endorser.(*Endorser).ProcessProposalSuccessfullyOrError
        /go/src/github.com/hyperledger/fabric/core/endorser/endorser.go:402
github.com/hyperledger/fabric/core/endorser.(*Endorser).ProcessProposal
        /go/src/github.com/hyperledger/fabric/core/endorser/endorser.go:344
github.com/hyperledger/fabric/internal/pkg/gateway.(*EndorserServerAdapter).ProcessProposal
        /go/src/github.com/hyperledger/fabric/internal/pkg/gateway/gateway.go:38
github.com/hyperledger/fabric/internal/pkg/gateway.(*Server).Evaluate
        /go/src/github.com/hyperledger/fabric/internal/pkg/gateway/api.go:66
github.com/hyperledger/fabric-protos-go/gateway._Gateway_Evaluate_Handler.func1
        /go/src/github.com/hyperledger/fabric/vendor/github.com/hyperledger/fabric-protos-go/gateway/gateway.pb.go:1185
github.com/hyperledger/fabric/internal/peer/node.unaryGrpcLimiter.func1
        /go/src/github.com/hyperledger/fabric/internal/peer/node/grpc_limiters.go:44
github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1
        /go/src/github.com/hyperledger/fabric/vendor/github.com/grpc-ecosystem/go-grpc-middleware/chain.go:25
github.com/hyperledger/fabric/common/grpclogging.UnaryServerInterceptor.func1
        /go/src/github.com/hyperledger/fabric/common/grpclogging/server.go:92
github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1
        /go/src/github.com/hyperledger/fabric/vendor/github.com/grpc-ecosystem/go-grpc-middleware/chain.go:25
github.com/hyperledger/fabric/common/grpcmetrics.UnaryServerInterceptor.func1
        /go/src/github.com/hyperledger/fabric/common/grpcmetrics/interceptor.go:31
github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1
        /go/src/github.com/hyperledger/fabric/vendor/github.com/grpc-ecosystem/go-grpc-middleware/chain.go:25
github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1
        /go/src/github.com/hyperledger/fabric/vendor/github.com/grpc-ecosystem/go-grpc-middleware/chain.go:34
github.com/hyperledger/fabric-protos-go/gateway._Gateway_Evaluate_Handler
        /go/src/github.com/hyperledger/fabric/vendor/github.com/hyperledger/fabric-protos-go/gateway/gateway.pb.go:1187
google.golang.org/grpc.(*Server).processUnaryRPC
        /go/src/github.com/hyperledger/fabric/vendor/google.golang.org/grpc/server.go:1180
google.golang.org/grpc.(*Server).handleStream
        /go/src/github.com/hyperledger/fabric/vendor/google.golang.org/grpc/server.go:1503
google.golang.org/grpc.(*Server).serveStreams.func1.2
        /go/src/github.com/hyperledger/fabric/vendor/google.golang.org/grpc/server.go:843
runtime.goexit
        /usr/local/go/src/runtime/asm_amd64.s:1371 channel=mychannel txID=611d2cc7
2021-09-14 16:04:06.481 UTC 00ed WARN [endorser] ProcessProposal -> Failed to invoke chaincode channel=mychannel chaincode=basic error="error in simulation: transaction returned with failure: Error: You've asked to invoke a function that does not exist: ReadAsset1"
2021-09-14 16:04:06.481 UTC 00ee INFO [comm.grpc.server] 1 -> unary call completed grpc.service=gateway.Gateway grpc.method=Evaluate grpc.peer_address=172.21.0.1:34772 grpc.code=OK grpc.call_duration=5.8841ms

Synchronize access to Go PKCS#11 session

As an application developer
I want the HSM signer implementation to be usable concurrently
So that I can use Gateway instances concurrently regardless of the signing implementation

PKCS#11 session handles must not be used concurrently from multiple threads. The current HSM signer implementation does document this, but the knock-on effect is that the concurrency constraints of a Gateway instance are different depending on whether it is using an HSM or non-HSM signing implementation. This places an unnecessary burden on the application developer and increases the likelihood of concurrency bugs in client applications.

Synchronizing access to the session within the HSM signer avoids the possibility of concurrency bugs in client applications. There may still be performance impacts on concurrent use of a Gateway instance to submit transactions using an HSM signer without a more advanced session pooling implementation, but behaviour will be functionally correct.

No automatic recovery for chaincode events when the gateway peer restarts

A client application performing submits and evaluates will automatically recover (ie the client doesn't have to do anything special such as recreating connections/gateways/getting networks/getting contracts). The client can continue to retry submission and evaluation until the gateway peer comes back up and then they will be processed as normal.

This is not the case for listening for chaincode events. If the peer gateway goes down then the event sink becomes unusable and must be closed. This pattern is reflected in the sample code

const events = await network.getChaincodeEvents(chaincodeName, { startBlock: BigInt(101) });
try {
    for async (const event of events) {
        // Process event
    }
} finally {
    events.close();
}

As the reasoning behind this is not explicit this understanding may not be realised so I think this needs to be clarified in the documentation (also note that although for async is the preferred way to handle async iterators, users may still opt to use it a different way).

Automatic recovery should be documented as the client app's responsibility and we should have a sample that demonstrates this (Rest Sample comes to mind)

This issue is only a documentation issue, but maybe a future capability for the gateway might be to have automatic recovery ?

Note this was only tested on node thin client, no testing of java or Go has been done

Use ECMAScript private fields for private methods

As a maintainer
I want to prevent accidental use of internal implementation methods by client applications
So that client applications are not broken if those internals change in the future

Note: this requires at least Node 14 for native support, but the TypeScript compiler generates a polyfill implementation.

The Node client API currently uses ECMAScript private fields for properties but only TypeScript private modifiers on methods. TypeScript visibility modifiers do not change runtime code visibility, and may not help avoid accidental usage by plain JavaScript applications that may not consider the TypeScript visibility modifiers at development time. Using ECMAScript private fields for methods will make them invisible to client application code at both development and runtime.

Define Java interfaces for ChaincodeEvent and Status

As an application developer
I want interface definitions for ChaincodeEvent and Status
So that I can create my own instances for unit testing as the Gateway implementation classes cannot be instantiated

Include display of transaction return values in Node API overview documentation

The code example in the overview of the Node API documentation should include converting the Uint8Array return values into string so they can be output to the console.

import { TextDecoder } from 'util';

const utf8Decoder = new TextDecoder();

console.log('Transaction result:', utf8Decoder.decode(result));

Async submit code example in Node API documentation is out-of-date

Typedoc code example for Contract needs to be updated to reflect current Commit/Status flow.

Update README and Makefile

When I follow the README instruction, and running make build-go, it raised:

go build -o bin/gateway cmd/gateway/*.go
malformed import path "cmd/gateway/*.go": invalid char '*'
make: *** [build-go] Error 1

Then I found that the cmd is not contained in the repo, so is there something wrong?

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.