onyx-protocol / onyx Goto Github PK
View Code? Open in Web Editor NEWOnyx
Home Page: https://Onyx.org
License: GNU Affero General Public License v3.0
Onyx
Home Page: https://Onyx.org
License: GNU Affero General Public License v3.0
Bug does not appear in 68c85f9 (tag sdk.ruby-1.0.1
), but appears in 7579cb1
Queries such as the following used to be valid:
inputs(spent_output.transaction_id='ad929976dbde976d2b74b798632c762fb8146ad0c0d1e0d883487fcee1ebb2c4')
However, I now get the following error message:
Code: CH602 Message: Malformed query filter Detail: invalid attribute: spent_output Request-ID: b2e381c701e04ec73e65
spent_output
is deprecated in 1.1, but should still work until the property is fully removed.
This is a draft of a list of breaking (hard forking) changes for P1 before we fully commit to it long-term.
Each feature has its own PR open which is meant to be merged when fully implemented and covered with tests.
Asset ID should not include version tag to allow their reuse in new version of outputs. Initially it was thought as a necessary security feature for non-upgraded nodes, but balancing rules are enough to force assets to balance in such case. And removing version tag allows nice reuse of already deployed asset ids in the new versions of outputs.
PR: #261
They are unnecessary, outdated, and (in the case of SHA1, anyway) weak.
PR: #268
SHA3(txid || index || SHA3(output))
- constant size, commits to the output contents directly for HSM-friendliness.
PR: #417
Optional Hash is a premature optimization that mixes two concerns: minimizes amount of data to transmit when the hashed string is actually empty, and minimizes amount of data to be hashed. The first one belongs to the transport protocol that can compress common patterns (we have an assumption that such protocol will exist already for other pieces). The second concern (hashing a bit more data) seems to not a serious overhead VS added complexity to implementation.
This is going to be a side effect of the new txgraph redesign. See below.
Replace two lists of inputs/outputs with a single flat list of entries. Entries can be issuances, inputs, outputs or retirements. We can remove OP_FAIL
opcode. We can have multiple "reference data" entries, so each party in the transaction can attach that data. Arbitrary accounting rules and behaviours can be added with future entry types.
PR: #264
We need to carefully review implementation of extensibility rules.
Change validate+validate-well-formedness+apply to validate+apply procedures for txs. Same for block.
It is extra-hard to combine programs in multisig manner which is necessary for consensus programs. Since block VM version is hard to change, we should have a workable THRESHOLD instruction to make it possible to combine individual programs. This will not eliminate, but augment CHECKMULTISIG.
For instance, it could be a TAKE instruction, which takes m
specified unique indexes from a set of items: index[m-1] ... index[0] items[n-1] ... items[0] m n TAKE
-> items[index[m-1]] ... items[index[0]]
. This is the start of a more flexible approach to CHECKMULTISIG and other threshold-type operations.
PR: TBD.
Program can be tested to be well-formed and in-bounds offline, so the network does not need to impose extra internal runtime limits in addition to the already imposed per-input runlimit.
PR: #262
UPDATE: Ivy will target VM2 anyway, so we can avoid cosmetic changes to VM1.
Simplify VM validation rules by having outer program and OP_CHECKPREDICATE
programs not fail, instead of requiring true
value on stack.
PR: #263
UPDATE: Ivy will target VM2 anyway, so we can avoid cosmetic changes to VM1.
In the transaction format, all integers are varints, but on the stack they should be fixed-length 8-byte signed ints for simplicity of implementation and security of smart contracts: fixed-length ints can be concatenated with other data without extra precautions like length-prefixing or extra hashing.
PR: #267
UPDATE: Ivy will target VM2 anyway, so we can avoid cosmetic changes to VM1.
msg
in CHECKMULTISIG and CHECKSIGSo that signed message is the first argument, rather than in the middle, making them easier to compile to.
PR: Not to be done in P1.
UPDATE: Ivy will target VM2 anyway, so we can avoid cosmetic changes to VM1.
It is extra-hard to make multisig rule out of signature programs (when each signature is actually a signed program). We need to generalize CHECKMULTISIG to support a selection of a subset (m-of-n) of programs to be executed.
For instance, it could be a TAKE instruction, which takes m
specified unique indexes from a set of items: index[m-1] ... index[0] items[n-1] ... items[0] m n TAKE
-> items[index[m-1]] ... items[index[0]]
. This is the start of a more flexible approach to CHECKMULTISIG and other threshold-type operations.
PR: Not to be done in P1.
This is necessary for redundancy/failover/HA.
To achieve a more broad usage of the Java SDK it would be nice to make it Java 7 compliant.
I've downloaded the source code and there are a few steps to convert the SDK to Java 7.
when use prod
tag build corectl, a build problem appered as follow:
# chain/cmd/corectl
cmd/corectl/main.go:42: cannot use reset (type func(*"database/sql".DB, []string)) as type func(*"chain/database/sql".DB, []string) in field value
and here have two links might be help
https://github.com/chain/chain/blob/main/cmd/corectl/prod.go#L5
https://github.com/chain/chain/blob/main/cmd/corectl/dev.go#L9
Attempt submitting a transaction with a nil
value. If using the Ruby SDK, attempt submitting a tx such as client.transactions.submit(nil)
to replicate
Expected a helpful error to return to point me in the right direction of what was causing it, and expected Chain Core to continue running
This crashed the locally running Chain Core, throwing the following error:
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x14376a]
goroutine 343 [running]:
panic(0x64f240, 0xc42000c0a0)
/usr/local/Cellar/go/1.7.1/libexec/src/runtime/panic.go:500 +0x1a1
chain/core.(*Handler).finalizeTxWait(0xc4201c80b0, 0x153e160, 0xc4202f97a0, 0xc4201ec0c0, 0x0, 0xc420ccb250, 0xc4202d6810)
/Users/chrisgarvin/src/chain/core/transact.go:164 +0x3a
chain/core.(*Handler).submitSingle(0xc4201c80b0, 0x153e160, 0xc4202f97a0, 0xc4201ec0c0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
/Users/chrisgarvin/src/chain/core/transact.go:102 +0x116
chain/core.(*Handler).submit.func1(0xc4201c80b0, 0x153e1a0, 0xc4202d6810, 0xc4202dab00, 0x1, 0x4, 0x0, 0xc420ccb1f0, 0x1, 0x1, ...)
/Users/chrisgarvin/src/chain/core/transact.go:259 +0xec
created by chain/core.(*Handler).submit
/Users/chrisgarvin/src/chain/core/transact.go:267 +0x16a
Create a transaction in the dashboard with some inputs and no outputs, and try to submit. The error you get is CH705: "Unsafe transaction: leaves assets to be taken without requiring payment." But that error is intended for users of the API, where unbalanced transactions might legitimately need to be created for e.g. atomic swaps. The dashboard deals only in balanced transactions, so "unbalanced transaction" would be a more informative and actionable error message.
How to add signers in a running chain and update generator's quorum property ?
corectl config and config-generator cmd just can init the chain,is there any tools can do this?
Ran into this error while testing #486.
72010274Z status=500 chaincode=CH000 path=/build-transaction error="json: cannot unmarshal array into Go value of type map[string]interface {}"
/Users/jackson/src/chain/core/rpc/rpc.go:67 - chain/core/rpc.(*Client).Call
/Users/jackson/src/chain/core/api.go:331 - chain/core.(*Handler).forwardToLeader
/Users/jackson/src/chain/core/transact.go:78 - chain/core.(*Handler).build
/Users/jackson/src/chain/core/api.go:127 - chain/core.(*Handler).(chain/core.build)-fm
/usr/local/go/src/runtime/asm_amd64.s:516 - runtime.call128
app=cored buildtag=dev processID=chain-mba.local-25974-d430288902dc23074f0e reqid=36285cb64c07e3db6e75 at=http.go:52 t=2017-02-08T00:57:40.975637505Z status=500 chaincode=CH000 path=/build-transaction error="json: cannot unmarshal array into Go value of type map[string]interface {}"
The spec includes the hash of the transaction common witness fields when computing the transaction witness hash, but the implementation does not.
(The fact that the common witness fields are presently empty doesn't mean that we can omit the empty-string's hash from the computation.)
Output commitments are extensible for all asset versions. It's only tx v1 that disallows use of extra space, but tx v2 might allow that even for existing asset version. Therefore the following code looks incorrect. When we read AV1 output, we should remember the suffix data, and enforce it being empty string only if tx version == 1, but otherwise we need to keep it around in the data structure. Same for AV2 etc: need to read full data and keep it around properly.
Immutable snippet: https://github.com/chain/chain/blob/359827001fc222136648cd34158f80cbbd19c03c/protocol/bc/txoutput.go#L119-L123
Snippet on main: https://github.com/chain/chain/blob/main/protocol/bc/txoutput.go#L119-L123
Works for non-empty asset definitions though, as reported by @chrisgarvin. Will need to prepare a unit test to see exact issue.
In the web console each transaction has several detail information (ID, TIMESTAMP, BLOCK ID, BLOCK HEIGHT, POSITION, LOCAL, REFERENCE DATA), it would be nice to have some of this details back in the com.chain.api.Transaction.SubmitResponse object.
@bobg, I'm looking at this rebase conflict where I added serialization flags and you extracted WriteExtensibleString
helper. The problem is that WriteExtensibleString
expects a func that takes only a writer. Adding extra params such as serialization flags feels imprudent.
This makes me think we should have blockchain.Writer
interface that implements io.Writer
and has serializationFlags()
method so we can access them when writing txs and blocks? Not sure we need TxWriter and BlockWriter, but we can add those as "subtypes" of blockchain.Writer with tx- or block-specific functionality if needed later.
WDYT? /cc @kr
// assumes w has sticky errors
func (t *TxInput) writeTo(w io.Writer, serflags uint8) {
blockchain.WriteVarint63(w, t.AssetVersion) // TODO(bobg): check and return error
<<<<<<< f0c469ed8fa7e0af92dec7441d42dca5e4199826
blockchain.WriteExtensibleString(w, t.WriteInputCommitment)
=======
buf := bufpool.Get()
defer bufpool.Put(buf)
t.WriteInputCommitment(buf, serflags)
blockchain.WriteVarstr31(w, buf.Bytes())
>>>>>>> fixing up outputids and unspentids
blockchain.WriteVarstr31(w, t.ReferenceData)
if serflags&SerWitness != 0 {
blockchain.WriteExtensibleString(w, t.writeInputWitness)
}
}
<<<<<<< f0c469ed8fa7e0af92dec7441d42dca5e4199826
func (t *TxInput) WriteInputCommitment(w io.Writer) error {
=======
func (t *TxInput) WriteInputCommitment(w io.Writer, serflags uint8) {
>>>>>>> fixing up outputids and unspentids
if t.AssetVersion == 1 {
switch inp := t.TypedInput.(type) {
case *IssuanceInput:
_, err := w.Write([]byte{0}) // issuance type
if err != nil {
return err
}
_, err = blockchain.WriteVarstr31(w, inp.Nonce)
if err != nil {
return err
}
assetID := t.AssetID()
_, err = w.Write(assetID[:])
if err != nil {
return err
}
_, err = blockchain.WriteVarint63(w, inp.Amount)
return err
case *SpendInput:
<<<<<<< f0c469ed8fa7e0af92dec7441d42dca5e4199826
_, err := w.Write([]byte{1}) // spend type
if err != nil {
return err
}
_, err = inp.OutputID.WriteTo(w)
if err != nil {
return err
}
err = inp.OutputCommitment.writeTo(w, t.AssetVersion)
return err
=======
w.Write([]byte{1}) // spend type
inp.OutputID.WriteTo(w)
if serflags&SerPrevout != 0 {
inp.OutputCommitment.writeTo(w, t.AssetVersion)
} else {
prevouthash := inp.OutputCommitment.Hash(t.AssetVersion)
w.Write(prevouthash[:])
}
>>>>>>> fixing up outputids and unspentids
}
}
return nil
}
Sending a null or undefined client_token
param to the /create-transaction-feed
endpoint will yield the following:
app=cored ... at=http.go:51 t=2016-11-02T23:07:29.398290512Z
status=500 chaincode=CH000 path=/create-transaction-feed
error="pq: null value in column \"client_token\" violates not-null constraint"
This is the client's problem, not the server's, so CH000
isn't quite right here. The endpoint should either tolerate a blank idempotency token (recommended), or return a 400-class error.
The spec requires transaction signature hashes to be made from, among other things, an output commitment hash. The current implementation hashes an extensible string containing the output commitment, which is wrong - it should be the bare concatenation of output commitment fields that gets hashed.
Please upload the Java SDK to Maven Central. It would make it much easier to integrate the SDK in a project. At the moment it is possible to run the application locally by using a dependency with scope system
. However, this does not include the jar in the build and it is cumbersome to setup a local repository just for the chain SDK.
[Note: this document was originally written a few months ago, and it's slightly out of date in a few places where it talks about "current" behavior.]
Currently the protocol.Chain calls a set of callbacks whenever a block is applied to the chain. Let’s call the block callbacks processors. Currently, committing a block and running the processor happens in lockstep. The next block will be validated once all the processors for the previous block have finished.
There are some problems with the current situation.
It’s possible for a processor to encounter an error. Maybe the database’s disk has filled up, or we have a syntax error in one of the SQL statements. These sorts of errors are different from validation errors. If a block fails validation, we need to reject it and wait for a valid block. The system halts entirely. But once the block is validated, we are obligated to process it. If a processor fails for any reason, it’s our fault, and we need to alert a human and fix the error, so we can resume processing.
Currently, we handle processor failure (when we handle it at all) by crashing the process, meaning we stop everything, even validating new blocks and serving non-blockchain requests such as “list account managers”. This is bad for availability.
While a processor is failing, we can certainly continue to validate new blocks, as long as we keep track of which ones haven’t yet been processed. The system as a whole is already asynchronous, so this won’t change our basic assumptions. It will allow us to provide partial availability (validating blocks, serving requests) even while processors are failing.
This means we have, notionally, two pointers: one pointing to the last block that was committed and another one pointing to the last block that was processed. The processor pointer necessarily trails the commit pointer. They can be equal, but processors always run after a block is committed. (For crash recovery, these pointers also need to be put somewhere persistent, but for our purposes here, that is an implementation detail.)
The processor pointer also serves a second function: it “pins” all subsequent blocks to persistent storage. Processing is the last step in the pipeline for ingesting each block. Once processing has finished, that block can be released—it’s okay to prune.
We can take the idea of partial availability further. We’ve decoupled failure of validation from failure of the processors, and we can also decouple failure of one processor from another. Giving each processor a name and its own pointer lets us continue running most of them even when one has failed. This implies they will run concurrently.
With multiple processor pointers, it’s safe to prune the min.
The core leader is responsible for launching processors but shouldn't have sole responsibility for executing them. If other servers in the same core have cycles to spare, the leader should be able to farm out the execution of a processor to another server.
Imagine a Chain Core with all optional features turned off: no account manager, no issuer, no explorer, no long-term block archiving. All it does is ingest blocks and validate them. It’s been running this way for months. Now let’s say the operator wants to turn on all services and optional features. In theory, is this easy? Is it even possible?
For some services, no problem. Creating a new account manager is easy. Generate an xpriv and begin indexing UTXOs for known public key hashes. By construction, the set of UTXOs held by the new account manager is empty at the time it’s created.
For other services, impossible. Indexing historical UTXOs requires knowing the history. In this example, the history has been thrown away.
We don’t have a coherent way to handle these transitions, or to express which ones are safe and which aren’t.
The Chain Core shouldn’t use environment variables to control which features and services are enabled. Instead, it should store the configuration in database tables and mediate access with logic that can prevent unsafe configurations and make safe transitions more convenient.
If block archiving is enabled, then essentially any service can safely be enabled (even though it might take a long time to catch up). If block archiving is disabled, it can in principle be enabled if at least one other node on the network has a complete archive. Enabling block archiving will cause the system to begin fetching all blocks from other nodes, and then other features can be enabled.
There are even some features that don’t need to start from the very beginning, but do need at least a complete copy of the current state. Historical UTXOs is one such feature. We can support this style of transition using the same principles.
Finally, considering all these things, I think callbacks are inside out. It might be better instead to have processors request blocks from the protocol.Chain and explicitly update pointers with GetBlock(height)
, and Pin(pointerName, height)
and Release(pointerName, height)
calls.
Then, we have a somewhat more straightforward path to enable a feature F that requires indexing the entire blockchain. The first step is to pin block 1 (which guarantees that all subsequent blocks are stored locally). If this operation fails, it’s not safe to enable F, and block archiving needs to be turned on first. Once that’s done, F can proceed at its own pace, fetching blocks and processing them until it’s all caught up. (It’s even safe to turn block archiving off again, any time after F has been enabled, because it’s pinned the necessary blocks. In that case, the blocks will be automatically discarded as they’re no longer needed.)
A similar process applies if the feature can start from a more recent state snapshot.
#426 removed some (misplaced) checks on various commitment and witness structures to ensure that they contain no unrecognized additional data (for known data versions - unrecognized data is fine in unknown versions). Those checks need to be added back in a few places in validation.
In a core with > 100 tokens, the next
key in the response from /list-access-tokens
for the first page of results is:
next: {page_size: 0, timeout: 0, after: "", type: "client"}
after: ""
cannot be used to build a query for the next page.
I noticed there's a consensus-critical dependency on stability of JSON encoding function serializeAssetDef
: https://github.com/chain/chain/blob/main/core/asset/asset.go#L325-L333
The DB stores definition as a map and when one issues, we rely on this function to always produce the same JSON encoding that matches the original one when the asset ID was defined for the first time. Till the end of times.
I don't think it's the source of the bug that Chris experienced (see #382), but it will bite us in the future when Go 2.3 will introduce a slightly different JSON encoding and half the network produces different asset id.
We should store the binary string representing asset def and decode from JSON when needed, not the other way around.
There's a race condition when a cored is leader, gets demoted and becomes leader again. leader.Run
will spawn several goroutines when cored becomes leader. These goroutines are expected to exit whenever the context is cancelled. If for whatever reason they don't exit immediately (maybe they're stuck in a slow DB call), it's possible for the old leadership and the new leadership goroutines to be running concurrently.
For most of the leadership goroutines, that's okay, but some rely on the fact that only one leadership goroutine should be running when they access data structures. For ex, core/generator.Generator.Generate
will read/write the latestBlock
and latestSnapshot
fields without using a mutex. This is okay as long as there's only one Generator goroutine running at a time.
We can either:
sync.WaitGroup
and wait for all of the leadership goroutines to recognize that they've been cancelled and exit before moving on and trying to become leader againWe probably should allow block signers to commit UTXO state every some number of blocks. This should give some flexibility in terms of optimizing recalculation process. The nodes would need to keep the diff set of UTXO modifications in-between these commitments.
@kr, I wonder if the patricia trie/set recomputation requires some optimization like that?
Currently, the API returns inputs in a transaction with a control_program field. This has never been a documented property of the API, although it appears in response objects returned by the Node JS SDK (which performs no filtering on the client side), and by the Ruby SDK (a historical mistake; see #542).
Internally, the control_program annotation appears to be important for adding account information from the spent output into the spending input's annotations (see issue 1754 in our internal archival repo), but it's also not something we need or necessarily want to reveal in an API response itself.
It would be simple to just leave this in the API, but I would prefer to either 1) document it, or 2) even better, figure out a way to ensure that it's not included in API responses.
There's a line in the ruby sdk that uses required keyword arguments which appear in ruby 2.1.
We should either specify ruby 2.1 or remove this syntax.
I noticed this error when failing over between two cored processes with high transaction throughput. Shortly after the secondary becomes leader, these nil pointer dereferences happen. A little while later, all build-transaction calls succeed. It doesn't always happen on a failover. It seems like it's probably a race condition surrounding some leader-only state (maybe the state tree?).
I wish we reported a stack trace of where the panic was.
app=cored buildtag=dev processID=chain-mba.local-28703-d06d338b073853927021 reqid=c9adf2bf2996e67cc980 at=http.go:52 t=2017-02-08T01:33:48.007350845Z subreqid=782d7ec96a950d4a654f status=500 chaincode=CH000 path=/build-transaction error="runtime error: invalid memory address or nil pointer dereference"
app=cored buildtag=dev processID=chain-mba.local-28703-d06d338b073853927021 reqid=f47f7464280f03e67a68 at=http.go:52 t=2017-02-08T01:33:48.014702926Z subreqid=3d069c6bd5ea0fdd782e status=500 chaincode=CH000 path=/build-transaction error="runtime error: invalid memory address or nil pointer dereference"
Thank you for reporting an issue! Please read the following before submitting:
Github issues are for reporting specific bugs with Chain Core Developer Edition. If you have general questions about Chain Core or questions regarding building an application on Chain Core, please visit our developer support center:
https://support.chain.com
Enterprise customers can reach out to their support contact directly.
Please check open issues prior to opening a new issue:
https://github.com/chain/chain/issues
Please provide the following information. We will only respond to issues that conform to the following format:
Try to supply the shortest, simplest set of instructions that reliably demonstrates the problem. Complete running programs or screenshots are helpful too.
A recent change in the site seems to have broken the NodeJS snippets. Visit: https://chain.com/docs/core/build-applications/transaction-basics
NodeJS snips
"Snippet "multiasset-within-core" is not in "core/examples/node/transactionBasics.js"
Install latest version of Chain Core for Mac and Run
For the visualization to go away
md2html
fails upon encountering node_modules
directories in the documentation source. I haven't pinned down the precise reason in the tool source code, but this would be a nice thing to clean up eventually.
To repro, you can add a valid node_modules
directory (from npm install
) somewhere in the documentation path; core/examples/node
is a sensible spot, since the examples cannot be run without access to the Chain SDK.
Ideal behavior would be to completely ignore directories named "node_modules" (and maybe other things, a la gitignore)--its contents would be neither parsed nor copied to the destination directory.
I found config-generator cmd can only config signer's pubkey and url,but can not config signer's access token.
This will cause a problem in multi signers environment, the generator will receive 401 response from signer
While running the java integration tests locally, I noticed the log lines:
Running com.chain.integration.FailureTest
app=cored buildtag=dev processID=chain-mba.local-40566-65f0aa67d90cb6706c94 reqid=728c36bc5c57d812ae2f at=http.go:52 t=2017-01-19T00:09:23.310804967Z subreqid=1f2aef6136b76285693e status=400 chaincode=CH202 path=/create-account error="at least one xpub is required"
app=cored buildtag=dev processID=chain-mba.local-40566-65f0aa67d90cb6706c94 reqid=6f677949f17bafae0937 at=http.go:52 t=2017-01-19T00:09:23.314696257Z subreqid=915b5c75c5130998fda5 status=400 chaincode=CH202 path=/create-asset error="at least one xpub is required"
app=cored buildtag=dev processID=chain-mba.local-40566-65f0aa67d90cb6706c94 reqid=051b9969d2a994a8bc0b at=http.go:52 t=2017-01-19T00:09:23.319110197Z subreqid=0889080bd5743baac20b status=400 chaincode=CH003 path=/create-control-program error="unknown control program type \"\": httpjson: bad request"
app=cored buildtag=dev processID=chain-mba.local-40566-65f0aa67d90cb6706c94 reqid=a31e023083b4da3276f7 at=http.go:52 t=2017-01-19T00:09:23.346207285Z subreqid=cd23151fcadb8c0e96bf status=400 chaincode=CH706 path=/build-transaction error="errors occurred in one or more actions"
app=cored buildtag=dev processID=chain-mba.local-40566-65f0aa67d90cb6706c94 reqid=54c533e2ab27e673589f at=http.go:52 t=2017-01-19T00:09:23.372051558Z subreqid=6f15cea3e09d970c7000 status=500 chaincode=CH000 path=/submit-transaction error="runtime error: invalid memory address or nil pointer dereference"
There's a nil pointer dereference at the end of snippet. This test is designed to fail, so the integration test still passes. However, it should probably fail some other way than a nil pointer.
This is a preliminary roadmap of external feature updates for the upcoming 1.1 release.
Chain Core 1.1.x is compatible with client applications written for Chain Core 1.0.x, but you should review the changes in this document and make sure to migrate away from deprecated features, which are likely to be removed in 1.2.x.
Due to changes in the network-level protocol, old blockchain data created on Chain Core 1.0.x is not compatible with Chain Core 1.1.x. When upgrading to 1.1.x, you'll need to re-create your assets and accounts.
Chain 1.1.x introduces the concept of a receiver, a cross-core payment primitive that supersedes the Chain 1.0.x pattern of creating and paying to control programs. Control programs still exist in the Chain protocol, but are no longer used directly to facilitate cross-core payments.
A receiver wraps a control program with other pieces of payment-related metadata, such as expiration dates. Receivers provide the basis for future payment features, such as the transfer of blinding factors for encrypted outputs, as well as off-chain proof of payment via X.509 certificates or some other crypto-based authentication scheme.
Initially, receivers consist of a control program and an expiration date. Transactions that pay to a receiver after the expiration date may not be tracked by Chain Core, and application logic should regard such payments as invalid. As long as both the payer and payee to do not tamper with receiver objects, the Chain Core API will ensure that transactions that pay to expired receivers will fail to validate.
The create-receiver API call supersedes and deprecates the create-control-program API call.
// Java
ControlProgram controlProgram = new ControlProgram.Builder()
.controlWithAccountByAlias("alice")
.create(client);
// Node
controlProgramPromise = client.accounts.createControlProgram({
accountAlias: 'alice'
})
# Ruby
cp = client.accounts.create_control_program(
account_alias: 'alice'
)
Creating control programs in this fashion is deprecated.
You can create receivers with an expiration time, which defaults to 30 days into the future.
// Java
Receiver receiver = new Account.ReceiverBuilder()
.setAccountAlias("alice")
.setExpiresAt("2017-01-01T00:00:00Z")
.create(client);
// Node
receiverPromise = client.accounts.createReceiver({
accountAlias: 'alice',
expiresAt: '2017-01-01T00:00:00Z'
})
# Ruby
receiver = client.accounts.create_receiver(
account_alias: 'alice',
expires_at: '2017-01-01T00:00:00Z'
)
The control-with-receiver transaction builder action supersedes and deprecates the control-with-control-program action.
// Java
Transaction.Template template = new Transaction.Builder()
.addAction(
new Transaction.Action.ControlWithProgram()
.setControlProgram(controlProgram.controlProgram)
.setAssetAlias("gold")
.setAmount(1)
).addAction(
...
).build(client);
// Node
templatePromise = client.transactions.build(builder => {
builder.controlWithProgram({
controlProgram: controlProgram.controlProgram,
assetAlias: 'gold',
amount: 1
})
...
})
// Ruby
template = client.transactions.build do |builder|
builder.control_with_program(
control_program: control_program.control_program,
asset_alias: 'gold',
amount: 1
)
...
end
// Java
Transaction.Template template = new Transaction.Builder()
.addAction(
new Transaction.Action.ControlWithReceiver()
.setReceiver(receiver)
.setAssetAlias("gold")
.setAmount(1)
).addAction(
...
).build(client);
// Node
templatePromise = client.transactions.build(builder => {
builder.controlWithReceiver({
reciever: receiver,
assetAlias: 'gold',
amount: 1
})
...
})
// Ruby
template = client.transactions.build do |builder|
builder.control_with_receiver(
receiver: receiver,
asset_alias: 'gold',
amount: 1
)
...
end
In Chain Core 1.0.x, transaction inputs uses a compound value called spent_output
(consisting of a transaction ID and position) to refer to the output consumed by a particular input. Chain 1.1.x deprecates this scheme in favor of a single value, output_id
.
Transaction output objects and unspent outputs now have an id
property, which is unique for that output across the history of the blockchain.
// Java
Transaction tx;
UnspentOutput utxo;
System.out.println(tx.outputs.get(0).id);
System.out.println(utxo.id);
// Node
console.log(tx.outputs[0].id)
console.log(utxo.id)
// Ruby
puts tx.outputs.first.id
puts utxo.id
Transaction inputs now reference a spent output ID. The spent output property is deprecated.
// Java
Transaction tx;
System.out.println(tx.inputs.get(0).spentOutputId);
// Node
console.log(tx.inputs[0].spentOutputId)
// Ruby
puts tx.inputs.first.spent_output_id
The spend-account-unspent-output transaction builder action now accepts an output ID parameter. It still accepts a pair of transaction ID and position, but this usage pattern is deprecated.
// Java
Transaction.Template template = new Transaction.Builder()
.addAction(
new Transaction.Action.SpendAccountUnspentOutput()
.setTransactionId("abc123")
.setPosition(0)
).addAction(
...
).build(client);
// Node
templatePromise = client.transactions.build(builder => {
builder.spendAccountUnspentOutput({
transactionId: 'abc123',
position: 0
})
...
})
// Ruby
template = client.transactions.build do |builder|
builder.spend_account_unspent_output(
transaction_id: 'abc123',
position: 0
)
...
end
// Java
Transaction.Template template = new Transaction.Builder()
.addAction(
new Transaction.Action.SpendAccountUnspentOutput()
.setOutputId("xyz789")
).addAction(
...
).build(client);
// Node
templatePromise = client.transactions.build(builder => {
builder.spendAccountUnspentOutput({
outputId: 'xyz789'
})
...
})
// Ruby
template = client.transactions.build do |builder|
builder.spend_account_unspent_output(
output_id: 'xyz789'
)
...
end
To retrieve transactions that were partially consumed by a given transaction input, you can query against a specific output ID.
// Java
Transaction.Items results = new Transaction.QueryBuilder()
.setFilter("id=$1")
.setFilterParameter(spendingTx.inputs.get(0).spentOutput.transactionId)
.execute(client);
// Node
client.transactions.queryAll({
filter: 'id=$1',
filterParameters: [spendingTx.inputs[0].spentOutput.transactionId]
}, (tx, next, done, fail) => {
...
})
// Ruby
client.transactions.query(
filter: 'id=$1',
filter_parameters: [spending_tx.inputs.first.spent_output.transaction_id]
) do |tx|
...
end
// Java
Transaction.Items results = new Transaction.QueryBuilder()
.setFilter("outputs(id=$1)")
.setFilterParameter(spendingTx.inputs.get(0).spentOutputId)
.execute(client);
// Node
client.transactions.queryAll({
filter: 'outputs(id=$1)',
filterParameters: [spendingTx.inputs[0].spentOutputId]
}, (tx, next, done, fail) => {
...
})
// Ruby
client.transactions.query(
filter: 'outputs(id=$1)',
filter_parameters: [spending_tx.inputs.first.spent_output_id]
) do |tx|
...
end
The network version will increment to 2 (or 3, depending on the fate of the intermediate 1.1rc release), due to various updates to low-level data structures and serialization/hashing schemes.
Chain Core Version - chaincore/developer:latest (docker image)
Host - Docker
docker run -it -p 1999:1999 chaincore/developer
Initialization successful.
Initializing Chain Core...
error: init schema creating migration table: dial tcp `MY_PUBLIC_IP:5432: getsockopt: connection refused
Listening on: http://localhost:1999
Client access token:
Chain Core is online!
Looks like the the migration is not able to proceed because it cannot to the database. Is there a way to configure the connection on the docker setup?
TxInput always writes the entire prevoutput without checking the serialization flags: https://github.com/chain/chain/blob/9731c85c3473d9ededb00e63c530db8b8c16a64f/protocol/bc/txinput.go#L325
This issue will most likely be resolved by PR #270.
See also discussion here: #270 (comment)
#421 removed the spent_output
annotation for transaction inputs. We merged this change for the sake of moving forward with a big PR (nice work @oleganza!), but there are good reasons to retain that annotation--it provides easy access to transaction history (a feature of past use cases), and in any case, we should avoid the breaking change. I spoke with Oleg and he proposed filing this issue after landing the initial change.
To reproduce:
is_local='yes'
.Most of the exception used in the java SDK are masked by ChainException, it would be a better solution to expose some of the exception "flavors" so that different situations are manageable differently.
E.g.: Transaction.Builder.build throws ChainException, but it's just a mask for: JsonParseException, BadURLException, JSONException, HTTPException, ConnectivityException, APIException.
I'd suggest to delete ChainException and to use proper exception handling.
Code to review:
Trying to build an action with amount
99999999999999999999
results in the following error:
Action 1 amount must be an integer.
However, attempting to build an action with amount
10000000000000000000
returns an action-specific error of:
amount 10000000000000000000 exceeds maximum value 2^63
We want to commit changes to the monorepo such that all affected packages can be updated simultaneously. Typically, this means updates to the API are accompanied by corresponding changes in the SDK. However, with commits such as these, it may be impossible to maintain backward compatibility in both of the following situations:
This is best explained by example: #306 lays the groundwork for an explicit "retire" action, which is anticipated by various experimental features (entry-based tx structures, etc.). It adds an explicit "retire" action to the API and updates the SDKs accordingly. The change to the API is backward compatible with respect to older versions of the SDK. The change to the SDK is not, however, backward compatible with respect to older versions of the API; it requires the corresponding update to the API.
Unfortunately, the monorepo doesn't live in a vacuum: we have a versioning policy (nascent, but a policy nonetheless) stating that version x.y.foo
of cored should be compatible with version x.y.bar
of the SDK. If we adhere to this policy, #306 would motivate an uptick of the minor version. But minor version updates are a grey area: on more than one occasion, we have attached extra political and marketing baggage to the prospect of a minor version update, with the implication that minor version updates cannot be performed casually.
Given the above, minor engineering improvements that happen to break backward compatibility between packages (in some direction, if not bi-directionally) now produce a conundrum: new releases are blocked until we deem the main
branch worthy of a new minor version. If build version updates are always made on recent commits on the main
branch, that means we are also blocked on releasing build version updates (i.e., those that have no backward-compatibility issues whatsoever), simply because they arrived on main
after a breaking change.
Some initial ideas that could break the logjam here:
x.y.foo
of cored must work with any version x.y.bar
of the SDK, or vice versa, but not both. This puts some cognitive load on external developers; we can't reasonably demand that devs remember our version conventions, so we'd have to be super explicit about version requirements in release notes/changelogs. For example, any arbitrary release of the SDK may contain the note "requires version x.y.foo of cored`. There is a burden on external devs to pay very close attention to release notes.main
if it is not feasible to uptick the minor version and the updates occur after a change that breaks backward compatibility for the package in question. This requires some discipline for the branch maintainers to be very aware of the current state of each branch, and to be aware of which features from main
ought to be included, and which should be deferred.At first glance, I can't say I'm in love with any of the above, so I'm hoping someone with more imagination might have some insight. Thoughts welcome!
There's a race condition when a block lands that includes a transaction issuing a new, remote asset. Since block indexing callbacks run in parallel now, it's possible for the core/query
callback to annotate the transaction before the core/asset
package has recorded the new asset in the database. The resulting annotated transaction will be missing the asset definition.
Transactions using the asset in any subsequent blocks will be correctly annotated with the asset definition.
Possible solutions:
Reproducible on 3579bf1
If you attempt to sign for a spending transaction without first adding the relevant key to the SDK signer object, the signer will not add a signature to the transaction. During the submit step, you are not told that signatures are missing; instead, you will receive a CH736 "Transaction not final" error. While this error is technically correct, it is not the right one to deliver to the user. The error should be that the signature is missing, and possibly hint that an xpub needs to be added to the client-side signing object.
The following Ruby script can recreate the problem:
$:.unshift ENV['CHAIN'] + '/sdk/ruby/lib'
require 'chain'
c = Chain::Client.new
s = Chain::HSMSigner.new
asset_key = c.mock_hsm.keys.create
s.add_key(asset_key, c.mock_hsm.signer_conn)
account_key = c.mock_hsm.keys.create
# Don't add the key to the signer
#s.add_key(account_key, c.mock_hsm.signer_conn)
asset = c.assets.create(root_xpubs: [asset_key.xpub], quorum: 1)
acc = c.accounts.create(root_xpubs: [account_key.xpub], quorum: 1)
c.transactions.submit(s.sign(c.transactions.build { |b|
b.issue asset_id: asset.id, amount: 100
b.control_with_account account_id: acc.id, asset_id: asset.id, amount: 100
}))
c.transactions.submit(s.sign(c.transactions.build { |b|
b.spend_from_account account_id: acc.id, asset_id: asset.id, amount: 1
b.retire asset_id: asset.id, amount: 1
}))
The immediate fix for this should be to provide a better error message.
Some suggestions for longer-term fixes:
sign
method could return the number of signatures added. Based on the semantics of the multi-HSM signing interface, it is not an error to add zero signatures, but the user could check.Pretty sure that this is just a race specific to the TestCancelReservation
test. We alter the protocol.Chain
's state tree as test setup which violates the protocol.Chain
's contract.
==================
WARNING: DATA RACE
Write at 0x00c4201421d0 by goroutine 35:
chain/protocol/patricia.(*Tree).Insert()
/Users/jackson/src/chain/protocol/patricia/patricia.go:133 +0x3e4
chain/core/account.TestCancelReservation()
/Users/jackson/src/chain/core/account/reserve_test.go:52 +0x694
testing.tRunner()
/usr/local/go/src/testing/testing.go:657 +0x107
Previous read at 0x00c4201421d0 by goroutine 36:
chain/protocol/state.Copy()
/Users/jackson/src/chain/protocol/state/snapshot.go:38 +0x9d
chain/protocol/memstore.(*MemStore).SaveSnapshot()
/Users/jackson/src/chain/protocol/memstore/memstore.go:52 +0x8a
chain/protocol.NewChain.func2()
/Users/jackson/src/chain/protocol/protocol.go:142 +0x195
Goroutine 35 (running) created at:
testing.(*T).Run()
/usr/local/go/src/testing/testing.go:697 +0x543
testing.runTests.func1()
/usr/local/go/src/testing/testing.go:881 +0xaa
testing.tRunner()
/usr/local/go/src/testing/testing.go:657 +0x107
testing.runTests()
/usr/local/go/src/testing/testing.go:887 +0x4e0
testing.(*M).Run()
/usr/local/go/src/testing/testing.go:822 +0x1c3
main.main()
chain/core/account/_test/_testmain.go:70 +0x20f
Goroutine 36 (running) created at:
chain/protocol.NewChain()
/Users/jackson/src/chain/protocol/protocol.go:148 +0x51b
chain/protocol/prottest.NewChainWithStorage()
/Users/jackson/src/chain/protocol/prottest/block.go:37 +0x14b
chain/protocol/prottest.NewChain()
/Users/jackson/src/chain/protocol/prottest/block.go:25 +0x10e
chain/core/account.TestCancelReservation()
/Users/jackson/src/chain/core/account/reserve_test.go:26 +0x7e
testing.tRunner()
/usr/local/go/src/testing/testing.go:657 +0x107
==================
--- FAIL: TestCancelReservation (0.56s)
testing.go:610: race detected during execution of test
FAIL
We'd like to rollback the changes introduced by 7394ca1, which required signer clients to provide a whitelist of xpubs that the signer was allowed to use. This whitelist was in addition to the list of xpubs already present in the transaction template(s) included in the same signing request.
This will break compatibility between new versions of the SDK and old versions of the core, so it probably constitutes a minor version bump.
When a transaction is constructed in multiple steps, all steps but the last one must specify "allow additional actions" to instruct the signing phase to produce signature programs that commit only to the aspects of the tx already present (so that additional actions don't invalidate the ones that have been signed). But for idempotence, and to harden against certain kinds of replay attacks, when publishing the tx we require the presence of at least one input that commits to the tx as a whole via its hash.
But this doesn't work when the last build step involves only issuance inputs. (Exception: when all inputs are issuances.) That's because the check for tx finality is looking for the specific signature program
DATA_32 <hash> TXSIGHASH EQUAL
but we don't use signature programs for issuances.
This is producing spurious "transaction is not final" errors.
Although there are workarounds, it would be very helpful to have decimal support to the amount field in transactions. For example, if my asset is a currency (USD, EUR, etc) it would be easier to work with floating point numbers that to use strictly integer numbers.
Hi,
Creation of issuance transaction fails for Assets created from dashboard with "Generate new MockHSM key" option.
Works well for the assets created using dashboard If you provide an existing xpub key.
Transaction successfully created..
Chain::APIError: Code: CH735 Message: Transaction rejected Detail: validation fa
iled in script execution, input 0 (program [0x7b7d DROP DUP TOALTSTACK SHA3 0x58
7f76b87e2a4151a0331037ecd15cc7a4a76d249ce15669543ab8d5417cc293 0x01 0x01 CHECKMU
LTISIG VERIFY FROMALTSTACK FALSE CHECKPREDICATE] args [ 207d71e944839c0e76cb0c37
839b64c8a252c6252e95647a370a5f51701ae152c0ae87]): VERIFY failed Request-ID: 5af7
4f4ca6f53b5c71b1
So we continuously deploy our documentation from main.
Debugging an issue earlier, we encountered a transaction with very many inputs - "dust," or numerous small utxos, that had accumulated in an account, being assembled into one big spend. Each of those inputs required a signature from the same root xprv, but with different derivation paths. Rather than a separate roundtrip to the HSM for each input, we could batch HSM requests on a per-xprv basis.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.