Description

Shelley transactions support metadata, We should allow people to send metadata when invoking combine.

Description

• /block/transcation endpoint receives a BlockIdentifier and we are currently not performing any check against it (it's not needed actually)
• To be comply with Rosetta Spec, we need to do so
• Reference https://community.rosetta-api.org/t/block-transaction-block-identifier-parameter-as-required-parameter/109/2?u=alan.verbner

Description

The following two quereis are performed one after the other. Both perform almost the same query. Ven if it might seem better, just folding the value over the detail one will be probably better as not too many unspents per account (< hundreds) are expected an it might be faster to not to invoke the multiple JOIN statements required

const balanceForAddress = await blockService.findBalanceByAddressAndBlock(accountAddress.address, block.hash);
const details = await blockService.findUtxoByAddressAndBlock(accountAddress.address, block.hash);

Since metadata is beeing filtered by fastify schema, asserts on metadata fields inside response are not being asserttable. We must:

• fix this fastify schema filter
• assert those metadata fields

Description

If queryinf for a transaction on block - 0 this like casts it to boolean and the query breaks https://github.com/input-output-hk/cardano-rosetta/blob/master/src/server/db/blockchain-repository.ts#L345

1. Change the findTransactionByHashAndBlock to not to accept optional parameters
2. Validate the parameters are properly received in the service

When adding this method we should also check in the endpoints that network identifier matches tthe server one

Description

There are some error, for example if the db is down, that it's not returning the Rosetta spec Error definition.

Description

After upgrading cardano-db-sync from 1.5.0 to 2.1.0 I've noticed several changed were made to the schema. We need to fix all our queries to make everything work again.

Description

UPDATE: based on discussions with @patrick-ogrady over here the specs have been updated

This is an online method to be used when building a transactions. This means, it can query the blockchain for information that's needed to create it. In Cardano, the only online information required to do so it's the ttl:

Time-to-live (TTL) - represents a slot, or deadline by which a transaction must be submitted. The TTL is an absolute slot number, rather than a relative one, which means that the –ttl value should be greater than the current slot number. A transaction becomes invalid once its ttl expires. ¹

This endpoint should:

• Receive the amount of slots the user is willing to wait until de transaction is expired named relativeTtl
• Return the ttl as latestBlockNumber.slock + relativeTtl
• Keep in mind that if ttl is a block number and block number is a BigInt, the following type should be used when adding both values

Description

Update: I'm reopening this issue do to some changed made in Roseta Spec

1. Pre-process should receive the relative_ttl and return it (as it is) so it can be sent to /construction/metadata. If not, nothing gets sent to the later edpoint.

Description

This endpoint should receive a list of operations and return a:

1. unsigned transaction represented as hex bytes (this should be txBody.to_bytes()
2. the transaction hash which is going to be signed by the users CardanoWasm.hash_transaction(txBody);

In order to process this request, we need to consider:

• Tansaction inputs wlll be received as operations with the following scheme where we only need to process the coin identifiers:

{
  "operation_identifier": {
    "index": 0
  },
  "type": "Vin",
  "status": "",
  "account": {
    "address": "{{ SENDER }}"
  },
  "amount": {
    "value": "{{ SENDER_VALUE }}",
    "currency": null
  },
  "coin_change": {
    "coin_action": "coin_spent",
    "coin_identifier": {
      "identifier": "{{ COIN_IDENTIFIER }}"
    }
  }
}

• Transaction outputs will be received as:

{
    "operation_identifier": {
     "index": 1
    },
    "type": "Vout",
    "status": "",
    "account": {
     "address": "{{ RECIPIENT }}"
    },
    "amount": {
     "value": "{{ RECIPIENT_VALUE }}",
     "currency": null
    }
   }

Don't forget to add Vin and Vout as accepted operations in https://www.rosetta-api.org/docs/NetworkApi.html#networkoptions

• ttl will come as part of the metadata
• fee will be sum(inputs) - sum(outputs) ... be careful when doing math with BigInt values

See https://docs.cardano.org/projects/cardano-node/en/latest/reference/building-and-signing-tx.html for reference

Description

We are returning the same error code BAD_REQUEST but Rosetta (actually rosetta-cli check) requires different error codes.

Description

roseta-cli and most likely any client will need to query by block 0 but there is no such block in Cardano Blockchain. We should be able to return genesis block when doing so.

@rhyslbw how did you solve it in cardano-grahphql?

Description

This endpoint should implement the following API method.

1. network_idendifier should be checked before executing next steps
2. To support historical balances see the query below taken from cardano-db-sync (now gone)
3. We should filter by account or block_number
4. It should return:
   • A single AccountBalanceResponse.balances.amount object with the total sum of the utxo values for that account
   • AccountBalanceResponse.Metadata should contained the utxo details

It's imporant to mention that when adding metadata, openApi.json file needs to be changed manually otherwise the data won't be returned. See this example

select
    "TransactionOutput".address,
    "TransactionOutput".value,
    "TransactionOutput"."txHash",
    "TransactionOutput".index
  from tx
  join tx_out
    on tx.id = tx_out.tx_id
  join "TransactionOutput"
    on tx.hash = "TransactionOutput"."txHash"
  left outer join tx_in
    on tx_out.tx_id = tx_in.tx_out_id
    and tx_out.index = tx_in.tx_out_index
  where tx_in.tx_in_id is null
  and tx.block <= (select id from block where hash = "hash")

Description

It seems that fastify-openapi-glue noAdditional configuration is being ingnored. I'm almost sure I tried it before and it worked properly sadly I cannot reproduce correct behavior anymore.

How to reproduce

1. Start the app
2. Execute the following request

echo '{ "network_identifier": { "blockchain": "cardano", "network": "mainnet" } }' | http post http://localhost:8080/network/list

3. As /network/list doesn't expect any parameters, a 400 error should be returned

Since we have added the mock db, and we need this latestBlock for some of the current features ongoing, this issue is to fix that query and to add related test about latestBlock

Description

When looking for a transaction, we also need to check the transaction has been included in the block sent as parameter. This is an extra check that might impact performance so that should be considered when doing so.

For reference it has been discussed here

/block endpoint should return transactions as well

In order to make Cardano Rosetta work we will need several tools to work:

• cardano-db-sync
• cardano-node
• cardano-cli
• postgres

Description

This endpoint is supposed to receive a signed transaction so the following snippet will let return the transaction hash:

const parsed = CardanoWasm.Transaction.from_bytes(req.signed_transaction)
CardanoWasm.hash_transaction(parsed.body());

This is the same issue as one of the fixes in here. The query's condition that looks for block_no gives a false value since genesis' block_no is null AND block.block_no = $2. So, the fix remains the same.

Also, in the second img we can see that if clause between 176 and 187 is not necessary and MUST be deleted since it does not have any effect in response from this endpoint.

Description

Current implementation seems to require to put topology file as a relative path to where is read, network-service.ts. It's a little bit confusing and cumbersome for the users to do so. We should change to accept an absolute path. For example:

• current:

TOPOLOGY_FILE_PATH="../../../test/e2e/utils/topology.json"

• desired:

TOPOLOGY_FILE_PATH="~/cardano-node/topology.json"

Considerations

• Please also add a test to check proper error is thrown when file is not found

Description

In order to manipulate addresses and transactions we are going to use https://github.com/Emurgo/cardano-serialization-lib because:

1. It already has working JS bindings (and output seems to be correct when mapped to cardano-cli
2. It follows the standard. [^1|

That being said, this library should be wrapped so we could eventually change it withouth requiring to refactor the whole API

Implementation

• Implement a Cardano service to wrap all the function calls, including this one.
• Implement

const generteAddress(network: NetworkId, publicKey: String ): EnterpriseAddress

• Use the following snippet in the implementation:

const pub = CardanoWasm.PublicKey.from_bytes(
  Buffer.from(keyAsHex, "hex" )
);

const address = CardanoWasm.EnterpriseAddress.new(
  1,
  CardanoWasm.StakeCredential.from_keyhash(pub.hash())
);

How to Test

• The following test vector was generated using cardano-cli:

./cardano-cli shelley address key-gen --normal-key --verification-key-file /tmp/verif.txt --signing-key-file /tmp/sign.txt
./cardano-cli shelley address build --payment-verification-key-file /tmp/verif.txt --mainnet

See verif.txt and sign.txt

So:

• PublicKey: 1B400D60AAF34EAF6DCBAB9BBA46001A23497886CF11066F7846933D30E5AD3F
• NetworkId = 1
• Result: addr1vxa5pudxg77g3sdaddecmw8tvc6hmynywn49lltt4fmvn7cpnkcpx

Description

When querying for transactions we are doing extra queries to the DB that might be avoided:

1. Introduced in #44 we are looking for the txs and then, after checking the amount is less than page_size we perform the same query again
2. If block transactions_count field is zero (and it's not the genesis) we should not query for transactions

Description

It would be great for documentation and testing purposes to write down a simple script as rosetta-cli does to create an address, wait for funds to be received, create a transaction and broadcast it to the network.

Description

When looking for the parent of the block with hash 0x89d9b5a5b8ddc8d7e5a6795e9774d97faf1efea59b2caf7eaf9f8c5b32059df4 an error is returned:

$ echo '{
      "network_identifier": {
          "blockchain": "bitcoin",
          "network": "mainnet",
          "sub_network_identifier": {
              "network": "shard 1",
              "metadata": {
                  "producer": "0x52bc44d5378309ee2abf1539bf71de1b7d7be3b5"
              }
          }
      },
      "block_identifier": {
          "hash": "0x5f20df933584822601f9e3f8c024eb5eb252fe8cefb24d1317dc3d432e940ebb"
      }
  }' | http post http://localhost:8080/block

{
    "error": "Internal Server Error",
    "message": "Cannot read property 'toString' of null",
    "statusCode": 500
}

Description

Based on the following comment by Coinbase team:

We are about to release our automated construction API tester which can infer the txId based on where the output Operation is present. For it to work correctly, the OperationIdentifier.NetworkIndex should be populated with the Index needed to construct a transaction with the output

We need to include the transaction input index in BlockResponse.Block.Transactions[].Operations[].OperationIdentifier.NetworkIndex

• Check if this is the expected behavior
• Ask Coinbase about this specific case

Description

After upgrading to 1.4.1 we forgot to update coin_change 1 in the operation https://github.com/input-output-hk/cardano-rosetta/blob/master/src/server/services/block-service.ts#L37

Description

In order to get the peers, processing topology.json file is needed. It's important to mention that the path to that file should be configured in the .env file as it depends on how the deploy is going to be made.

Description

Some of the tests were marked as todo as we weren't able to implement them, for example, the ones involving latest block.

Now that we have #35 we should be able to develop them

Description

When running rosetta-cli check:data --start 1 the following error occurs:

It's related to #44 because in block 3314 the ADA received in a genesis tx is sent somewhere else but the cli hasn't processed the formet one so, for it's own checks, sees the balance going negative.

Description

Analyze coverage report and analyze if more tests are needed in order to cover some important areas. Do not add tests just to improve coverage, they must make sense.

When providing invalid hex_bytes to sign but a valid public key and unsigned transaction there is no error from emurgo lib. We should better test that behaviour

Description

Rosetta Spec 1.4.1 has been released. It will help us to implement construction api (or at least test them).

1. Import the new spec
2. Don't forget to apply our custom metadata changes
3. Generate the corresponding typescript types (using the yarn script)
4. Implement
   a. coinbase/mesh-specifications#30

Link: https://github.com/coinbase/rosetta-specifications/releases/tag/v1.4.1

Description

This endpoint should receive both a signed and unsigned transaction and parse them accordingly using:

if(signed) {
   const parsed = CardanoWasm.Transaction.from_bytes(request)
}
else {
   const parsed = CardanoWasm.TransactionBody.from_bytes(request)
}

Description

Our current service-xxx functions contain two different types of logic:

• response formatting (and maybe some request parameters check)
• invoking proper db (repository) functions

We should separate this functions in two:

• controller: handles request validation (if any not already done by our schema checks) and response formatting
• services: data manipulation (cross service invocation included and db querying)

We should test the case whenever a repository function call returns a null value to properly test that errors are properly thrown and handled.

(This issue was created because of /network/list PR discussion)

Descripton

This endpoint should receive unsigned transaction and the signatures and create a transaction object. To do so consider:

// ./cardano-cli shelley transaction build-raw --tx-in 10c3c63f2a97ce531730fd2bd708cda1eb08920f79d2abeeb833c7089f13c54e#0 --tx-out Ae2tdPwUPEYwvsWu2uaXGK8NQN2R97J9qDbYqqVnFnBotch3Sjvmzoop3eM+20 --ttl 20 --fee 177398 --out-file /tmp/alan.txt
const txBody = TransactionBody.from_bytes(
  Buffer.from(
    "a4008182582010c3c63f2a97ce531730fd2bd708cda1eb08920f79d2abeeb833c7089f13c54e00018182582b82d818582183581c0b40138c75daebf910edf9cb34024528cab10c74ed2a897c37b464b0a0001a777c6af614021a0002b4f60314",
    "hex"
  )
);

// ./cardano-cli shelley transaction witness --tx-body-file /tmp/alan.txt --mainnet --out-file /tmp/witness --witness-signing-key-file ../sign.txt
const witness = Vkeywitness.from_bytes(
  Buffer.from(
    "8258201b400d60aaf34eaf6dcbab9bba46001a23497886cf11066f7846933d30e5ad3f58406c92508135cb060187a2706ade8154782867b1526e9615d06742be5c56f037ab85894c098c2ab07971133c0477baee92adf3527ad7cc816f13e1e4c361041206",
    "hex"
  )
);

const txWitnesses = Vkeywitnesses.new();
txWitnesses.add(witness);

const txWitnessesSet = TransactionWitnessSet.new();
txWitnessesSet.set_vkeys(txWitnesses);

const tx = Transaction.new(txBody, txWitnessesSet, undefined);

console.log(Buffer.from(tx.to_bytes()).toString("hex"));

Test vector

{
    "network_identifier": {
        "blockchain": "cardano",
        "network": "mainnet"
    },
    "unsigned_transaction": "a4008182582010c3c63f2a97ce531730fd2bd708cda1eb08920f79d2abeeb833c7089f13c54e00018182582b82d818582183581c0b40138c75daebf910edf9cb34024528cab10c74ed2a897c37b464b0a0001a777c6af614021a0002b4f60314",
    "signatures": [
        {
            "signing_payload": {
                "address": "addr1vxa5pudxg77g3sdaddecmw8tvc6hmynywn49lltt4fmvn7cpnkcpx",
                "hex_bytes": "31fc9813a71d8db12a4f2e3382ab0671005665b70d0cd1a9fb6c4a4e9ceabc90",
                "signature_type": "ecdsa"
            },
            "public_key": {
                "hex_bytes": "58201b400d60aaf34eaf6dcbab9bba46001a23497886cf11066f7846933d30e5ad3f",
                "curve_type": "ecdsa"
            },
            "signature_type": "ecdsa",
            "hex_bytes": "8258201b400d60aaf34eaf6dcbab9bba46001a23497886cf11066f7846933d30e5ad3f58406c92508135cb060187a2706ade8154782867b1526e9615d06742be5c56f037ab85894c098c2ab07971133c0477baee92adf3527ad7cc816f13e1e4c361041206"
        }
    ]
}

Should return

{
    "type": "TxSignedShelley",
    "description": "",
    "cborHex": "83a4008182582010c3c63f2a97ce531730fd2bd708cda1eb08920f79d2abeeb833c7089f13c54e00018182582b82d818582183581c0b40138c75daebf910edf9cb34024528cab10c74ed2a897c37b464b0a0001a777c6af614021a0002b4f60314a100818258201b400d60aaf34eaf6dcbab9bba46001a23497886cf11066f7846933d30e5ad3f58406c92508135cb060187a2706ade8154782867b1526e9615d06742be5c56f037ab85894c098c2ab07971133c0477baee92adf3527ad7cc816f13e1e4c361041206f6"
}

Generation

cardano-cli shelley transaction build-raw --tx-in 10c3c63f2a97ce531730fd2bd708cda1eb08920f79d2abeeb833c7089f13c54e#0 --tx-out Ae2tdPwUPEYwvsWu2uaXGK8NQN2R97J9qDbYqqVnFnBotch3Sjvmzoop3eM+20 --ttl 20 --fee 177398 --out-file /tmp/unsigned.txt
cardano-cli shelley transaction witness --tx-body-file /tmp/unsigned.txt --mainnet --out-file /tmp/witness --witness-signing-key-file ../sign.txt
cardano-cli shelley transaction sign --tx-body-file /tmp/unsigned.txt --mainnet --out-file /tmp/signed --signing-key-file ../sign.txt

The block schema definiton includes a metadata field. Do we need to include information like:

• Slot
• Slot Leader
• Epoch
• Size?

Description

As developer I should be able to run e2e tests using a mocked (but real) version of cardano-db-sync postgres instance

Searching for genesis block like this:
SELECT * FROM block WHERE previous IS NULL
gets a block with no txs because tx_count = 0
BUT, whenever we query for txs for that block like this:
SELECT count(*) FROM tx JOIN block ON block.id = tx.block WHERE previous IS NULL
we get a count of 14505 tuples (!!!!) which is kinda weird.

We need to ask Coinbase as this endpoint might not be needed becasuse we are returning everything with the block call

This endpoint should only be used when querying a node for a block does not return all transactions contained within it. All transactions returned by this endpoint must be appended to any transactions returned by the /block method by consumers of this data.

Description

There are three versions that must be returned:

• middleware: this app, it should be taken from package.json version field
• rosetta_version taken from openapi.json info.version field
• node_version hardcode in this issue, it will be tackled in https://github.com/input-output-hk/cardano-rosetta/issues/28

When implementing this task we should also need to extract hardcoded strings for example in block-service.ts

Description

Add logs over the internal layers (services, repository) to improve troubleshooting. To do so, we need to use the default fastify logger pino-logger and be able to send it to the controllers and services when cofiguring them, as when we send the repositories as parameters.

For example:

const configure = (networkRepository: NetworkRepository, blockService: BlockService, logger: Logger): AccountService => ({

Then, we need to use such logger to log every important step with a tag that will help us identify what's going on. It should let users to recreate what happened in a requeste step by step. For example:

logger.debug(accountBalanceRequest, '[accountBalance] Looking for block')
const block = await blockService.findBlock(accountBalanceRequest.block_identifier || {});
logger.debug(block, '[accountBalance] Block find result')

Don't forget:

• to add also log lines for controllers requests and responses.
• to use different log levels

Setup dotenv to be able to use different environment based configurations (db connection string, file paths, etc)

Description

updated

In order to get the node version we need to introduce a new service, similar to CardanoCli what will let us execute commands on CardanoNode so we can invoke, cardano-node --vesion and return that.

1. Introduce a new service that will look like this one
2. Create a function version that will execute cardano-node --version
3. Add a new environment variable CARDANO_NODE_PATH to target the binary
4. Update docker-compose, ci.yml and ecosstem.config.js with proper paths
5. Update readme