terra-money / mantle-sdk Goto Github PK

Currently indexer.RunIndexeRound() runs all indexers in goroutines, hence each errors in either querier or committer per indexer may throw errors but are not handled correctly.

As a result committer.Commit() is still being called, committing whatever depsResolver has in cache.

mantle-sdk breaks when there is a validator/distribution related message gets processed, due to the fact there is no history of validator updates being kept.

Instead of hardcoding this behaviour, make use of tendermint's BlockExecuter and State.

as mantlemint was introduced, mantle no longer maintains transactional db commit. This results in occasional state mismatch between tmdb and application db.

current design of testkit requires user to care about validators and sign methods as well.

abstract them in a nicer interface

func NewSimMantle(
    NumberOfAccounts uint64,
    NumberOfValidators uint64,
    Indexers...
) SimMantle 

type SimMantleConfiguration {
    CreateValidators []SimMantleValidator
    CreateAccounts []SimMantleAccount
}

// 10 accounts are pre-defined, first one is set as validator
func DefaultSimMantleValidator() []SimMantleValidator
func DefaultSimMantleAccounts() []SimMantleAccount

func NewBlock() SimBlock
func (sm SimBlock) ProposedBy() tmtypes.Block

func NewTx(messages sdk.Msg) SimTx
func (st SimTx) Sign(signer Account) tmtypes.Tx

in auth.StdTx, msgs are defined as interface. msgpack doesn't know how to handle the underlying structure

• use scalar.

If an indexer output is inlined by another entity, the graphql arguments are merged to the top level. This behaviour:

• reduces granularity of graphql queries; filters on nested/relational entities cannot work properly
• blocks deep level KVIndex generation as root level kvIndexEntry will always take precedence

I am sorry if this is not the appropriate repository, but it's hard to find a relevant one for my question.

I query from all pool info from mantle:

https://mantle.terra.dev/?allPairsQuery
json
{"query":"\n  query WasmContractsContractAddressStore($contract: String, $msg: String) {\n    WasmContractsContractAddressStore(\n      ContractAddress: $contract\n      QueryMsg: $msg\n    ) {\n      Height\n      Result\n    }\n  }\n","variables":{"contract":"terra1ulgw0td86nvs4wtpsc80thv6xelk76ut7a7apj","msg":"{\"pairs\":{\"limit\":30,\"start_after\":[{\"token\":{\"contract_addr\":\"terra1pxuknzmg3kup7790vjzssjjuelrq3a3ys4aml6\"}},{\"native_token\":{\"denom\":\"uusd\"}}]}}"}}

which returns different pools, but importantly this pool:

\"contract_addr\":\"terra1nx2auchhaq346scypzrksvf5h60sywwygcrgvk\",\"liquidity_token\":\"terra1pkfpkm4cjd4elhp4kwt46anguj76vkh4azyfxw\"},{\"asset_infos\":[{\"token\":{\"contract_addr\":\"terra1pfynaj6mslxnlekjx7f6rxgfastl46c4hg3kg7\"}},{\"native_token\":{\"denom\":\"uusd\"}}],

terra1nx2auchhaq346scypzrksvf5h60sywwygcrgvk

Now when I query https://hive.terra.dev/graphql for this pool info, I get Internal server error:

https://hive.terra.dev/graphql
{
terra1nx2auchhaq346scypzrksvf5h60sywwygcrgvk:wasm{contractQuery(contractAddress:"terra1nx2auchhaq346scypzrksvf5h60sywwygcrgvk"query:{pool:{}})}
}

{
	"errors": [
		{
			"message": "[Internal Server Error] error",
			"locations": [
				{
					"line": 2,
					"column": 51
				}
			],
			"path": [
				"terra1nx2auchhaq346scypzrksvf5h60sywwygcrgvk",
				"contractQuery"
			],
			"extensions": {
				"code": "INTERNAL_SERVER_ERROR",
				"exception": {
					"status": 500,
					"name": "LCDClientError",
					"message": "[Internal Server Error] error",
					"stacktrace": [
						"LCDClientError: [Internal Server Error] error",
						"    at WasmService.contractQuery (/app/dist/wasm/wasm.service.js:62:19)",
						"    at runMicrotasks (<anonymous>)",
						"    at processTicksAndRejections (internal/process/task_queues.js:95:5)",
						"    at async target (/app/node_modules/@nestjs/core/helpers/external-context-creator.js:77:28)"
					]
				}
			}
		}
	],
	"data": null
}

All the other pools seem to work.

What is the problem with this particular pool? Is there a blacklist somewhere so I could omit this certain pool that causes error?

Or can I query the graphql so it ignores internal server error and returns answers for other queries(I query info for multiple pools, with 100kb requests.)

Overview

This repo will be used for making ETL solution for Terra. We need to gather all types and use cases for making the spec.

To Do

• Find and gather all types and use cases from Terra ecosystem.
• Find out what needs to added or transformed from core data to support current applications.
  • Example 1: we need ProposalTallyingParams at voting end time to every Proposals for displaying the result but core doesn't have it on the Proposal structure.
  • Example 2: we need oracle price table for providing historical data

db LSM tree must be compacted periodically to avoid slow read performance. Given that it's easy that mantle-db grows a few hundreds of GB's, compaction takes time, and during compaction all db operation is locked; this affects service availability.

How?

query {
   BaseState(Height: 25) {
    ...
  }
  BaseStates(Height: "@range(2, 25)") {
    ...
  }
}

• mantle allows range search with @range parameter,
• however the index parameter scalar type only accepts the type of index type (i.e. uint64 -> gql Int, string -> gql string)
• so if the user needs to do @range(2, 25) on for example Height whose type would have been uint64, the behaviour is currently not supported
• either make graphql scalar type that supports both, or take plural form of the index parameter and use that as range search parameter.

currently every commit call will index the entity with Height by default. This is useful for transactional commits, but for caching purposes (as in the entity is expected to be overwritten over time, i.e. only persisting the latest info) the current behaviour just takes up a lot of storage for nothing.

Current way of making relations (i.e. inlining output of other indexer in an entity) is unoptimised and takes up too much disk space.

Make use of pointer syntax to make relations.

depsResolver.GetState() can be called during an indexer commit round.