Code Monkey home page Code Monkey logo

cardano-explorer's Introduction

Disclaimer

⚠️ Since 2020-02-13, development of these components has been split and moved to different locations ⚠️.

Cardano Explorer

The new cardano-explorer consists of a set of components:

  • cardano-explorer-db which defines common data types and functions that are shared by the following two components. In particular, it defines the database schema.
  • cardano-explorer-node which acts as a Cardano node, following the chain and inserting data from the chain into a PostgreSQL database.
  • cardano-explorer-webapi which serves data from the PostgreSQL database via HTTP.
  • cardano-tx-submit-webapi allows submission of pre-formed transmissions via a HTTP POST operation.

Architecture

The explorer is written in a highly modular fashion to allow it to be as flexible as possible.

The cardano-explorer-node connects to a locally running cardano-node (ie one connected to other nodes in the Cardano network over the internet with TCP/IP) using a Unix domain socket, retrieves blocks and stores parts of each block in a local PostgreSQL database. The database does not store things like cryptographic signatures but does store enough information to follow the chain of blocks and look at the transactions within blocks.

The PostgreSQL database is designed to be accessed in a read-only fashion from other applications. The database schema is highly normalised which helps prevent data inconsistencies (specifically with the use of foreign keys from one table to another). More user friendly database queries can be implemented using Postgres Views to implement joins between tables.

The cardano-explorer-webapi is a client than serves data from the PostgreSQL database as JSON via a HTTP REST API.

Further Reading

  • BuildingRunning: Building and running the explorer node and webapi.
  • SchemaManagement: How the database schema is managed and modified.
  • Validation: Explanation of validation done by the explorer and assumptions made.

cardano-explorer's People

Contributors

alexiohk avatar angerman avatar arjunyel avatar arturwieczorek avatar cleverca22 avatar codiepp avatar coot avatar craigem avatar dcoutts avatar disassembler avatar erikd avatar iohk-bors[bot] avatar jbgi avatar johnalotoski avatar karknu avatar ktorz avatar mrbliss avatar oneedoubled avatar rhyslbw avatar sebastiengllmt avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cardano-explorer's Issues

Come up with a sane way to handle schema versioning and migrations

The database schema is going to change at least once (to add tables required for Shelly). It is rather unlikely that is the only change. Therefore its good to think about this now.

I have previously used the Persistent library's migration functionality to manage schema versioning. This consisted of a set of numbered migration files (most SQL, some in Haskell to do more complicated things) like:

0000-initial-schema.sql
0001-first-change.sql
....
0010-prepare-for-difficult-change.sql
0011-difficult-change.hs
0012-post-difficult-change.sql
...

The SQL migrations would be generated by Persistent, and the Haskell ones hand written. WHen setting up a database from scratch the migrations would be applied in order to get to the most recent version. In addition the database had a table with the current schema version so the migrations could be generated to skip any migration that had already been applied.

However, we are currently thinking of hand writing the schema definition and that means we need to come up with am alternative to the above which hopefully works as well as the above.

Cardano Explorer local /api/blocks/summary/{hash} showing a different result

API: /api/blocks/summary/{hash}

Testing: api/blocks/summary/1b7031d140af37c0eed1f4cc56581c63565567532f4b0b60dbe460c3231bf9e3

Cardano Explorer testnet Web API: https://cardano-explorer.cardano-testnet.iohkdev.io/api/blocks/summary/1b7031d140af37c0eed1f4cc56581c63565567532f4b0b60dbe460c3231bf9e3

Result correct

{
    "Right": [
        {
            "ctbId": "1ae5f6b2ebab97e5cf968abe3368e20cc82de6e5de62ed8962c2ceda5fdf839e",
            "ctbTimeIssued": 1564426936,
            "ctbInputs": [
                {
                    "ctaAddress": "2cWKMJemoBaiqZvEgXxMiLMF4wM8HK2WMfJnv1MPmVVQo7YdYxiNMSa6hSF8PLtdhBmFL",
                    "ctaAmount": {
                        "getCoin": **"20000000000000"**
                    },
                    "ctaTxHash": "Genesis Distribution",
                    "ctaTxIndex": 0
                }
            ],
            "ctbOutputs": [
                {
                    "ctaAddress": "37btjrVyb4KDFnQMJ7xE7jtiK3zfC2rXjbiBHgpk6iBv4TuPN6jfBCUZ51h7KSqGoVjemHHkNcPc5RSfbgCwgLaxbqSkH7BUY3Rci1ujkqvT2vHxRK",
                    "ctaAmount": {
                        "getCoin": "20000000000000"
                    },
                    "ctaTxHash": "1ae5f6b2ebab97e5cf968abe3368e20cc82de6e5de62ed8962c2ceda5fdf839e",
                    "ctaTxIndex": 0
                }
            ],
            "ctbInputSum": {
                "getCoin": "20000000000000"
            },
            "ctbOutputSum": {
                "getCoin": "20000000000000"
            },
            "ctbFees": {
                "getCoin": "0"
            }
        }
    ]
}

Testing Local API:

http://localhost/api/blocks/summary/1b7031d140af37c0eed1f4cc56581c63565567532f4b0b60dbe460c3231bf9e3

{
    "Right": {
        "cbsEntry": {
            "cbeEpoch": 0,
            "cbeSlot": 21366,
            "cbeBlkHeight": 20336,
            "cbeBlkHash": "1b7031d140af37c0eed1f4cc56581c63565567532f4b0b60dbe460c3231bf9e3",
            "cbeTimeIssued": 1564426936,
            "cbeTxNum": 1,
            "cbeTotalSent": {
                "getCoin": "**20000000000000000000"**
            },
            "cbeSize": 916,
            "cbeBlockLead": "9a804607ce670b2d5e60e9b4fdc54b99acf6d66837132e6bdc621ba5",
            "cbeFees": {
                "getCoin": "0"
            }
        },
        "cbsPrevHash": "d4e97a6bc69d123ea649d70b3978f8bf715fe6f5260957320f7c7e73f9b49b79",
        "cbsNextHash": "b02a8245717b5828705168d4719d4cda5a8ccceb1203139ec94bdca6ce7227a6",
        "cbsMerkleRoot": "a5272a68a9b21df30d1200a9d9ab8efc8d2a5c68f6502f9db59d655045a533c3"
    }
}

The total sent according to /api/blocks/summary is 20000000000000000000 , and the total send according to /api/blocks/txs is 20000000000000 (a factor of 10^6 difference).

Changes in cardano-foundation/incentived-testnet-stakepool-registry not reflected in explorer

My pull request #1023 to cardano-foundation/incentivied-testnet-stakepool-registry to update the name, ticker, description, and homepage for stake pool with hash 9ebb7c1efc6262dbbad14ba9ec6b1654d8caf7f9ee37b92cd719ebe8fe11bdaf was merged 7 days ago but the explorer still shows the old information.
The old info was CCP1, Cardano Community Pool, cardanocommunitypool.com, etc. and the new info is SQUID, Squid Pool, squidpool.com, etc.

I'm not sure whether this info is updated in the explorer automatically or manually.

If it is manual, please update the pool information.
If it is automatic, there may be an issue with the update mechanism.

Thanks!

Nix code review

Please review the general nix architecture, how nixos services/scripts function and the docker image generation. Make suggestions as to what could be changed and how long it would take to make the changes.

Issues with filtering for api/genesis/address

Querying http://localhost:8100/api/genesis/address?pageSize=20&page=2 will return list of adresses where each of them has "cgaiIsRedeemed":false

{
	"Right": [
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZB2zrbrdDkQNPdCndghMUJN8o8XjMaJ1jXgwVevf7TUrdmsSP",
			"cgaiGenesisAmount": {
				"getCoin": "781381495000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZ9dH9VC4iVXZRNYe5HGc73AKVMYHExpgYBmDMkgCUgnJGqqqq",
			"cgaiGenesisAmount": {
				"getCoin": "648176763000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZK3YTwof6pQ7dyR7gvAjEFuUJFtDmwkrF1kVBzw7ZhYCdviTw",
			"cgaiGenesisAmount": {
				"getCoin": "300149999000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZ9X7cASSFNG6w2xRzWaEkLojLteLzKNgo26NGAMJ1vCKisd7Y",
			"cgaiGenesisAmount": {
				"getCoin": "132113541000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZCo1gun5HbfcEdEWyVSLc7uj8Tg8U9NhMCfDRWqGsSRce2Uka",
			"cgaiGenesisAmount": {
				"getCoin": "131406876000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZ4zU46CcMwd6QZLf23jyHLdNLcEBua9QAeKX8DyMV5m4o6HYE",
			"cgaiGenesisAmount": {
				"getCoin": "123701030000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZK77edAqpNaPVhaUfTCcoaGepvQPDrSuX1czw5uZmp47F1k73",
			"cgaiGenesisAmount": {
				"getCoin": "115384615000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZ11hWKCn4n66gusxDhXwRee3xxfXA1w3KTM6qD514E9xjb4rG",
			"cgaiGenesisAmount": {
				"getCoin": "109831430000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZ3ix6Xs1tLa4vKmUHYLnwtyDwnp2gyKB58MMTsitgj4SFSZTX",
			"cgaiGenesisAmount": {
				"getCoin": "103431730000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZNEEuiFn5BjuCDbw1KVLEjnMp47UZp2PP4E7v7v2u6F31VJfx",
			"cgaiGenesisAmount": {
				"getCoin": "101363636000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZ4VEgkqXHqk41FiSFbwxjhSwGsibzF2S3tpv7NvvLG8Kj1DW6",
			"cgaiGenesisAmount": {
				"getCoin": "87373125000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZ4smketL8fm5dKMTwZALiCeuQroYRr6CeJBA3tQzR8MT5gpAV",
			"cgaiGenesisAmount": {
				"getCoin": "84885536000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZ3MBDocNEA9H41bPA4MRtc1N4jdgUzWQCqN6g6EXmQ49fU8eE",
			"cgaiGenesisAmount": {
				"getCoin": "82024819000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZ7tCytKv8Qx4DsLPBoEiYadjEbMQSASZmZDNCfYFVCWUcaKKU",
			"cgaiGenesisAmount": {
				"getCoin": "76651905000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZBgmcp7jtA3jTqg8HfnjNYdm9Ln33PT3gZVoYp9HeZ3kxtVTZ",
			"cgaiGenesisAmount": {
				"getCoin": "75692998000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZ43NhMYvw1bkcGnEzdDbm9QTWiRux6Xpy8sorgfSJfazneEsP",
			"cgaiGenesisAmount": {
				"getCoin": "68681632000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZEcMcWsyCiW74DEDmdyuE7A6GeTCZ8NYP7A7hxKAdQxonMwo7",
			"cgaiGenesisAmount": {
				"getCoin": "66714570000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZMB92JqgxAEWfXJo6Ex7wWLoS7REmh81Ue6GgsNrDNs3MeQKA",
			"cgaiGenesisAmount": {
				"getCoin": "63598711000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZ5sz3qbLRZpJZ7nQbV4HdA95yNsummkfWzyEgHhrjx1za5KHg",
			"cgaiGenesisAmount": {
				"getCoin": "62363921000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZGArcqwLkxS8mjCeByh5gVF2jWbC4FTC5YF4NULWPW2ZuCdq5",
			"cgaiGenesisAmount": {
				"getCoin": "59999882000000"
			},
			"cgaiIsRedeemed": false
		}
	]
}

... now same query but with added filter http://localhost:8100/api/genesis/address?pageSize=20&page=2&filter=redeemed returns:

{
	"Right": [
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZC5VVyF7QntbTXEhF1FnEY2XF4Bb7uPRi6CsTJBoR8fLwNU1q",
			"cgaiGenesisAmount": {
				"getCoin": "1000000"
			},
			"cgaiIsRedeemed": true
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZAN64wuATkJCH5xUuJ95PYhfz72BmoEiyP392QaD5mjcXJiBD",
			"cgaiGenesisAmount": {
				"getCoin": "1000000"
			},
			"cgaiIsRedeemed": true
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZJHU4Nkqo1W6pxPWrcBBv5NfeTBYssJWTjp2iJUnjGP5A5ApT",
			"cgaiGenesisAmount": {
				"getCoin": "1000000"
			},
			"cgaiIsRedeemed": true
		}
	]
}

If there was no redeemed entries on previous list, why they appeared after filtering exactly the same list ? Is this a bug ?

What's the function of Dockerfile?

I don't know what's the Dockerfile in this project. I build it, and start it, but it have Error logs, it looks like need something else.

Runit scripts don't allow services to stop/start

While runit is able to start the services in the docker container, it does not work to stop the services!

Ideally it should allow services to be stopped and started again. Plus it should provide accurate status.

Epoch table to resolve performance issue with current View

The Epoch View is a poor performer due to the sheer size of the dataset it needs to parse. This is a critical part of the Explorer App, as it's use for client-side logic, and also takes the primary homepage position. The initial load on the latest epochs (see below) makes for a poor UX, and the load on backend services is significant.

Screenshot 2019-12-03 at 11 36 38

While application caching helps with the load, the optimal place to enable highly performant and low demand queries is at the time of receiving the block, writing to an epoch table instead of the view calculating the result.

/api/genesis/address returns the same addresses for different pages

If I have this query:

GET: http://localhost:8100/api/genesis/address?pageSize=20&page=1

and I change page arg to 2 and then 3, I can see that there is, for example, this address Ae2tdPwUPEZ9dH9VC4iVXZRNYe5HGc73AKVMYHExpgYBmDMkgCUgnJGqqqq present on results for all pages (1,2,3) with the same value {"getCoin":"648176763000000"}
Is this correct behavior ?

If I change page to 4 it won't be present on the results list.

Details below:

GET: http://localhost:8100/api/genesis/address?pageSize=20&page=1
{
	"Right": [
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZGcVv9qJ3KSTx5wk3dHKNn6G3a3eshzqX2y3N9LzL3ZTBEApq",
			"cgaiGenesisAmount": {
				"getCoin": "2074165643000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZB2zrbrdDkQNPdCndghMUJN8o8XjMaJ1jXgwVevf7TUrdmsSP",
			"cgaiGenesisAmount": {
				"getCoin": "781381495000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZ9dH9VC4iVXZRNYe5HGc73AKVMYHExpgYBmDMkgCUgnJGqqqq",
			"cgaiGenesisAmount": {
				"getCoin": "648176763000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZK3YTwof6pQ7dyR7gvAjEFuUJFtDmwkrF1kVBzw7ZhYCdviTw",
			"cgaiGenesisAmount": {
				"getCoin": "300149999000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZ9X7cASSFNG6w2xRzWaEkLojLteLzKNgo26NGAMJ1vCKisd7Y",
			"cgaiGenesisAmount": {
				"getCoin": "132113541000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZCo1gun5HbfcEdEWyVSLc7uj8Tg8U9NhMCfDRWqGsSRce2Uka",
			"cgaiGenesisAmount": {
				"getCoin": "131406876000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZ4zU46CcMwd6QZLf23jyHLdNLcEBua9QAeKX8DyMV5m4o6HYE",
			"cgaiGenesisAmount": {
				"getCoin": "123701030000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZK77edAqpNaPVhaUfTCcoaGepvQPDrSuX1czw5uZmp47F1k73",
			"cgaiGenesisAmount": {
				"getCoin": "115384615000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZ11hWKCn4n66gusxDhXwRee3xxfXA1w3KTM6qD514E9xjb4rG",
			"cgaiGenesisAmount": {
				"getCoin": "109831430000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZ3ix6Xs1tLa4vKmUHYLnwtyDwnp2gyKB58MMTsitgj4SFSZTX",
			"cgaiGenesisAmount": {
				"getCoin": "103431730000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZNEEuiFn5BjuCDbw1KVLEjnMp47UZp2PP4E7v7v2u6F31VJfx",
			"cgaiGenesisAmount": {
				"getCoin": "101363636000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZ4VEgkqXHqk41FiSFbwxjhSwGsibzF2S3tpv7NvvLG8Kj1DW6",
			"cgaiGenesisAmount": {
				"getCoin": "87373125000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZ4smketL8fm5dKMTwZALiCeuQroYRr6CeJBA3tQzR8MT5gpAV",
			"cgaiGenesisAmount": {
				"getCoin": "84885536000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZ3MBDocNEA9H41bPA4MRtc1N4jdgUzWQCqN6g6EXmQ49fU8eE",
			"cgaiGenesisAmount": {
				"getCoin": "82024819000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZ7tCytKv8Qx4DsLPBoEiYadjEbMQSASZmZDNCfYFVCWUcaKKU",
			"cgaiGenesisAmount": {
				"getCoin": "76651905000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZBgmcp7jtA3jTqg8HfnjNYdm9Ln33PT3gZVoYp9HeZ3kxtVTZ",
			"cgaiGenesisAmount": {
				"getCoin": "75692998000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZ43NhMYvw1bkcGnEzdDbm9QTWiRux6Xpy8sorgfSJfazneEsP",
			"cgaiGenesisAmount": {
				"getCoin": "68681632000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZEcMcWsyCiW74DEDmdyuE7A6GeTCZ8NYP7A7hxKAdQxonMwo7",
			"cgaiGenesisAmount": {
				"getCoin": "66714570000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZMB92JqgxAEWfXJo6Ex7wWLoS7REmh81Ue6GgsNrDNs3MeQKA",
			"cgaiGenesisAmount": {
				"getCoin": "63598711000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZ5sz3qbLRZpJZ7nQbV4HdA95yNsummkfWzyEgHhrjx1za5KHg",
			"cgaiGenesisAmount": {
				"getCoin": "62363921000000"
			},
			"cgaiIsRedeemed": false
		}
	]
}
GET: http://localhost:8100/api/genesis/address?pageSize=20&page=2

{
	"Right": [
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZB2zrbrdDkQNPdCndghMUJN8o8XjMaJ1jXgwVevf7TUrdmsSP",
			"cgaiGenesisAmount": {
				"getCoin": "781381495000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZ9dH9VC4iVXZRNYe5HGc73AKVMYHExpgYBmDMkgCUgnJGqqqq",
			"cgaiGenesisAmount": {
				"getCoin": "648176763000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZK3YTwof6pQ7dyR7gvAjEFuUJFtDmwkrF1kVBzw7ZhYCdviTw",
			"cgaiGenesisAmount": {
				"getCoin": "300149999000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZ9X7cASSFNG6w2xRzWaEkLojLteLzKNgo26NGAMJ1vCKisd7Y",
			"cgaiGenesisAmount": {
				"getCoin": "132113541000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZCo1gun5HbfcEdEWyVSLc7uj8Tg8U9NhMCfDRWqGsSRce2Uka",
			"cgaiGenesisAmount": {
				"getCoin": "131406876000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZ4zU46CcMwd6QZLf23jyHLdNLcEBua9QAeKX8DyMV5m4o6HYE",
			"cgaiGenesisAmount": {
				"getCoin": "123701030000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZK77edAqpNaPVhaUfTCcoaGepvQPDrSuX1czw5uZmp47F1k73",
			"cgaiGenesisAmount": {
				"getCoin": "115384615000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZ11hWKCn4n66gusxDhXwRee3xxfXA1w3KTM6qD514E9xjb4rG",
			"cgaiGenesisAmount": {
				"getCoin": "109831430000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZ3ix6Xs1tLa4vKmUHYLnwtyDwnp2gyKB58MMTsitgj4SFSZTX",
			"cgaiGenesisAmount": {
				"getCoin": "103431730000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZNEEuiFn5BjuCDbw1KVLEjnMp47UZp2PP4E7v7v2u6F31VJfx",
			"cgaiGenesisAmount": {
				"getCoin": "101363636000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZ4VEgkqXHqk41FiSFbwxjhSwGsibzF2S3tpv7NvvLG8Kj1DW6",
			"cgaiGenesisAmount": {
				"getCoin": "87373125000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZ4smketL8fm5dKMTwZALiCeuQroYRr6CeJBA3tQzR8MT5gpAV",
			"cgaiGenesisAmount": {
				"getCoin": "84885536000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZ3MBDocNEA9H41bPA4MRtc1N4jdgUzWQCqN6g6EXmQ49fU8eE",
			"cgaiGenesisAmount": {
				"getCoin": "82024819000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZ7tCytKv8Qx4DsLPBoEiYadjEbMQSASZmZDNCfYFVCWUcaKKU",
			"cgaiGenesisAmount": {
				"getCoin": "76651905000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZBgmcp7jtA3jTqg8HfnjNYdm9Ln33PT3gZVoYp9HeZ3kxtVTZ",
			"cgaiGenesisAmount": {
				"getCoin": "75692998000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZ43NhMYvw1bkcGnEzdDbm9QTWiRux6Xpy8sorgfSJfazneEsP",
			"cgaiGenesisAmount": {
				"getCoin": "68681632000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZEcMcWsyCiW74DEDmdyuE7A6GeTCZ8NYP7A7hxKAdQxonMwo7",
			"cgaiGenesisAmount": {
				"getCoin": "66714570000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZMB92JqgxAEWfXJo6Ex7wWLoS7REmh81Ue6GgsNrDNs3MeQKA",
			"cgaiGenesisAmount": {
				"getCoin": "63598711000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZ5sz3qbLRZpJZ7nQbV4HdA95yNsummkfWzyEgHhrjx1za5KHg",
			"cgaiGenesisAmount": {
				"getCoin": "62363921000000"
			},
			"cgaiIsRedeemed": false
		},
		{
			"cgaiCardanoAddress": "Ae2tdPwUPEZGArcqwLkxS8mjCeByh5gVF2jWbC4FTC5YF4NULWPW2ZuCdq5",
			"cgaiGenesisAmount": {
				"getCoin": "59999882000000"
			},
			"cgaiIsRedeemed": false
		}
	]
}

webapi: Provide Previous Output References in Transaction Payloads

Provide Previous Output References in Transaction Payloads

Changes Needed:

The structure of each input needs to include a reference to the previous outputs that they relate to, namely:

  • The transaction hash of the previous output
  • The index of the previous output within the transaction

Relevant Endpoints

/api/blocks/txs/{block_hash}
http://cardanoexplorer.com/api/blocks/txs/fe8b3ca7929960f3aecf9178cdeff70f3bee4ffcab0e98c077389efdbc86dc13

/api/txs/summary/{tx_hash}
http://cardanoexplorer.com/api/txs/summary/2db79d74c7a1ce6a4f04654aea7554c47fcc5e693d88028097b16b828b961e85

Why is this required?

This is needed to appropriately mark the UTXOs as they are spent

Additional References:

Desired Response Structure

Main changes are highlighted in yellow. Note: This also changes the input and output payloads to use structured responses, but this isn’t absolutely required.

GET /api/txs/summary/15cb5b8b95005486ae3055ed4f16afc63c5eda7b209c5fedd9636e232f0da6fa

{ "id": "15cb5b8b95005486ae3055ed4f16afc63c5eda7b209c5fedd9636e232f0da6fa"
, "txTimeIssued": 1568920191
, "blockTimeIssued": 1568920191
, "blockHeight": 1234
, "blockHash": "275c77651a7e6cd416a839d5ae5d856a02d2aaa1191775615510a4f188905f3a"
, "fees": "180211"
, "inputs": 
    [ { "address": "DdzFFzCqrhsmcJZyFmxLb1VtiWh7eNWL5UXYpamRceky9esqQp9XTN7zGVfJ4TcmcMydBLmkw5h4SM9J6tVFzR7bGjeFWGydZ1ERdb7a"
      , "amount": "819613"
      , "outputTxHsh": "15cb5b8b95005486ae3055ed4f16afc63c5eda7b209c5fedd9636e232f0da6fa"
      , "outputIndex": 2
      }
    ,
        ...
    ]
, "outputs":
    [ { "address": "DdzFFzCqrhsyE2NRAiFPmazW2E7YwNJzgXDz9EVkkL7HAA2apfyEuuT876ZgFoU3PWz3mgVZfqLXAsrvgkcxtXhwfsmrKbSCNRnnsdsq"
       , "amount": "1278088301"
       , "txHash": "2db79d74c7a1ce6a4f04654aea7554c47fcc5e693d88028097b16b828b961e85"
       , "index": 0
       }
    , 
      ...
    ]
}

pageSize argument is ignored for endpoint: /api/blocks/pages

Run:
curl http://localhost:8100/api/blocks/pages?page=1&pageSize=20

and then change pageSize to different values - it doesn't affect the result

Also - for old explorer we got information about page size limit - https://cardanoexplorer.com/api/blocks/pages?page=1&pageSize=15
{"Left":"We currently support only page size of 10."}

Which is not present in new one.

NixOS tests for cardano-explorer

Create a NixOS test using chairman cluster from cardano-node to ensure explorer services at least run and 2 blocks are successfully imported into postgres.

Error policies for the explorer

The explorer should not silently go down with an exception, but retry subscription.
This is a blocker for releasing the explorer.

how explorer determines the database name

the explorer currently parses the pgpass file to determine the database to connect to.
this is problematic as it prevents me from having a pgpass file:

localhost:5432:*:cexplorer:<pw>

where I allow the user 'cexplorer' to connect to any db in that server.
And, it even ignores the PGDATABASE env var though it is present.

I would prefer that reading from the env var would have priority, and parsing the pgpass file only be a fallback solution.

ERROR: could not resize shared memory segment not handled

Scenario: Running a docker stack with the following services, where the memory allocation for the PostgreSQL container was 64MB (the default)

  • cardano-node
  • cardano-byron-proxy
  • cardano-explorer
  • postgres

Deep into the sync, at least 40 but I also saw another attempt make it to 80 epochs in, postgres is looking for more memory:

ERROR:  could not resize shared memory segment "/PostgreSQL.1197429420" to 12615680 bytes: No space left on device
  1. 64MB is pretty slim, but worth looking for possible leak since it occurs after running for some hours. Edit: This is in the scope of the postgres container, so a leak would be out of our scope here.
  2. The error is not handled by the node client, and seems to just endlessly wait. The logs show no error, just the latest write operation info

/api/blocks/txs/{hash} output incorrectly lists outputs from other txs in the same block

API : /api/blocks/txs/{hash}

Testing /api/block/txs/e4c2b1b3ab91db689ddf295fc6a90320541d9139d59b21279d12ed8207b1f514

Block Hash: e4c2b1b3ab91db689ddf295fc6a90320541d9139d59b21279d12ed8207b1f514
Tx Hashes: 55c5e09389939e7c84b43b630e1ecf23ffc785c57caccd6aef85422293d4059f and 12c9840c0f40350351572433165141d4f63def3ae95e564f1b964b8117db384a

It appears as the outputs for both transactions appear in both transactions:

{
    "Right": [
        {
            "ctbId": "55c5e09389939e7c84b43b630e1ecf23ffc785c57caccd6aef85422293d4059f",
            "ctbTimeIssued": 1571229496,
            "ctbInputs": [
                {
                    "ctaAddress": "37btjrVyb4KF4VKhPCXXmv6poj6YWoy1FxAbaMXQa4kEgb5uqAVKrYbavA9g4ji9igCN8UExpjxqWSB6dCpXug6e2qk91zyuYzCzWTE4CMzkUS44Nt",
                    "ctaAmount": {
                        "getCoin": "601851392"
                    },
                    "ctaTxHash": "cae6db09cbab94b5dde1d110705d5c527b697e3a50493d3db153426dc771a33c",
                    "ctaTxIndex": 1
                }
            ],
            "ctbOutputs": [
                {
                    "ctaAddress": "37btjrVyb4KBBJfmirV9pwuTQtP9n5v5ZhAyg6aodmVhEJw3b9P8FX5NLHnasTcN1DgsguuYQHg9uAfq56typXRH2qiuTVSUoPZXbpUMoBuXpXDoAW",
                    "ctaAmount": {
                        "getCoin": "12000000"
                    },
                    "ctaTxHash": "55c5e09389939e7c84b43b630e1ecf23ffc785c57caccd6aef85422293d4059f",
                    "ctaTxIndex": 0
                },
                {
                    "ctaAddress": "37btjrVyb4KBEaxSLmcG1duygfyGuHZ3M1peccfnMbdiGzXAnZRtewuiUS6WzhWB4GcACizLrvs18sevwDsxUgdXDvJ3yeYLd8rKpEqXnRWYAebKpe",
                    "ctaAmount": {
                        "getCoin": "589679311"
                    },
                    "ctaTxHash": "55c5e09389939e7c84b43b630e1ecf23ffc785c57caccd6aef85422293d4059f",
                    "ctaTxIndex": 1
                },
                {
                    "ctaAddress": "37btjrVyb4KBoQF41K6DxCKJETjda8Anq9uvHKys2MUSRwKe3KFksueKceEUhuYWZkKjW5ixM1mjFgkMyDYKsq3y6MKycaVcJbqXHaMPY5puRXLynt",
                    "ctaAmount": {
                        "getCoin": "704862287"
                    },
                    "ctaTxHash": "12c9840c0f40350351572433165141d4f63def3ae95e564f1b964b8117db384a",
                    "ctaTxIndex": 0
                },
                {
                    "ctaAddress": "37btjrVyb4KDy5mPNiXugMbjRFh6j4NhWUTABA3Xv5kzsKMN8k4N73DyWU7RkSwx6YheB31mveMVeh8DGVMyXiz58GvHVdBf2ALd4u7b8xy8RvayrT",
                    "ctaAmount": {
                        "getCoin": "12000000"
                    },
                    "ctaTxHash": "12c9840c0f40350351572433165141d4f63def3ae95e564f1b964b8117db384a",
                    "ctaTxIndex": 1
                }
            ],
            "ctbInputSum": {
                "getCoin": "601851392"
            },
            "ctbOutputSum": {
                "getCoin": "1318541598"
            },
            "ctbFees": {
                "getCoin": "-716690206"
            }
        },
        {
            "ctbId": "12c9840c0f40350351572433165141d4f63def3ae95e564f1b964b8117db384a",
            "ctbTimeIssued": 1571229496,
            "ctbInputs": [
                {
                    "ctaAddress": "37btjrVyb4KDLgdheXH4HeaLrE442Cj2wBqixmBwh5Ht2LFVwDeoz47QD7hKESRrnfYNciTfLcPPTXXWeZrtmSxHkb86Z2hbwuVJ9XmmuX9pY9VL97",
                    "ctaAmount": {
                        "getCoin": "717034368"
                    },
                    "ctaTxHash": "a15a765b748e0e461e58c58773d5aaa77cbd0edb5acbebc9a36999597bdda720",
                    "ctaTxIndex": 0
                }
            ],
            "ctbOutputs": [
                {
                    "ctaAddress": "37btjrVyb4KBBJfmirV9pwuTQtP9n5v5ZhAyg6aodmVhEJw3b9P8FX5NLHnasTcN1DgsguuYQHg9uAfq56typXRH2qiuTVSUoPZXbpUMoBuXpXDoAW",
                    "ctaAmount": {
                        "getCoin": "12000000"
                    },
                    "ctaTxHash": "55c5e09389939e7c84b43b630e1ecf23ffc785c57caccd6aef85422293d4059f",
                    "ctaTxIndex": 0
                },
                {
                    "ctaAddress": "37btjrVyb4KBEaxSLmcG1duygfyGuHZ3M1peccfnMbdiGzXAnZRtewuiUS6WzhWB4GcACizLrvs18sevwDsxUgdXDvJ3yeYLd8rKpEqXnRWYAebKpe",
                    "ctaAmount": {
                        "getCoin": "589679311"
                    },
                    "ctaTxHash": "55c5e09389939e7c84b43b630e1ecf23ffc785c57caccd6aef85422293d4059f",
                    "ctaTxIndex": 1
                },
                {
                    "ctaAddress": "37btjrVyb4KBoQF41K6DxCKJETjda8Anq9uvHKys2MUSRwKe3KFksueKceEUhuYWZkKjW5ixM1mjFgkMyDYKsq3y6MKycaVcJbqXHaMPY5puRXLynt",
                    "ctaAmount": {
                        "getCoin": "704862287"
                    },
                    "ctaTxHash": "12c9840c0f40350351572433165141d4f63def3ae95e564f1b964b8117db384a",
                    "ctaTxIndex": 0
                },
                {
                    "ctaAddress": "37btjrVyb4KDy5mPNiXugMbjRFh6j4NhWUTABA3Xv5kzsKMN8k4N73DyWU7RkSwx6YheB31mveMVeh8DGVMyXiz58GvHVdBf2ALd4u7b8xy8RvayrT",
                    "ctaAmount": {
                        "getCoin": "12000000"
                    },
                    "ctaTxHash": "12c9840c0f40350351572433165141d4f63def3ae95e564f1b964b8117db384a",
                    "ctaTxIndex": 1
                }
            ],
            "ctbInputSum": {
                "getCoin": "717034368"
            },
            "ctbOutputSum": {
                "getCoin": "1318541598"
            },
            "ctbFees": {
                "getCoin": "-601507230"
            }
        }
    ]
}

Expected Result

The expectation is the outputs to be isolated into the transaction they are associated with.

The corresponding response on the old version for this particular transaction looks correct:
https://cardano-explorer.cardano-testnet.iohkdev.io/api/blocks/txs/e4c2b1b3ab91db689ddf295fc6a90320541d9139d59b21279d12ed8207b1f514

Query performance

Two queries currently have poor performance when the address being queries has a large number of transactions (10000 or more). Those queries are:

  • /api/block/{blkHash}/address/{address}
  • /api/addresses/summary/{address}

/api/addresses/summary/<address> result missing inputs which affects final balance

API:/api/addresses/summary/

Testing:/api/addresses/summary/DdzFFzCqrhsjMibnSGdcS1DquG6HbWX84iUKcjvJQnynmKcfF5gRQb95dMgGSUkbpubrLwRqjWop3ppZ2pPhrRcRWYN95bVVuPXnNzdS

Below is the output from the old Explorer API

"Right": {
    "caAddress": **"DdzFFzCqrhsjMibnSGdcS1DquG6HbWX84iUKcjvJQnynmKcfF5gRQb95dMgGSUkbpubrLwRqjWop3ppZ2pPhrRcRWYN95bVVuPXnNzdS"**,
    "caType": "CPubKeyAddress",
    "caTxNum": 16,
    "caBalance": {
      "getCoin": "0"
    },
    "caTxList": [
        ...
      {
        "ctbId": **"d383a83c9ad8c883baef6299cb97a5558d254e9692360d14de40b5cc30754bfc"**,
        "ctbTimeIssued": 1565353231,
        "ctbInputs": [
          [
            **"DdzFFzCqrhsjMibnSGdcS1DquG6HbWX84iUKcjvJQnynmKcfF5gRQb95dMgGSUkbpubrLwRqjWop3ppZ2pPhrRcRWYN95bVVuPXnNzdS",**
            {
              "getCoin": "2842000000"
            }
          ],
          [
            **"DdzFFzCqrht44bkApWAvdXeDvivDRaJCCMCADvD26hgwAdSTEpF8Z2C545WrwBDg5qnDhQHzDdVqbRFbThd8hwAWQKa5yDcXg1LTTmhx"**,
            {
              "getCoin": "23881315761"
            }
          ],
          [
            **"DdzFFzCqrhshwcV5BY69hMDCydNV6q56WWG3msbUqSS4kDUWCiMzMRcR71jVgDM39CSayA8UCvGXRpwWnJ5jNx2gycUUmgwEAstStELh"**,
            {
              "getCoin": "21651121459"
            }
          ]
        ],

Below is the output from the new Explorer API

{
  "Right": {
    "caAddress": {
      "unCAddress": "DdzFFzCqrhsjMibnSGdcS1DquG6HbWX84iUKcjvJQnynmKcfF5gRQb95dMgGSUkbpubrLwRqjWop3ppZ2pPhrRcRWYN95bVVuPXnNzdS"
    },
    "caType": "CPubKeyAddress",
    "caChainTip": {
      "ctBlockNo": 3646124,
      "ctSlotNo": 3648181,
      "ctBlockHash": "ce1111ebd1668379e99a18086ab09d1324a92e0b3948701980e860ddcd7c6bee"
    },
    "caTxNum": 54,
    "caBalance": {
      "getCoin": "361896000001"
    },
    "caTotalInput": {
      "getCoin": "706955000001"
    },
    "caTotalOutput": {
      "getCoin": "345059000000"
    },
    "caTotalFee": {
      "getCoin": "-331559152331029"
    },
    "caTxList": [
        ...
      {
        "ctbId": "d383a83c9ad8c883baef6299cb97a5558d254e9692360d14de40b5cc30754bfc",
        "ctbTimeIssued": 1565353231,
        "ctbInputs": [
          {
            "ctaAddress": {
              "unCAddress": "DdzFFzCqrhsjMibnSGdcS1DquG6HbWX84iUKcjvJQnynmKcfF5gRQb95dMgGSUkbpubrLwRqjWop3ppZ2pPhrRcRWYN95bVVuPXnNzdS"
            },
            "ctaAmount": {
              "getCoin": "2842000000"
            },
            "ctaTxHash": "a2633f350ebad5a34584588812458fb0806f5e3ef27cfcd6f6e594c26628d1ea",
            "ctaTxIndex": 1
          }
        ],
        "ctbOutputs": [
          {
            "ctaAddress": {
              "unCAddress": "DdzFFzCqrhtCSrLj9EFH7GNjLVjGv1MA9CMMVx7vsjNCzzu66E1r4k9ee9dXPzHC5PfywSEQEkgeSX2LpkdKpwbbG8PYbzmguMeEcdN8"
            },
            "ctaAmount": {
              "getCoin": "25503421810"
            },
            "ctaTxHash": "d383a83c9ad8c883baef6299cb97a5558d254e9692360d14de40b5cc30754bfc",
            "ctaTxIndex": 1
          },
          {
            "ctaAddress": {
              "unCAddress": "DdzFFzCqrhtDCMvPYYGVoAgx6bC97K2RcL8tyWfPw86aQJDsgcHc94N9hb3z4ajScr6eGAGkT6D6o1pxcUowhkhf9F5ZjQezTYv8ALpP"
            },
            "ctaAmount": {
              "getCoin": "22870826717"
            },
            "ctaTxHash": "d383a83c9ad8c883baef6299cb97a5558d254e9692360d14de40b5cc30754bfc",
            "ctaTxIndex": 0
          }
        ],
        "ctbInputSum": {
          "getCoin": "2842000000"
        },
        "ctbOutputSum": {
          "getCoin": "48374248527"
        },
        "ctbFees": {
          "getCoin": "-45532248527"
        }
      },
        ...
    ]
  }
}

Result: The new API does not appear to return all the inputs for the transactions,
specifically d383a83c9ad8c883baef6299cb97a5558d254e9692360d14de40b5cc30754bfc. It only returns 1 of 3 that the old API returns.
This then throws off the total input sum, and the fees (which end up negative).

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.