Some tx gives unexpected errors. Raw data of it loads though.

Expected behavior

Shouldn't give any error since it's a valid tx.

Actual behavior

Gives unexpected error from the XSN network and TX not found.

Steps to reproduce the behavior

Open: http://xsnexplorer.io/transactions/a27b5e0bf7ed19b4d0bd038029a474fef0e4ccb1543544ff7f4263862d3faab7

Raw data loads though.

Expected behavior

The database should keep a consistent linear chain where there are no repeated blocks by height and every block exist on the rpc server.

Actual behavior

On rare rechain events, the database could get corrupted.

Steps to reproduce the behavior

1. The rpc server publish a message for the new latest block.
2. The explorer starts processing that block, loads it from the RPC server and before loading the transactions related to the block, the rpc server starts processing a rechain event where the block and its transactions are removed, then, when the explorer tries to load the transactions, they don't exist anymore.

These steps cause the explorer to keep retrying the processing of the block without success, if there was more than 1 block involved in the rechain, we could have blocks until N in the database, on this rare rechain event, it might process block N - 2 with a hash that is not in the database and the explorer will add it as a missing block, hence, having two blocks with N - 2 as height.

The explorer can process a missing block and then, fill the database backwardly until there is an existing block, let's say, the database contains blocks from 1 to N and from N + 10 until the latest one, while processing the block N - 9, it will fill all holes until block N, the problem is that if block N is present but the hash doesn't matches the previous blockhash from the block N + 1, the explorer will insert block N again.

Some ideas:

• Add a unique constraint to the schema, this will prevent this situation and we'll be forced to delete an orphan block before adding it, we need to ensure that all related transactions / inputs / outputs are deleted too.
• While processing a new block, check if it is not present on the rpc server, if this occurs, delete it from the database.
• Consider adding a recurrent task that ensures the database has a consistent blockchain, repairing the data if necessary.

The P2WPKH address format is introduced with segwit, an example is bc1qzhayf65p2j4h3pfw22aujgr5w42xfqzx5uvddt, all addresses have the same length and they aren't currently supported by the explorer, we need to perform the following changes:

• Update the schema to allow P2WPKH addresses, this will allow the explorer to seed the segwit blocks / transactions.
• Update the address validator from the server project, this will allow clients to query the API with P2WPKH addresses.
• Update the finder from the web-ui, this will allow users to type the P2WPKH addresses in the finder component.

Currently, the database seeding process is quite flexible, we can fill any block and came back later to fill the holes.

As we are moving away from calls to the xsnd to retrieve the data and using the database instead, it would be better to guarantee consistency.

Update the seeding process to work linearly only, lets say, block N can't be added unless block N - 1 is present.

The latest blocks view doesn't provide as much information as we could

Expected behavior

The latest blocks view is rendered by querying the blocks API and sorting by the time in descending order, it paginates the results and allows the user to navigate to older blocks.

When the user is at the 1st page, we keep polling the server for new blocks, when the user is in another page, we don't poll the server for new blocks to avoid inconsistencies while navigating through the pages.

It would be useful to render the value transferred in each block.

Actual behavior

The latest blocks API returns the 10 latest blocks only, they are retrieved directly from the xsnd, hence, there is no pagination.

Running the backend locally is not that simple

Expected behavior

The following steps should be enough to run the backend locally:

• Replace the SeederModule with another module that doesn't depend on SQS.

Actual behavior

Running the backend project locally is not that simple, we need the following (at least) which is not that simple (or annoying) for new contributors:

• Create an AWS account
• Setup a SQS FIFO queue
• Send a message to the queue to start the sync process

Please list all relevant steps to reproduce the observed behavior.

Hide 0 balances from Richest Addresses.

@durkmurder Alex had a great question about this:

"what about addresses that had coins but they are spent?"

After segwit was introduced, the bech32 addresses started to appear, we need to support this format.

Expected behavior

1. Typing a bech32 address on the search bar is allowed.
2. Querying the API using bech32 addresses is possible.
3. Saving utxos having bech32 address works.

Actual behavior

The bech32 addresses are just rejected because the format is not supported.

Steps to reproduce the behavior

1. Type a bech32 address on the search bar, an error is displayed.
2. Query the API using a bech32 address, an error is returned.
3. Store a bech32 address on the balances table, an error occurs.

Like address, transaction and blockhash; now that we have added a Masternode tab, we should allow users to search a MN by its IP address:

The latest blocks view could be improved.

Expected behavior

When there is a new block discovered, we add a small color effect on it to make it simpler to be aware of, example:

Actual behavior

When a new block is discovered, we just update the block list with no effects at all.

Steps to reproduce the behavior

Go to https://xsnexplorer.io/ and wait for a minute or two to see a new block.

Resolve github vulnerability warning on the web-ui project

Expected behavior

There is no vulnerability warning on the GitHub project page.

Actual behavior

GitHub is displaying a vulnerability warning that seems to be related to the angular build tools.

Steps to reproduce the behavior

1. Fork the repository.
2. Go to the forked repository page.
3. The warning will be displayed.

Create a new endpoint for retrieving a exchange rate in other currencies, like BTC and USD.

Expected behavior

We can retrieve the current exchange rate for other currencies.

Actual behavior

There is no way to do this.

Integrate the sbt-errors-summary plugin

Expected behavior

The sbt-errors-summary plugin is integrated into the server project.

Actual behavior

The sbt-errors-summary is not integrated into the project.

Create the explorerCurrency pipe that takes a number and renders it our currency symbol

Expected behavior

It should be possible to render amounts with this: {{item.value | explorerCurrency}}

Ideally, the currency symbol will be taken from a configuration file.

Actual behavior

We use the following approach to render the explorer amounts: {{item.value}} {{'label.coinName' | translate}}

This has several drawbacks like requiring to repeat the translation key on every language.

There are balances not matching the data from the transaction inputs and outputs.

Expected behavior

The balances table should keep the information from transaction inputs and outputs aggregated.

Actual behavior

There are balances not matching the data from the transaction inputs and outputs, the table is being corrupted for some reason,

Steps to reproduce the behavior

Unknown, occurred in production.

The block view could be improved.

Expected behavior

The block view displays a summary, and a paginated list with a summary for each transaction, the list is sorted from newest to oldest, it displays 10 items by default, example:

Actual behavior

The block view displays the list of all the transactions which doesn't help too much because there are no details for the transactions:

Steps to reproduce the behavior

Go to https://xsnexplorer.io/blocks/95833f1e6a3475a0bf2478865e23a4e731d2f5ccf82883ee9a3d9a9668ee4e24

From every block, get the reward amount, the utxo amount and the wait time (time that has passed since that utxo received the last reward).

Calculate and show averages.

As the database seeding will be linear with the schema having strong consistency, we can get rid of the SQS seeder.

This will reduce the deployment complexity.

While displaying a transaction, there could be several inputs linked to a single address.

Expected behavior

When there are several inputs linked to the same address, the explorer collapses them into a single one, the (2) being the number of collapsed inputs, this number should not be displayed on single addresses:

A nice to have would be a way to click on the collapsed row and expand all the values.

Actual behavior

Several inputs having the same address are listed in a row per input which is not very user friendly:

Steps to reproduce the behavior

Go to https://xsnexplorer.io/transactions/92e941c087761afb28abd5e5f202a86e0c3e2fd2cb91748b4c3a5c269694336e

There are some unnecessary calls on the XSNService.

Expected behavior

There shouldn't be unnecessary calls.

Actual behavior

After the segwit integration, the addressindex was removed, leading to some useless calls on the XSNService, remove them:

def getAddressBalance(address: Address): FutureApplicationResult[rpc.AddressBalance]

def getTransactions(address: Address): FutureApplicationResult[List[TransactionId]]

Update the block API to retrieve a block from the database and fallback to the xsnd.

Expected behavior

There are two endpoints for loading a block:

1. GET /blocks/:query -> Loads the block data from the database.
2. GET /blocks/:query/legacy -> Loads the block data from the xsnd.

The web-ui tries to load a block from the database, when it isn't present, it tries to load it from the xsnd, a message is displayed if it doesn't exists.

When the 1st method is used, the confirmations field should be computed on the client.

Actual behavior

The GET /blocks/:query loads a block from the block from the xsnd while we might have it on the database.

Increase the backend test coverage.

Expected behavior

The test coverage is higher.

Actual behavior

The test coverage is not as good as it can be.

Steps to reproduce the behavior

• Go to https://app.codacy.com/app/AlexITC/block-explorer/files
• Pick up a file with coverage less than 100%.
• Create a meaningful test to increase the coverage of that file or remove unused code.

In order to integrate hardware wallets, we need a way to get unspent outputs for an address.

Expected behavior

Calling GET /api/addresses/:address/utxos returns all unspent outputs for an address, example from xsnd:

[
  {
    "address": "XeNEPsgeWqNbrEGEN5vqv4wYcC3qQrqNyp",
    "txid": "ea837f2011974b6a1a2fa077dc33684932c514a4ec6febc10e1a19ebe1336539",
    "outputIndex": 0,
    "script": "76a914285b6f1ccacea0059ff5393cb4eb2f0569e2b3e988ac",
    "satoshis": 1500000000000,
    "height": 22451
  }
]

Actual behavior

There is no way to get that information.

The previous_blockhash column on the blocks table has invalid data.

Expected behavior

The previous_blockhash column on the blocks table should point to the previous block unless it is the first block.

Actual behavior

At this time, the production database has several corrupted blocks.

The following query return the blocks having an invalid previous_blockhash:

SELECT height, blockhash, previous_blockhash
FROM blocks b
WHERE 0 = (SELECT COUNT(*) FROM blocks WHERE blockhash = b.previous_blockhash) AND
      height > (SELECT MIN(height) FROM blocks);

There are 61 corrupted blocks, all of them seem to be pointing to orphan blocks.

Steps to reproduce the behavior

Unknown, it's occurring in prod.

The next_blockhash column on the blocks table has invalid data.

Expected behavior

The next_blockhash column on the blocks table should point to the next block unless it is the last block.

Actual behavior

At this time, the production database has several corrupted blocks.

The following query return the blocks having an invalid next_blockhash:

SELECT height, blockhash, next_blockhash
FROM blocks b
WHERE 0 = (SELECT COUNT(*) FROM blocks WHERE blockhash = b.next_blockhash) AND
      height < (SELECT MAX(height) FROM blocks);

There are 65 corrupted blocks, 3 of them are pointing the next_blockhash to an orphan block while the rest have null on the next_blockhash, there should be at most 1 block having null on the next_blockhash (the last one).

Steps to reproduce the behavior

Unknown, it's occurring in prod.

Some ideas:

• It is likely that while processing rechain events, we are not clearing the next_blockhash field from the block previous to the orphan block, this could explain why there are blocks having the wrong next_blockhash.
• While adding a new block, there might be cases that we aren't setting the next_blockhash field to the block previous to the new block, this could explain why there are blocks having no next_blockhash.

Freeman requested this feature.

It would be the time when the last transaction occurred for that address.

Comment from Alex: Also, it's worth adding the rank number

Update the logback.xml to log exceptions root cause first

Expected behavior

When an exception is thrown and it has nested causes, the root cause should be the first displayed exception.

Actual behavior

When an exception is thrown and it has nested causes, the root cause is not the first displayed exception.

Steps to reproduce the behavior

Shutting the xsnd or postgres off and leaving the project running show start displaying exceptions.

The home page is not very mobile friendly.

Expected behavior

The details are readable while loading the explorer on small screens, example:

One point would be to render one decimal in the difficulty field.

Please note that all the tabs are affected.

See https://chainz.cryptoid.info/pivx/

Actual behavior

The details are far from readable on small screens:

Steps to reproduce the behavior

Go to http://xsnexplorer.io/ on a mobile device.

One line summary of the issue here.

Expected behavior

As concisely as possible, describe the expected behavior.

Actual behavior

As concisely as possible, describe the observed behavior.

Steps to reproduce the behavior

Please list all relevant steps to reproduce the observed behavior.

While the same time format is displayed in the application, we keep hardcoding the format.

Expected behavior

There is a pipe that allow us to render a time in the preferred application time format, something like {{timestamp | customReadableTime }} should render a readable timestamp.

Actual behavior

We use the following way across all the application to render times: {{blockDetails.block.medianTime * 1000 | date:'MMMM d, y, h:mm:ss a'}}.

In case we choose to use a different format, it will be a pain to keep them all consistent.

While displaying inputs / outputs of a transaction, the total values (sent, received) are not displayed.

Expected behavior

The transaction inputs / outputs display the total accumulated value:

Actual behavior

The total accumulated value from transaction inputs / outputs is not displayed, this makes is not user friendly:

Steps to reproduce the behavior

Go to https://xsnexplorer.io/transactions/92e941c087761afb28abd5e5f202a86e0c3e2fd2cb91748b4c3a5c269694336e

The test should keep the correct previous_blockhash on rare events has failed on travis, rerunning the test make it pass.

Expected behavior

The test should be stable, hence, never fail.

Actual behavior

The test is not stable, travis shows these logs:

[info] - should keep the correct previous_blockhash on rare events *** FAILED ***
[info]   Good(NewBlockAppended(Block(000004645e2717b556682e3c642a4c6e473bf25c653ff8e8c114a3006040ffb8,Some(000003fb382f6892ae96594b81aa916a8923c70701de4e7054aac556c7271ef7),Some(00000766115b26ecbc09cd3a3db6870fdaf2f049d65a910eb2f2b48b566ca7bd),67aa0bd8b9297ca6ee25a1e5c2e3a8dbbcc1e20eab76b6d1bdf9d69f8a5356b8,List(67aa0bd8b9297ca6ee25a1e5c2e3a8dbbcc1e20eab76b6d1bdf9d69f8a5356b8),33960,179,2,536870912,1520276299,1520276270,1161702,1e0ffff0,0000000000000000000000000000000000000000000000000000000000300030,0.000244140625,None))) did not equal Good(MissingBlockProcessed(Block(000004645e2717b556682e3c642a4c6e473bf25c653ff8e8c114a3006040ffb8,Some(000003fb382f6892ae96594b81aa916a8923c70701de4e7054aac556c7271ef7),Some(00000766115b26ecbc09cd3a3db6870fdaf2f049d65a910eb2f2b48b566ca7bd),67aa0bd8b9297ca6ee25a1e5c2e3a8dbbcc1e20eab76b6d1bdf9d69f8a5356b8,List(67aa0bd8b9297ca6ee25a1e5c2e3a8dbbcc1e20eab76b6d1bdf9d69f8a5356b8),33960,179,2,536870912,1520276299,1520276270,1161702,1e0ffff0,0000000000000000000000000000000000000000000000000000000000300030,0.000244140625,None))) (BlockEventsProcessorSpec.scala:323)

Steps to reproduce the behavior

Hard to reproduce, keep running the test several times until it fails.

Some user have complained that running the tests is not straightforward on Windows.

Expected behavior

The README.md from the server project specifies how to run the tests on Windows.

Actual behavior

It is not clear how to run the tests on Windows.

Create a shared component that keeps updating the statistics.

Expected behavior

The statistics are being stored in a component that can share the data with other components, it is being updated frequently (each minute should be enough).

A nice to have is to display notifications when there are important changes, like discovering a new block.

Ideally, the statistics view would display its data from this shared component, and it might be worth making it sticky.

Actual behavior

The statistics are loaded only on the main view, they aren't updated.

The explorer doesn't have a way to display a raw block

Expected behavior

On the block view, there should be a way to display the raw block.

Actual behavior

There is no way to display a raw block.

When #23 was developed, we just forwarded the input/output to the xsnd which requires the addressindex to work. Sadly, this index doesn't exist anymore on our segwit integration.

We need to update the database schema to store utxo related information and update the endpoint to get the data from the database, please note that the database seeder needs to be updated to keep this information.

Based on the segwit requirements, we'll need to update the schema to complete #34 and #35, while we could keep adding migration scripts, it is simpler to just rebuild the database.

Gladly, rebuilding the database is as simple as seeding it from the xsnd, compacting the scripts will simplify understanding the models.

Lets rebuild the migration scripts and add support for utxos.

Also, it would be useful to add foreign key constraints, as the seeding process will be linear (see #37), foreign keys will play an important role to keep consistency.

Please contact Andriy (designer) for links.

Expected behavior

As part of today's launch, We should match our rebranded logo.

Actual behavior

We are displaying the old logo.

Steps to reproduce the behavior

1. navigate to the explorer: https://xsnexplorer.io/
2. Notice the logo.

Ensure that the supplies are computed correctly.

Expected behavior

The supplies should not be that different than before with the segwit integration.

Actual behavior

At first glance, the supplies seem quite different which is likely to be wrong.

Steps to reproduce the behavior

Sync the database and verify the supplies.

Segwit is being integrated to xsnd, once it is done, the explorer won't be able to process new blocks due to format changes.

Expected behavior

The explorer is able to parse segwit blocks and transactions.

Actual behavior

There is no segwit support.

For an unknown reason, the genesis block has a transaction id that can't be retrieved from the xsnd (see X9Developers/XSN#32).

Add a conditional to never load it to be able to see the genesis block into the database.

This allow us to add foreign keys to the previous_blockhash (see #35).

It is tricky to deploy the projects, having scripts will avoid initial setup and deployment problems.

Expected behavior

There should be 1 script per project to deploy it, probably it is worth having another script to deploy all the projects.

Actual behavior

There are files with deployment steps.

Improvements to the richest addresses view

Expected behavior

The view could display a kind of bar on the percentage area, also, rounding the percentages using 2 decimals is enough, the PIVX explorer has this effect:

It might be useful to add a column with the last time the address was involved in a transaction, this could be done here or in #2

Actual behavior

The view doesn't display a bar on the percentage of coins, and the percentages have a lot of decimals:

Steps to reproduce the behavior

Load the richest addresses tab from https://xsnexplorer.io/

The logo is getting stretched, we need to keep it rounded.

Expected behavior

The logo is rounded.

Actual behavior

The logo is stretched:

Steps to reproduce the behavior

Go to https://xsnexplorer.io/

The address view could be improved.

Expected behavior

The address view displays a summary of its balance and then, a paginated list with a summary for each transaction, the list is sorted from newest to oldest, it displays 10 items by default, example:

Actual behavior

The address view displays the list of all the transactions which is slow and doesn't help too much because there are no details for the transactions:

Steps to reproduce the behavior

Go to https://xsnexplorer.io/addresses/Xx6bteTJSHkzF9v5vfedtWjZygXzhQNRqk

There is no simple way to detect breaking API changes from the RPC server.

Expected behavior

Before updating the RPC server, run a suite of integration tests that calls the actual RPC server to ensure there are no breaking changes.

Actual behavior

There are no tests calling the real RPC server.

The explorer doesn't have a way to display a raw transaction

Expected behavior

On the transaction view, there should be a way to display the raw transaction.

Actual behavior

There is no way to display a raw transaction.

In order to integrate hardware wallets, we need a way broadcast a transaction to the network.

Expected behavior

Calling POST /api/transactions with hex in the JSON request body, will broadcast the hex-encoded transaction to the network.

Actual behavior

There is no way to broadcast a transaction to the network.

Due to one unexpected corner case, the chain in the database was broken

Expected behavior

The database chain should be correct, in case it gets broken, it should be temporal and the explorer should recover the correct state.

Actual behavior

We rely on the xsnd to publish a message to SQS each time a new block is discovered, in order to detect rechain events, we expect xsnd to not swallow any event and publish all of them in the right order.

What happened in prod is that block X and Y were appended, then, a rechain occurred and block Y' was appended while event X wasn't published as expected, this resulted in two chains where X is stored instead of X':

1. X -> Y
2. X' -> Y'

Here are the logs displaying this behavior:

2018-06-18 22:26:57,240 [INFO] from com.xsn.explorer.tasks.SQSSeederTask in scala-execution-context-global-60588 - Block processed successfully = 9a35964e2cd2e9bc112654053cb52faf042520285b7ab24c82d59ce814e093de
2018-06-18 22:26:59,105 [INFO] from com.xsn.explorer.processors.BlockEventsProcessor in scala-execution-context-global-60588 - existing latest block = 9a35964e2cd2e9bc112654053cb52faf042520285b7ab24c82d59ce814e093de -> new latest block = 1ccf7a7177c55605deb4089b44e18bc2bd3bc5fa1b358d6727d8450f5c8b0dc0
2018-06-18 22:27:00,103 [INFO] from com.xsn.explorer.tasks.SQSSeederTask in scala-execution-context-global-60588 - Block processed successfully = 1ccf7a7177c55605deb4089b44e18bc2bd3bc5fa1b358d6727d8450f5c8b0dc0
2018-06-18 22:27:46,286 [INFO] from com.xsn.explorer.processors.BlockEventsProcessor in scala-execution-context-global-60601 - Adding possible missing block = e6183988a5c883a199966a5729c737e2133fbab6c59d36a72b3e987cc1f9cb63
2018-06-18 22:27:48,228 [INFO] from com.xsn.explorer.tasks.SQSSeederTask in scala-execution-context-global-60604 - Block processed successfully = e6183988a5c883a199966a5729c737e2133fbab6c59d36a72b3e987cc1f9cb63

This scenario results in a BlockEventsProcessor.ReplacedByBlockHeight which we can use to trigger the backwards synchronization tasks which will fix the chain.

Steps to reproduce the behavior

1. Publish a SQS message with block A.
2. Publish a SQS message with block B.
3. Publish a SQS message with block B'.