cryptonomic / pencroff Goto Github PK

The concurrency of queries to Tezos nodes should be made configurable and set to a reasonable default.

This is a follow up to Cryptonomic/Conseil#633.

We would like to initialize this repo with very simple code to accomplish the objective stated in the above RFC. Like Lorre, Pencroff should execute in an ever-recurring loop:

• Check the block head from the Tezos node
• Check the level of the latest block in the database
• For all blocks available from the node but not in the database:
  • Run all queries currently run by Lorre
  • Save the queries and their results as key-value pairs in the database.

The data could be stored with the following schema:

• Platform
• Network
• Path
• Payload (optional)
• Result

Example record:

• Tezos
• mainnet
• chains/main/blocks/BMa3vcUByVuTmYkySNptX6cbzeJ57EzDahN1y5ku5vBecrgUPXg
• [Some JSON document]

If possible, the Result column should be of JSON type so we can run JSON queries against it in the future.

Pencroff should also do the work of Conseil by exposing an API. This API should take queries of the following format:

http://server:port/$platform/$network/$path

For example, http://localhost/tezos/mainnet/chains/main/blocks/BMa3vcUByVuTmYkySNptX6cbzeJ57EzDahN1y5ku5vBecrgUPXg should return the JSON document stored in the DB.

Code constraints

• No function should be more than five lines
• As far as possible, functions should be pure
• The main file should be no more than ten lines long (excluding imports)
• Branch coverage should be at least 80%

Design constraints

• Use a minimum of frameworks
• Get any framework use preapproved
• Keep things as simple as possible

The final result of this ticket can be tested by running Lorre against it and ensuring that Lorre:

• Ends up with the same data as it would if it ran against a normal Tezos node
• Sees significant speedups for a full chain sync, preferably 100x but 10x is OK for now.