centrifuge / centrifuge-chain Goto Github PK

View Code? Open in Web Editor NEW

179.0 179.0 73.0 91.01 MB

Centrifuge Chain: the layer-1 blockchain for real-world assets, built on Substrate.

Home Page: https://centrifuge.io

License: GNU Lesser General Public License v3.0

Rust 98.73% Shell 0.54% Dockerfile 0.05% JavaScript 0.39% Nix 0.12% Python 0.16%

centrifuge-chain's People

Contributors

Stargazers

Watchers

Forkers

vimukthi-git philipstanislaus lucasvo mikiquantum ansermino juniuszhou c410-f3r greyecologist chainsafe ivanceras cleancoindev mewtopia williamwang1 jaybutera modriscoll12 xmxanuel web3capital branciard lmahanand asymmetric ahmeddh mustermeiszer branan daniel-flex ilinzweilin rapelli912 1crazymoney resovepant8 esimpsontheartist tankofzion girazoki xlc stanly-johnson 99kies lstrust amadeobrands dorucioclea cdamian akhilesharora datasingularity mpontus pinkdiamond1 thea-leake omahs nunoalexandre crypto-zone polkaverse web3-net bestxeosx-gm ilhanu iq-scm solwschewe bufordsnyde793 socool728 whitehatnomad hoanghungict sumanth-krishna chralt98 dantegpt muhtasimtanmoy tux972 geldnetwork umeeshanan yanilmyays wischli wizzard2099 star011saleh kai-610 peicuiping

centrifuge-chain's Issues

Setup Amber validator libp2p private keys in storage + setup our own accounts for validation instead of Alice and Bob

Implement GetByAnchorID with index

GetByAnchorID
Make sure that the “getByAnchorId” does not return anchor data after evict date has been reached.
This solves the issue that the bucket might be evicted already and the index is still existing. -> Then the evict has to assure that it clears the index one after another.

given anchor id = 𝐴𝐼𝐷
𝑡𝑒𝑣𝑖𝑐𝑡=𝐴𝑛𝑐ℎ𝑜𝑟𝐸𝑣𝑖𝑐𝑡𝐷𝑎𝑡𝑒𝑠.𝑔𝑒𝑡(𝐴𝐼𝐷) => O(1)
𝑎𝑛𝑐ℎ𝑜𝑟𝐷𝑎𝑡𝑎=𝑟𝑒𝑎𝑑_𝑐ℎ𝑖𝑙𝑑_𝑠𝑡𝑜𝑟𝑎𝑔𝑒(𝑡𝑒𝑣𝑖𝑐𝑡,𝐴𝐼𝐷) => 𝑂(𝑙𝑒𝑛(𝑐𝑜𝑛𝑐𝑎𝑡(𝑡𝑒𝑣𝑖𝑐𝑡,𝐴𝐼𝐷))) => 𝑂(1)

Eviction of pre-commit after expiration +x blocks (avoid state bloat)

[timebox] Storage upgrades

- How to upgrade storage(partial)

Track progress on proof generation for stored anchors in child generation

paritytech/substrate#3439

In addition we might be able to use light client protocol for extracting proofs?

Create functional test app scaffold

Anchor: Add worker to evict expired pre-commits

As the upgrade to Substrate 2.0 (master branch) is done, we can now add the worker to evict expired pre-commits.

Additional reading:

Discussion with todr - https://centrifuge.hackmd.io/dxSQCYK6TUi8gy4M_76-9Q
Im online module https://github.com/paritytech/substrate/blob/646b54cb0d35b0ec8d921998b259fce6ca77f8be/srml/im-online/src/lib.rs#L406

POA: Abstract governance from POA dependencies

Anchor: refine commit tests (pre-commit + commit)

Implement child tree based anchor buckets and commit extrinsic


// child storage for anchor data represented as a double map
AnchorStorage: Map EvictDate => AnchorID => AnchorData

#### Commit

Anchor data = $AnchorData$
Anchor ID = $AID$
Commit date = $t_{commit}$
User defined evict Date = $t_p$
Calculate User defined anchor period = $p_u$ = $t_p$ - $t_{commit}$
System defined minimum anchor period = $p_s$
Calculate actual anchor period = $p_{actual}$ = {$p_u$ < $p_s$ ? $p_s$ : $p_u$} (properly define case stmt :)
Calculate actual evict date = $t_{evict}$ = $t_{commit}$ + $p_{actual}$

**Store in child storage** -> $set\_child\_storage(t_{evict},  AID, AnchorData)$
**Store evict date index** -> $AnchorEvictDates.put(AID, t_{evict})$
**Calculate anchor index** = $AIDX = CurrentAnchorIndex + 1$
**Store anchor index** -> $AnchorIndexes.put(AIDX, AID)$
**Store current anchor index** -> $CurrentAnchorIndex = AIDX$

Total storage per anchor = $S_{anchor} = 2 * storage(AID) + storage(AnchorData) + storage(AIDX)$
Cost per storage unit per time unit = $c_{unit}$
Calculate anchor storage cost = $c_{storage} = S_{anchor} * c_{unit} * p_{actual}$  

Const gas cost (gas) = $c_{insert}$
Total tx cost = $c_{tx} = c_{storage} + c_{insert}$

complexity = $O(1)$ since trie inserts costs at most length of the anchor id string and its a fixed length hash.

Check Default Validator logic

This is to investigate how the current default validation logic work for cent-chain + substrate

Fees: what's the fee for simple value transfer?

set up spinnaker to redeploy docker image from master build

Setup permanant bootnodes for centrifuge chain fulvous testnet

Currently validator nodes are set to find eachother as bootnodes, which seem to cause some issues. Need to investigate the issue and fix.

cc @pstehlik

Fix docker image build

Currently on docker hub: https://cloud.docker.com/u/centrifugeio/repository/registry-1.docker.io/centrifugeio/centrifuge-chain/builds/1b4993a4-866c-481f-a4d9-01b65400c082

erroring out.
could either move into travis - or keep in dockerhub.
evaluate which one is faster to build.
we'd want to avoid hogging a travis slot for 2hrs and blocking other builds.

Copy tests from GSRPC over to functional test app

Can we build state rent in a more reusable way for cent chain storage items?

this is a future research topic.

Run functional test app on CI against master of Cent Chain

Implement rent calculation for state on anchors

Once we have the implementation for storage of anchors we need to implement the actual rent for state using the fee module.
Total storage per anchor = 𝑆𝑎𝑛𝑐ℎ𝑜𝑟=2∗𝑠𝑡𝑜𝑟𝑎𝑔𝑒(𝐴𝐼𝐷)+𝑠𝑡𝑜𝑟𝑎𝑔𝑒(𝐴𝑛𝑐ℎ𝑜𝑟𝐷𝑎𝑡𝑎)+𝑠𝑡𝑜𝑟𝑎𝑔𝑒(𝐴𝐼𝐷𝑋)
Cost per storage unit per time unit = 𝑐𝑢𝑛𝑖𝑡
Calculate anchor storage cost = 𝑐𝑠𝑡𝑜𝑟𝑎𝑔𝑒=𝑆𝑎𝑛𝑐ℎ𝑜𝑟∗𝑐𝑢𝑛𝑖𝑡∗𝑝𝑎𝑐𝑡𝑢𝑎𝑙

UI: How to approach?

Set up permanent centrifuge chain testnet

Add disk to deployed instance
Use real accounts (not Alice/Bob)
Use "real" genesis file

Migrations are not concern atleast for a while for new features.

Spinnaker pipelines for master depolys

Anchor: verify merkle root for pre-commit/anchoring

Implement purge for pre-commit

Use workers

Go implementation of "accounts" for functional test app

Deploy testnet with "real" genesis

Instead of taking the pre-defined testnet with Alice/bob/... the deployment should have a more realistic genesis file:

generate different accounts than Alice/Bob/..
specific root account with passed in public key for each testnet
setting up the initial authorities for the chain validation

Load test locally and come up with a more concrete number for per anchor storage requirement

[bug] Fulvous testnet error with storage root invalid

When running fulvous, node panics with,

====================

stack backtrace:
   0: substrate_panic_handler::set::{{closure}}::h22f0f1d19b29e46d (0x5570318b0ec0)
   1: std::panicking::rust_panic_with_hook::h057ff03eb4c8000f (0x5570319e9a59)
             at src/libstd/panicking.rs:478
   2: std::panicking::begin_panic::h01337335381120d3 (0x557031539645)
   3: srml_executive::Executive<System,Block,Context,UnsignedValidator,AllModules>::execute_block::h46420a4b51db8b11 (0x55703153d8ac)
   4: centrifuge_chain_runtime::api::dispatch::h63face78c359dd4d (0x557031514a8d)
   5: std::panicking::try::do_call::h62e7477d6f4b75b4 (0x557030f33692)
   6: __rust_maybe_catch_panic (0x5570319eedea)
             at src/libpanic_unwind/lib.rs:85
   7: substrate_executor::native_executor::safe_call::h3cbd82ffde27a0a3 (0x557030cc9728)
   8: std::thread::local::LocalKey<T>::with::h8e36660e2af3b325 (0x557030a53127)
   9: <centrifuge_chain::service::Executor as substrate_executor::native_executor::NativeExecutionDispatch>::dispatch::h0913301dcc5bbd02 (0x557030891792)
  10: std::thread::local::LocalKey<T>::with::hada49c4683f7f063 (0x557030a5be3e)
  11: substrate_state_machine::StateMachine<H,N,B,T,O,Exec>::execute_aux::h4c4976f3f2d5fbc8 (0x557030e0ca66)
  12: substrate_state_machine::StateMachine<H,N,B,T,O,Exec>::execute_call_with_native_else_wasm_strategy::h7a70dacfe7719c17 (0x557030e12122)
  13: <substrate_client::call_executor::LocalCallExecutor<B,E> as substrate_client::call_executor::CallExecutor<Block,substrate_primitives::hasher::blake2::Blake2Hasher>>::call_at_state::hf54a9cd0e8c0c6eb (0x557030d9af96)
  14: <&substrate_client::client::Client<B,E,Block,RA> as substrate_consensus_common::block_import::BlockImport<Block>>::import_block::h3b2cc48f55fe45c5 (0x557030a024b8)
  15: <substrate_finality_grandpa::import::GrandpaBlockImport<B,E,Block,RA,PRA,SC> as substrate_consensus_common::block_import::BlockImport<Block>>::import_block::hfa4176c6ffcdc91b (0x557030f173d6)
  16: <substrate_consensus_babe::BabeBlockImport<B,E,Block,I,RA,PRA> as substrate_consensus_common::block_import::BlockImport<Block>>::import_block::hf4b1416a1336ff4f (0x557030e56cce)
  17: core::task::poll::Poll<T>::map::h2800644b173cd077 (0x557030b0bd38)
  18: <futures_util::try_future::map_ok::MapOk<Fut,F> as core::future::future::Future>::poll::h44e75c53666d861d (0x5570307e100c)
  19: futures_util::try_future::try_chain::TryChain<Fut1,Fut2,Data>::poll::hf94a03e42d347958 (0x557030e3a9ad)
  20: <futures_util::try_stream::try_for_each::TryForEach<St,Fut,F> as core::future::future::Future>::poll::h1e4058bc162f42bf (0x557030b1ca59)
  21: futures_util::future::chain::Chain<Fut1,Fut2,Data>::poll::h7e04c1afbe139a43 (0x5570307eacda)
  22: <futures_util::compat::compat03as01::Compat<Fut> as futures::future::Future>::poll::h8dbc10bf30d58049 (0x557030ef3776)
  23: <futures::future::select::Select<A,B> as futures::future::Future>::poll::h61fa2b5b8f541b6a (0x557030c31b2a)
  24: futures::future::chain::Chain<A,B,C>::poll::h6107eb89c39c9da2 (0x557030ea93ec)
  25: <futures::future::map_err::MapErr<A,F> as futures::future::Future>::poll::hf2217414ed5ab382 (0x557030d2bf3e)
  26: futures::task_impl::std::set::h1d6c7394509821f8 (0x55703173b292)
  27: std::panicking::try::do_call::h4da7ee154caa89f2 (0x55703173adf2)
  28: __rust_maybe_catch_panic (0x5570319eedea)
             at src/libpanic_unwind/lib.rs:85
  29: tokio_threadpool::task::Task::run::h0116a589554a280b (0x5570317384b9)
  30: tokio_threadpool::worker::Worker::run_task::h59c5abb9988e7e20 (0x557031731e55)
  31: tokio_threadpool::worker::Worker::run::h818f5066b41d5b52 (0x55703173132e)
  32: std::thread::local::LocalKey<T>::with::h1fcd9efb210f2dd2 (0x55703171bcde)
  33: std::thread::local::LocalKey<T>::with::h5fbd4053ce8d547c (0x55703171be9f)
  34: std::thread::local::LocalKey<T>::with::h16eaac48bb88b444 (0x55703171bb2b)
  35: tokio::runtime::threadpool::builder::Builder::build::{{closure}}::h5c979caac29cea17 (0x5570317208de)
  36: std::thread::local::LocalKey<T>::with::h01b9617282c3e29b (0x55703173ba63)
  37: std::thread::local::LocalKey<T>::with::h076aff392ca7ff3f (0x55703173bb9e)
  38: std::sys_common::backtrace::__rust_begin_short_backtrace::hd8e94618c1c47325 (0x557031733991)
  39: std::panicking::try::do_call::h9f90deaf148b6ba7 (0x55703173ae3c)
  40: __rust_maybe_catch_panic (0x5570319eedea)
             at src/libpanic_unwind/lib.rs:85
  41: core::ops::function::FnOnce::call_once{{vtable.shim}}::he5b73f6ee3ce78f5 (0x557031734610)
  42: <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once::h805c3cc89d534c05 (0x5570319d71ff)
             at /rustc/a53f9df32fbb0b5f4382caaad8f1a46f36ea887c/src/liballoc/boxed.rs:704
  43: <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once::h9ebda9db5f03daf3 (0x5570319ee100)
             at /rustc/a53f9df32fbb0b5f4382caaad8f1a46f36ea887c/src/liballoc/boxed.rs:704
      std::sys_common::thread::start_thread::ha32838ccbb8a8179
             at src/libstd/sys_common/thread.rs:13
      std::sys::unix::thread::Thread::new::thread_start::h6f10b78f26c98dc6
             at src/libstd/sys/unix/thread.rs:79
  44: start_thread (0x7f52c66166ba)
  45: clone (0x7f52c613641d)
  46: <unknown> (0x0)

Thread 'tokio-runtime-worker-1' panicked at 'Storage root must match that calculated.', /root/.cargo/git/checkouts/substrate-7e08433d4c370a21/4e1d637/srml/executive/src/lib.rs:319

This started happening after implementing child storage anchor state rent.

Set up dynamic test accounts on Cent Chain for functional tests

Centralized way to create Cent Chain test accounts
All the functional tests and apps will require pre-funded test accounts.
Possible solution:

write a small JS script that funds test accounts from Alice (or another genesis account with lots of value)

Implement extend op code for anchors

Exactly like commit except

Anchor data is to be retrieved using GetByAnchorID
Cost calculation should take into account difference between new and last evictedate
(Probably no need to delete the previous storage as it would get evicted)

Deploy to GCP with two validators
Manual deploy OK via spinnaker/kube
tear down once testing is over

Implement off-chain worker for evict

Investigate fee configuration on substrate

Following things needs to be researched/investigated

List of payable transactions in Cent chain
How does setting transaction fee weights(gas) work?
Can we set gas amounts on genesis instead of having it hard coded.
Can we still use fee module to setup fee configurations (hooks?)

cc @pstehlik

Research into state tree design
- Estimate the cost of different items stored. Eg: transactions, anchors
- How can we store anchors in the tree efficiently(indexed buckets vs ttl trie) while allowing for
  - Efficient query by ID
  - Off-chain purge + verifiability after purge(may be with child tree headers)
  - Model long term behavior of each algo (math vs statistical model)
How different actors in the system would benefit/lose out with different designs

https://centrifuge.hackmd.io/3YhROyWbQY68s6dQM6AB1w

Anchor: State rent

Implement Evict

Evict
On a certain evict date = 𝑇𝑒𝑣𝑖𝑐𝑡
Get the root of the evicted child tree = 𝑅𝑒𝑣𝑖𝑐𝑡𝑒𝑑=𝑐ℎ𝑖𝑙𝑑_𝑠𝑡𝑜𝑟𝑎𝑔𝑒_𝑟𝑜𝑜𝑡(𝑇𝑒𝑣𝑖𝑐𝑡)
Delete all anchors for the date -> 𝑐𝑙𝑒𝑎𝑟_𝑐ℎ𝑖𝑙𝑑_𝑠𝑡𝑜𝑟𝑎𝑔𝑒(𝑇𝑒𝑣𝑖𝑐𝑡)
Set the root for the evict date -> 𝐸𝑣𝑖𝑐𝑡𝑑𝐶ℎ𝑖𝑙𝑑𝑆𝑡𝑜𝑟𝑎𝑔𝑒𝑅𝑜𝑜𝑡𝑠.𝑝𝑢𝑡(𝑇𝑒𝑣𝑖𝑐𝑡,𝑅𝑒𝑣𝑖𝑐𝑡𝑒𝑑)

Delete indexes:
𝑖=𝐿𝑎𝑡𝑒𝑠𝑡𝐸𝑣𝑖𝑐𝑡𝐼𝑛𝑑𝑒𝑥=>𝑖𝑑=𝐴𝑛𝑐ℎ𝑜𝑟𝐼𝑛𝑑𝑒𝑥𝑒𝑠.𝑔𝑒𝑡(𝑖)=>𝐴𝑛𝑐ℎ𝑜𝑟𝐸𝑣𝑖𝑐𝑡𝐷𝑎𝑡𝑒𝑠.𝑟𝑒𝑚𝑜𝑣𝑒(𝑖𝑑)=>𝐴𝑛𝑐ℎ𝑜𝑟𝐼𝑛𝑑𝑒𝑥𝑒𝑠.𝑟𝑒𝑚𝑜𝑣𝑒(𝑖)

until a evict date is found that is more than 𝑡𝑒𝑣𝑖𝑐𝑡

set 𝐿𝑎𝑡𝑒𝑠𝑡𝐸𝑣𝑖𝑐𝑡𝐼𝑛𝑑𝑒𝑥 to last removed index