osmosis-labs / beaker Goto Github PK

Currently, beaker only has the basic counter example. I believe there is a need to expand scaffolded-dApp variety to encourage more builders to build via a bit more structure. These additional examples can help demonstrate tooling (such as various requests for custom cosmwasm bindings) that you can integrate with beaker.

Current list

• tokenfactory (already done, just needs to make beaker compatible)
• Escrow test contract
• CW20 Pot

For the signer-account option I think having a few predefined ones works well for localosmosis but for testnet/mainnet it would be best to use:

• use system keyring so that I can use my own keys
• option to specify the keyring backend

https://docs.cosmos.network/master/run-node/keyring.html

I guess we could call them:
system-signer-account && beaker-signer-account?

This way we can allow people to request testnet tokens via the faucet for their own addresses, and eventually add an option to the faucet to approve contracts as well as we discussed yesterday.

• Use https://github.com/rust-cli/clap-md as a reference to create task that generates markdown documentation from clap app.
• in README.md refer to the docs

This issue is for tracking the clarification of the path to production (mainnet) for CosmWasm. Since Osmosis implements permissioned CosmWasm, See what are the steps to take, who is involved in order to have clear understanding on what Beaker should support, who else needs to be involved, what system needs to eventually be upgraded and later we can have a clear documentation for the dapps developer.

The following is my attempt to flesh out the understandings along with ❓ questions and 💡 ideas.

Grouping for wasm gov proposal types

According to the wasm gov docs, there are additional proposal types for wasm gov. Grouping by their functions for further reference:

• Deployment and Migration (**main focus for this issue)

  • StoreCodeProposal
  • InstantiateContractProposal
  • MigrateContractProposal

• Code Permission

  • UpdateAdminProposal
  • ClearAdminProposal
  • UpdateInstantiateConfigProposal

• Code Pinning

  • PinCodes
  • UnpinCodes

• Execution

  • SudoContractProposal
  • ExecuteContractProposal

[❓ 1] on mainnet, which of the above are required to be execute through governance?
📝 Answer 1 by @czarcas7ic

[💡 1] Same wasm gov configuration as mainnet for testnet (stagenet?) and/or localosmosis variation would be ideal for testing. @czarcas7ic @nicolaslara

For Code Pinning and Execution, if the [❓ 1] is clear, it doesn't seem to be important for this issue. Let's move on to the main topic.

Deployment and migration

StoreCode requirements

There was a discussion with @ValarDragon @daniel-farina about hash verification and contract-list

• #56

This is what I imagine would happen if there is a need to store code:

graph TD
    StoreCode[Submit `StoreCodeProposal`]
    SubmitRepo[Submit source and metadata]
    Val{validate}
    Voting[Gov Voting]

    StoreCode == wasm byte code ==> Val
    SubmitRepo == source code & build metadata ==> Val
    Val -->|passed| Voting

[❓ 2] should we require hash validation before triggering the voting period of the proposal? if so it sounds like a chain upgrade, what else could be an alternative? @ValarDragon

📝 Answer 2 by @daniel-farina

The StoreCodeProposal already contains wasm byte code, one can get that by querying the proposal.
For Submit source and metadata this would trigger a question:

[❓ 3] In what format should we store the source reference & build metadata (for deterministic build)?
📝 Answer 3 by @daniel-farina

[💡 3] Specific & special region in the proposal description for source reference & build metadata

[❓ 4] Do we require public git repository? or just upload an archive somewhere?
📝 Answer 4 by @daniel-farina

[❓ 5] Where does contract list update sits in the process?
📝 Answer 5 by @daniel-farina

Deployment

This is normally a 2 steps process of StoreCode -> InitiateContract, but common pattern of a dapp has the following:

1. StoreCode -> InitiateContract tends to happen sequentially.
2. A dapp can have more than 1 code/contract and could be depending on each other

The following graph is a general execution plan for a deploying a dapp.

[💡 6] Note that this structure is dependency graph so the process as code @daniel-farina mentioned in #48 could incorporate the whole flow like what terraform does.

[❓ 7] Is it possible / how hard it is to have multiple related proposals (like in the process above) to go through the gov together as bundled proposal? Does it make sense to do so? I can imagine how this could simplify the flow... @ValarDragon @sunnya97

Migration

The following graph is a general execution plan for a migrating multiple contract for a dapp. It shares the same question as for the deployment.

Lastly:

[❓ 8] Will we use https://commonwealth.im/osmosis/ for wasm governance?
📝 Answer 8 by @daniel-farina

Let me know your thoughts.

Todo

• try osimosisd to go through the process
• start with gov store-code

  • reconstruct this msg in rust https://github.com/CosmWasm/wasmd/blob/ac92fdcf37388cc8dc24535f301f64395f8fb3da/proto/cosmwasm/wasm/v1/proposal.proto and try rpc-ing it

  • make gov proposal works

  • refactor types -> gen types -> get path

  • expose proposal related stuffs to the interface

  • check mainnet config with command provided by Dev & see plan about gov

  • [BLOCKED] try editing genesis.json remove permission to test the process (with for nico / adam response)

  • fix state file: non has config for local and shared network

    • serialize config for state variant
    • get state variant from context (check by network)
    • update state file based on state variant (pass as arg)

  • PropsoalStoreCodeResponse

  • make deposit works (for today: 7 Jun)

  • check deposit

  • extract module

  • update state for proposal

    • make code_id optional
    • add proposal_id
    • add update_proposal_id function

  • find the hash function for code_hash -> sha256sum

  • create query for proposal

  • refactor: extract modules

  • move ops into their own module

  • beaker wasm proposal vote store-code counter --amount 10000uosmo {{base_tx}}

    • create proposal/ops/vote.rs
    • copy code from store code
    • get the right message in, make sure code compiles
      • get_ref . proposal_store_code_id to query
    • update entry point with respective stuffs needed

  • beaker wasm proposal store-code sync-state

    • update_in! for handling nested struct (see [1], [2])
    • new struct for proposal

// ultimately
struct Code {
  id: u64,
  instances: Map<String, String>
}

struct Proposal {
   store_code: Option<u64>
}

struct WasmRef {
    code: Option<Code>
    proposal: Proposal,
}

• build ts
• try import built artefacts into console.js
• start extracting lib type logic out
  • identify with comments
  • extracts

• expose network flag, change chain_id flag to network flag
• make client configuration base on network name
• [-] change everything to grpc instead of rpc
• test it on testnet (pending public testnet)
• update non-local state instead of local state

Endpoints

Testnet

• rpc-test.osmosis.zone
• grpc-test.osmosis.zone:9090

Mainnet

• rpc.osmosis.zone
• grpc.osmosis.zone:9090

When I made my beaker project, I was quite curious as to what the .beaker folder was for.

A short readme inside of it would be good to help the curious, who are trying out beaker!

When store code, the code stored can have permission of who could instantiate the contract. The permission can be put in the StoreCodeMsg.

This issue is for tracking the implementation and pose the question of it's priority which depends on the discussion on #59
edit: since instantiation is permissionless, this is now important..

• contract with default label with props:
  • address
  • query
  • execute().by(account)
• contract with other label

await contract.counter.execute({ increase: {}}).by(account.test1)
await contract.counter.query({ get_count: {} })

await contract.counter.$awesome.execute({ increase: {}}).by(account.test1)
await contract.counter.$awesome.query({ get_count: {} })

Cycle 4

Backlog

• #1
• #30
• #23

• impl --no-optimize
  • build: --optimize -> --no-optimize
  • store-code: add --no-optimize to store non-release target, faster build
  • deploy: add --no-optimize
• optimizer
  • Extract workspace optimizer version to config
• Instantiate
  • add funds
• use non-local cosmos-proto

Sub-issues

• #8
• #9
• #10
• #11
• #15
• #12

The idea here is to get the config struct documentation at runtime to generate md file that has details

• derive_get_docs
  • get docs from simple struct
  • move test to proper place > example
  • nested struct must be compatible with prev get docs
  • get docs from nested struct
• refactor md gen into its own package so that md package only focus on generating md and use cli and get_docs as deps
• gen README.md
• treat doc strings as cmark structure
• add type information
  • in general
  • take care of optional
  • take care of map
• Brush up description
• rename fn get_docs -> get_data_docs;
• rename crate derive_get_doc -> data_doc_derive, get_docs -> data_doc
• draft the config doc structure
• generate config doc

struct Docs(Vec<Doc>);

struct DataDoc {
  ident: String,
  desc: Vec<String>,
  sub_docs: Vec<DataDoc>
};

trait GetDataDocs {
  get_data_docs() -> Vec<DataDoc>;
}

Deploy

The deployment process = code => [build, optimize, store-code, instantiate], which can have multiple instantiation.
Assumption is, if they use this method, they will just use the same signer...

$ beaker deploy counter --signing-account test1 
$ beaker deploy counter --signing-account test1 
$ beaker deploy counter --signing-account test1 --no-rebuild

args:

contract_name,
raw
chain_id,
signer_args,
gas_args,
timeout_height,
label

Assumptions

• shared gas params (2 txs, so actual gas limit is doubled) make it better in #14
• shared signer params
• chain_id (change chain_id to network once working on #15 )

Todo

• use instantiate args & copy instanitate code
• add store-code logic before instanitate
• make building and optimizing default, add it to the logic before all
• add flag --no-rebuild and impl the logic

• serialize config as json and pass it into console args
• create wallets from mnemonics
  • args: menmonics, prefix, hdpath
• create signing client
  • args: rpcEndpoint, gasPrice
• Object.assign(globol, account) option with renaming to avoid scope polution

await contract.counter.increase().by(account.test1)
await contract.counter.getCount()

with global

await counter.increase().by(test1)
await counter.getCount()

interactive console has the following purposes:

• as oppose to writing integration test, which has the role to automate regression test, this provides a quick and straightforward way to understand how smart contract or set of smart contracts behave and inspect them.
  • when changing smart contract code, from code changes to updated behavior when interact with the console should flow smoothly
  • inspecting smart contract state should be simple, no need custom query command
  • client for the contract must be generated and if using beaker to instantiate, client for each contract instance must be provided, accessible via label

TODO

• Instantiate repl
  • find good repl solution: (syntax highlighting, object interaction, multiline editing)
    • default node repl (must)
    • https://github.com/gzuidhof/starboard-notebook/tree/master/packages/starboard-notebook (future)
    • https://observablehq.com/ (future)
  • making .beaker an npm package
  • create beaker npm module
  • add console integration for chosen solution
  • call from rust

• Store code
• Rename cw to wasm
• Save code id in the state file (only local one for now, leave the non-local to #12 )
  • create support::state module
    • define struct
    • load (use it to load file relative to root $WORKSPACE_ROOT/.membrane/state.local.json
    • modify
    • save

Defaulted to local network ( = --network local but not implemented at the moment)

$ membrane wasm store-code counter

|-- .membrane
|    |-- state.json
|    |-- state.local.json // if chain-id is defined to be local (localosmosis)

use latest only until the need arise...

{
  "<chain-id>": {
    "counter": {
      "code-id": 1,
      "address": {
        "default": "..",
      }
    }
  }
}

state file utils:

Not in scope

• #13
• Support testnet & mainnet store code #12

• templatize frontend
• symlink on generate
• fix .beaker: remove package
• set package version (-rc)
• publish on cargo
• npm (waiting for permission)
• test out fresh (potentially on gitpod)
• write getting started guide

Writing getting started guide to guide through the process user needs to go through.

This serves 2 purpose:

1. as the name suggest; documentation for getting started with the tool
2. design space to think in terms of the user how to use the tool

Mark work in progress section as WIP and link to the issue.

Definition of done

Narrate through the process of

• scaffolding project workspace
• create new contract in the workspace
• contract deployment process
• interactive console
• configure target to testnet / mainnet
• frontend integration

Changes are expected but this must serves as a skeleton for further modification

tips

beaker[local] ◇ contract.counter.query({ get_count: {} })
Promise {
  <pending>,
  [Symbol(async_id_symbol)]: 396,
  [Symbol(trigger_async_id_symbol)]: 97
}
beaker[local] ◇ await _
{ count: 101 }

Submitting a proposal is currently done this way:

beaker wasm proposal store-code --title "Testnet Contract" --description "Testnet Contract" --signer-account test1 --network testnet counter --gas 25000000uosmo --gas-limit 25000000

• We should be able hardcode maximum gas (In mainnet this won't be more than 1 OSMO)

I think we could simplify this by making a yaml file instead or json. This will be really useful as the description can be quite long.

beaker wasm proposal store-code --proposal proposal_407.yaml counter

Example proposal.yml idea

title: Proposal to allow DappName to be enabled in Osmosis
description: |
            A lengthy proposal description
            goes here  
            we expect this to be many lines...
code:
    github:   https://github.com/osmosis-labs/beaker/
    reviewed_by: https://....
settings: 
    gas:  25000000uosmo 
    network: testnet

Notes

• create struct that has all proposal options
• make the struct subcommand compatible and serde serializable
• allow serialization for toml, yaml
• put the code part as a metadata structure in the proposal description

• fix cw-template type bug (test by cargo generate with local git)
• beaker wasm ts-gen <contract_name>: leverage https://www.npmjs.com/package/cosmwasm-typescript-gen
  • call cargo schema
  • cosmwasm-typescript-gen generate \ --schema ./path/to/schema \ --out ./path/to/ts/sdk \ --name MyContractName
  • ts/sdk (> {{project-name}}-sdk}}) with ts boilerplate stuffs
• options
  • cargo schema schema gen command
  • schema outout dir
  • client gen output dir
• refactor
  • to ops
  • extract npx
• upgrade console
  • link beaker-console
  • specify beaker-console version on npx
• regen docs

await account.test1.sign(contract.counter).increase()
await contract.counter.getCount()

with global

await counter.with(test1).increase()
await counter.getCount()

• localosmosis subcommand
  • start
  • clean
  • update
  • setup-keplr
• config:
  • path to store localosmosis code

We need to define the upgrade process for a contract. What are the requirements and steps to accomplish this? We should do this via CLI first.

This is currently blocked by #58 [it's closed, no longer blocked]

• #42
• #43
• #44

• Detects Keplrs
• Add localosmosis chain’s config (chain-id, local endpoint)
• Load one of the localOsmosis wallets (test1 account) (manually for now)
• Keplr connects and gets the balance no problem

Nice to have adding to backlog

• Automatically add wallets to Keplr

Currently a project generated from beaker new counter-dapp has a blank gitignore, this is one step we should automate out!

• use https://github.com/github/gitignore/blob/main/Rust.gitignore as base
• add artifacts (for wasm artifacts)
• add **/state.local.json

This should be done once #56 is complete. Good item for the backlog since we can do this via CLI first.

Is there a reason why the CI generated for contracts right now is circleCI vs github actions? (e.g. speed?)

Would be nice to have the normal flow "just work" when put on github, whereas circleCI requires some extra configuring.

Hitting this error when using mainnet (an account with no funds on mainnet) test1.

beaker wasm proposal store-code --title "Mainner Contract" --description "Official Contract for" --signer-account test1 --network mainnet counter --gas 100000uosmo --gas-limit 100000

The expected result should be something about not having enough funds.
Output:

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: serde parse error

Caused by:
    expected value at line 1 column 1

Location:
    /Users/web/.cargo/registry/src/github.com-1ecc6299db9ec823/flex-error-0.4.4/src/tracer_impl/eyre.rs:10:9

Caused by:
   0: serde parse error
   1: expected value at line 1 column 1

Location:
    /Users/web/.cargo/registry/src/github.com-1ecc6299db9ec823/cosmrs-0.7.1/src/tx/raw.rs:34:12', /Users/web/.cargo/registry/src/github.com-1ecc6299db9ec823/beaker-0.0.4/src/support/cosmos.rs:246:77
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

refs:

• https://github.com/cosmos/cosmjs/blob/main/packages/stargate/src/fee.ts

• impl to_fee

• impl From for Gas

• add Option<gas_config>: gas_price, gas_adjustment

• change Fee::from(gas_args) to Gas::from(gas_args)

• change type signature and name in sign_and_broadcast

• fix type errors

• test out

Improvements

• #34
• #47
• #67
• #39

Contract Upgrades

• #38
• #58

Contract Verification

• #56
• #57

Open Questions - Research

• #59
  • #48
  • #49
  • #53
  • #46

DevOps

• #35

Inspecting contract

First, create new contract

https://docs.osmosis.zone/developing/dapps/get_started/cosmwasm-testnet.html

Once the contract is created we can check query the contract information. Assuming you are connected to the testnet.

contract_info

osmosisd query wasm contract osmo1mpf0guu0t363xrshhedandypq003ahzaxvsxzgu69n3ej03mh2zqx5gk8l

Output:

contract version

osmosisd query wasm contract-state raw osmo1mpf0guu0t363xrshhedandypq003ahzaxvsxzgu69n3ej03mh2zqx5gk8l 636F6E74726163745F696E666F --node https://rpc-test.osmosis.zone:443 --output json | jq  -r .data | base64 -d | jq

What in the world is 636F6E74726163745F696E666F? 😕

ContractInfo is must be stored under "contract_info" key which translates to "636F6E74726163745F696E666F" in hex format. As documented here.

Output:

Downloading the Contract from the network

osmosisd query wasm code 205 205_code.wasm --node https://rpc-test.osmosis.zone:443

Output:

Getting the hash

sha256sum 205_code.wasm

Output :

0b662785042cd938d0f0e8142f69d2d23fdf0addec965d1fbeacf44080330016

Verify hash

Now it's time to verify the hash with the source repos. All contracts should provide a repository with a hash for their contracts the same way that cw-plus does on their checksum as shown below:

10bc1a8cf82055106b26136f26137d6bf132df99c2d2098a75d03b64e7fe75a5  cw1155_base.wasm
539b62678532596c73b27764186623a2bb868f5e67c3e588508c4a28e105e8c9  cw1_subkeys.wasm
b67bc69fef770c28d48730feb800ea9c91eaae1a608e3ea7819aac64a6e99d92  cw1_whitelist.wasm
704890409d0e82f33951f0f4d06e42b27082d91b5c8c98853969296191244baf  cw1_whitelist_ng.wasm
db366741dcbad5f2e4933cda49133cd2a11fdb32b08c67cb1d22379bd392448e  cw20_base.wasm
ac452ac5c96ac843cad0267a0f465d39f6dd43d4e47631e6deefd23a4cf77a61  cw20_ics20.wasm
b3d44429dfb1e445a7ea3c0125b71efdc85185666f35e4e62581ffc1310905bd  cw3_fixed_multisig.wasm
62131b508b448dad7d7878188f55b674c24a1b705791179b87e85e0854538837  cw3_flex_multisig.wasm
163566b6f65819c2ea6cadddc847cbaaa59f5218dbbd895bcc2fcefe1afd7d0e  cw4_group.wasm
0c44f17035eb51c0115d95f14e83ce2cdfbcb804ffa95e1f8203d22300d6ba90  cw4_stake.wasm

• item 1
• item 2
• item 3

test

In docs.osmosis.zone, we note that users need to run

rustup target add wasm32-unknown-unknown
cargo install cargo-generate --features vendored-openssl
cargo install cargo-run-script

It'd be cool if any beaker command could set these up for you, if its not already handled. (Reduces the amount of installation steps folks have to do)

If deposit data is included in the YAML proposal file then execute the deposit command else if empty allow user to do the deposit as a secondary step.

pre:

• refactor to typescript

$ membrane wasm instantiate counter --raw '{ "counter": 0 }' # imply --label default
$ membrane wasm instantiate counter --raw '{ "counter": 0 }' --label "sth"

# won't work with --code-id
$ membrane wasm instantiate counter # use counter/instantiate-msgs/default.json
$ membrane wasm instantiate counter one # use counter/instantiate-msgs/one.json, imply --name "one"

• instantiate with raw string
  • without name
    • test request (let's try cosmwasm js)
    • save address
    • check if same label can use twice or not?
    • return instantiate response
    • refactor
      • shared arguments
        • create SignerArg
        • implement SignerArg method
        • create GasArg
        • Implement GasArg method
        • compute fee logic before passing into ops (-> fix too many args)
        • create coin.rs
        • parse denom from gas input
        • remove denom global config
      • add gas input
      • shared logic
      • format result
        • Event formatter (create support/format)
        • marcro for formatter
        • Add important info, implications to result: address, code_id, label, creator, admin, (ibc_port_id could be separated part, IBC support)
  • with label
• instantiate with file
  • default.json
  • {label}.json
• fill all the mnemonics

I believe we need beaker wasm instantiate to support admin since it's required during the contract upgrade process.

This was pointed out in the source code here:

Just calling it out as a blocker for mainnet deployments as contracts would become immutable without an admin.

Purpose:
when in console, if you want to keep the state and update stuffs without leaving console, you can hit .redeploy, it will just rebuild, store-code, instantiate everything that exists

• option for global account & contract
• .redeploy

.deploy counter test1