Provide better documentation regarding how to maintain the operational stability of a Substrate-based chain. Topics may include: launching and maintaining operators, sentries, bootnodes, RPC pools and more.

Ref: https://wiki.polkadot.network/docs/en/maintain-index

As the chain extension feature has been implemented and proved to work well, maybe we could add a section on introducing how to use it?

ref: use-ink/ink#645

So some background. I have only started with Rust at the beginning of this month and I got interested in blockchains last Sunday. Therefore that might explain why I had difficulty following the instructions on the installation steps page.

Now the problems I had are more related to Rustup but regardless of this fact, the documentation might need some reworking in my opinion.

There is a section related to installing Rustup where we are told to run:

source ~/.cargo/env

I didn't install Rustup via the shell script but instead used the Arch community package and as result, I didn't have the env file.

However, it didn't seem to matter really as I was eventually able to go through the creation of Substrate chain tutorial without any issues. I agree that yes it's my fault for not following the instructions to the letter but maybe this issue can be highlighted for people who installed Rustup the way I did.

While the previous issue was my fault, the section about the toolchain really slowed me down as I wasn't sure what to do because, Polkadot was mentioned while I was interested in Substrate. I simply ploughed ahead and installed the latest nightly toolchain and I had no problem going through the above referenced tutorial. This might well be due to my lack of understanding about the whole Rust ecosystem but I recall reading that knowledge of Rust would not be required for that tutorial.

Now, I would be happy to help rewrite that part of the documentation as long as it is deemed to be necessary/desirable and also if I can get some degree of support either from here or in Matrix.

Just my 2 cents as they say...

Dan Shields:

In .mdx is it possible to enable the default x syntax actually mean target="_blank" (in tags as I am used to)

If so, is there a way to override this somehow? Specifically for internal links this would make sense (in page for sure, and within the docs in most cases)

Imad:

This was pointed out to be a while back, and generally speaking a good convention to follow.
That being said, I'm was thinking about making a component available in MDX, so in case there has to be an external link, it'll be styled a little differently. This is so users can easily differentiate visually between and external link and an internal link.

quick example would be:

<ExternalLink to="https://www.someexternalwebsite.come">BUTTON TEXT</ExternalLink>

Dan Shields:

I think this reason is the one I am after they think is "good" - but also most of these links are meant to add reference material while you read, so navigating away and back isn't intended.

We should include convention here in the style guide for contributors to if we have this external or whatever syntax to use letting people know when to and not to use it (@sacha-l @jimmychu0807 what do you think?)

Sacha:

I think that this could be a neat feature we introduce to our Devhub. I would bring it one step further, where each link type (knowledge base, Rust docs, HTGs, Tutorials and other) has a component that tells the reader where that link would take them. We have so many links and its hard to know where each will lead to.

Staying a little more on topic here, I don't think that all links need to open a new tab. IMO those that should open a new tab would be:

Rust docs (crates.io)
any other link that's going to bring someone outside of our devhub.
This is just one opinion, we should make a poll in our community and internal channels and also decide whether this is a feature we want for this first release.

substrate-developer-hub/substrate-node-template#196

Presently there is no way to add block producers to an active chain (that I am aware of) - you need to add this to your node to allow for it. You can add this functionality: here is a (presently out of date) example of a pallet to expose this via an extrinsic call: https://github.com/gautamdhameja/substrate-validator-set

Picking up this pallet and maintaining it as part of the devhub might be required, or some more minimal version of it.

https://substrate.dev/docs/en/knowledgebase/learn-substrate/account-abstractions has a reasonable overview of one type of account abstraction for stash & controller key use. I think we could expand this page a fair amount, and/or refactor to give a more broad overview with references to the polkadot wiki and other sources for deeper dives.

Some references I would point to in either case:

• https://wiki.polkadot.network/docs/en/learn-accounts
  • wiki on polkadot specific accounts.
• https://wiki.polkadot.network/docs/en/learn-keys
  • wiki on use of keys, and stash & controller keys specifically for NPoS (most related to present content on our account abstraction page)
• https://wiki.polkadot.network/docs/en/learn-proxies
  • wiki on proxies
• https://www.parity.io/building-a-hot-wallet-with-substrate-primitives/
  • a great conceptual overview of how multisig, proxies, and more can be used to construct complex and nuanced account abstractions.
  • abstractions mentions: multi-signature accounts, proxy accounts, and derivative accounts
• **Open to more!! 😁 **

One caveat here - IMHO there is no pressing need for this nuanced information for those building runtimes as much as those building applications that may want to have complicated account schemes. So it may be of interest to get this kind of content more visible to that demographic than substrate devs 🤔

What are the thoughts from the team on this?

raised by @athei in Element

We found there's a page about Substrate data migration:

• https://docs.substrate.io/build/upgrade-the-runtime/#storage-migration
• https://github.com/apopiak/substrate-migrations
• https://hackmd.io/@apopiak/Hyhroh0Xv/edit

However it's more like an introduction, with a bunch of example PR and some caveats, but not a tutorial. Specifically, it would be helpful for readers if we can have:

• A step-to-step instructions for how to write a migration, and tear down the related code after finished (the later point is especially useful because it's not mentioned in any of the above docs)
• As said in the migration guide, new migration should support pallet semantic versioning. How can we leverage it?
• The lifecycle of a migration (what will happen once we start a runtime upgrade, etc)
• Any performance consideration? (Looks like a migration can be much heavier than a regular block and 6s is not a limitation?)
• Reference of the related traits

Background: We are trying to migrating a StorageMap to a Child Tire

• Phala-Network/phala-blockchain#107

Hello,

I was wondering if there was a guide available for setting up finality-grandpa-warp-sync with substrate-node-template.

Any suggestions or tips would be greatly appreciated :-)

For Message components in particular, it's nice to be able to use code tags to highlight code snippets.

Update: there is a workaround this currently documented in #4.

Create a plugin to have a sidepanel codeview in some how-to guides.

Per substrate-developer-hub/substrate-node-template#160 - there is a need to explain the new template telemetry additions, knowledge base and/or tutorial and/or recipe would be important. IMHO a tutorial on telemetry in general, coupling with the Node visualizations may be good?

Work w. @imadarai on getting link checkers in-place in CI process

[WIP]
This is an issue to keep track of the items we want to address in our updated contribution guidelines for the new devhub launch.

Components

List all different components: Tip, Advice, Note, Warning, Important
Convention and examples of which instances to use them in
Docs, Tutorials, How-to guides

Full-stops after bullet-points. Convention tbd
TODO: Add link to the different guidelines (htgs, kb docs) here.

Migrating substrate-developer-hub/knowledgebase#56

Related:
w3f/PSPs#6
w3f/polkadot-spec#306

Each top level "Overview" page for tutorials uses the AccentButton after the TutorialObjective components.

It would be nice to remove the generic "Next" button for just those top level pages as to avoid having two buttons serving the same purpose. Maybe by adding some frontmatter tag?

Use components to organize content in a more uniform way across all tutorials.

• Next steps: format "Next steps" in consistent way, using components to structure content. Perhaps RelatedMaterial block
• Add new content to use TutorialObjective component in every Tutorial Overview page

(from Jimmy's original issue)

If I want to scroll down on the left sidebar, I have to scroll the main content all the way down so the left sidebar will scroll downward.

Will be better if I can scroll on the left sidebar independent of where I am at in the main content. Possibly apply to the right sidebar as well if it is longer than the screen height.

Anywhere you don't want the bottom navigation to show up, just add the hideNav: true flag to the frontmatter.

Given how mistakes in deployed smart contracts are difficult to correct, would it make sense to encourage those following the tutorial to run the tests first to see them fail before then implementing the code to make them pass?

I was just trying that in https://substrate.dev/substrate-contracts-workshop/#/1/storing-a-mapping and was interested to review the following failures:

❯ cargo +nightly test
   Compiling incrementer v0.1.0 (/Users/samueljoseph/Documents/Github/tansaku/incrementer)
    Finished test [unoptimized + debuginfo] target(s) in 1.01s
     Running target/debug/deps/incrementer-d10a7711a3840851

running 2 tests
test incrementer::tests::it_works ... ok
test incrementer::tests::default_works ... ok

test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

❯ cargo +nightly test
   Compiling incrementer v0.1.0 (/Users/samueljoseph/Documents/Github/tansaku/incrementer)
error[E0433]: failed to resolve: use of undeclared crate or module `ink`
  --> lib.rs:61:11
   |
61 |         #[ink::test]
   |           ^^^ use of undeclared crate or module `ink`

error[E0599]: no method named `get_mins` found for struct `incrementer::Incrementer` in the current scope
  --> lib.rs:65:33
   |
9  |     pub struct Incrementer {
   |     ---------------------- method `get_mins` not found for this
...
65 |             assert_eq!(contract.get_mins(), 0)
   |                                 ^^^^^^^^ method not found in `incrementer::Incrementer`

error: aborting due to 2 previous errors

Going forward I'm going to use this approach and see how I can remove errors and fix tests as I implement the code.

BTW I love the approach of showing examples and then challenging the learner to implement their own code - great fun! And great learning!

https://github.com/substrate-developer-hub/substrate-developer-hub.github.io/blob/source/docs/knowledgebase/runtime/frame.md

this lists a large selection of pallets substrate comes with. I would think it be better places in
https://github.com/substrate-developer-hub/substrate-developer-hub.github.io/blob/source/docs/knowledgebase/runtime/pallets.md I would also call it "official pallets" and add a section on known community ones (awesome substrate link?)

I would opt for pointing to the whole selection in source, but having these listed for seach-ability on devhub is great.

This likely needs to be revisited every release or substrate and/or FRAME.

Examples:

• https://github.com/chevdor/srtool-actions/blob/master/action.yml
• https://github.com/paritytech/cumulus/blob/master/.github/workflows/srtool.yml

see substrate-developer-hub/substrate-developer-hub.github.io#1033 and substrate-developer-hub/substrate-developer-hub.github.io#993 for details

Based on this original OCW recipe, this how-to guide steps developers through the process of constructing HTTP request for OCWs including:

• sending a request to a Github API remote endpoint
• using the serde library to parse a JSON string from a HTTP response into a data structure
• manipulating that data structure in the node's runtime

Tasks:

• Decide domain to use: https://dev.substrate.io or https://developers.substrate.io?
  Decide to be https://docs.substrate.io

• Figure out who is managing the URL redirection of https://substrate.dev/rustdocs and who is managing the substrate.io link redirection.

The usage of FunctionOf here is out-of-date https://substrate.dev/docs/en/knowledgebase/runtime/fees#dynamic-weights. Although it does not appear anything else on this page is out-of-date, the entire page should be reviewed as a part of addressing this bug.

• use compact wasm for parachain upgrades
• Relaychain upgrade?
• (more?)

Note added about runtime upgrades for rococo/relay chain needs to be added:
For now you shouldn't expect upgrades to function. (needs to be tested!)

• not like a solo chain. You break parachains if you upgrade your relaychain.
• as your wasm blob for you parachain depends on the relay chain's runtime
• collator sends a PoV, but the runtime will be looking for the wrong PoV, so blocks are denied
• setcode will fail on relaychain
• for now unclear, but work is being done to fix this

This topic was previously covered well in devhub both in the cryptokitties tutorial and in the RC5 and earlier recipes. Now that content should be moved into a tutorial.

A how-to guide stepping through the basic configuration of an off-chain worker. This includes:

• configuring the runtime for an OCW
• basic off-chain worker logic
• sending transactions back on chain (signed and unsigned with or without signed payloads)

Category: TBD

(Based on conversation - putting this here to keep track of progress)

See PR #26 with the first iteration of the new search.

Here are some key questions to ponder over:

what are some search metrics that MUST return certain results?
what devs are searching for on DevHub?
Does this provide them with what they are looking for?
Are the intended results in the top 5 or 10?
We are using Lunr Search and Indexing under the hood, and we can tweak it to the serve us best. Feel free to read this guide and chime in on search features that can help us enhance this feature further.

how can i use xcmp in a smart contract
like this https://wiki.polkadot.network/docs/learn-crosschain#high-level-xcmp said

We are soon going to be in a place where we will need to start bringing in the Chinese documentation (either from Crowdin) or from Lion Bridge into the documentation folders.

It's common for new users to setup a remote node for testing the 101 materials and gets stuck needing the unsafe RPC and WS flags, as well as disabling mDNS and possibly other suggestions

I propose that we have a reference document (possible a how-to guide?) that we can point builders to that has a FAQ for unsafe remote nodes for dev/testing ONLY and points to further production resources to have nodes hosted with best practices (that we don't maintain on firewalls, rate limiting, rev, proxies, etc.)

Examples:
might need -- --rpc-cors=all
--ws-external
--rpc-external

related topic of remote development: https://github.com/paritytech/subport/discussions/222

This tutorial: https://substrate.dev/docs/en/tutorials/upgrade-a-chain/sudo-upgrade

Uses Node Template + Polkadot JS

But doesnt ask the user to update their types, thus they run into this error: https://polkadot.js.org/docs/api/FAQ/#the-node-returns-a-could-not-convert-error-on-send

The insert sub-command info on the subkey page is commented out in substrate-developer-hub/substrate-developer-hub.github.io#992 , as it is not included anymore. This should be migrated to a new page dedicated to the key command, as it is now diverging from subkey.

• Add information about differences, and when to use which
• insert sub-command migrated
• Add content that informs why this BEEFY key format exists (see PR for details)

• Migrate MD docs
• port over assets needed (files, images)
• archive workshop repo

Give the ability for us to customize the title in the FastTrackPlayground component.

For e.g. when using this component for X we'll want to title it:
"Try your X in just one click".
And there may be an instance where we'll want to use this same component for Y.

Add in Message and Docs components in KB articles where relevant.

All Ink workshop content ported over.

Need to coordinate with maintainers and rework the workshop a little to make it FRAME v2.

It would be useful to have an article e.g. in the knowledge base that explains the fundamental behavior of the transaction pool.
What's the life cycle of a transaction? Which transactions are propagated?

One thing included there would be that transactions that return an error here in validate_transaction are dropped and not propagated.

Keywords and related material in How-to guides and Knowledgebase.

Based on this OCW recipe, this guide aims to step developers through:

• defining a persistent storage value
• defining a storage lock that sets the locking time limit by amount of time passed in blocks
• use the lock to perform an external process and write data back to storage
• use the offchain_index API to write to the offchain database (Note: this may be its own How-to guide)

Other ressources:

• https://substrate.dev/rustdocs/v3.0.0/sp_runtime/offchain/storage/struct.StorageValueRef.html
• https://github.com/paritytech/substrate/tree/master/frame/example-offchain-worker

Add a definition of block height/number to the glossary. cc @spencerbh

• understanding the coin flip
• understanding the outer assumptions one can and can't take from the traits
• hardening the randomness, which are useful for what scenario, which drawbacks comes with them

One does not need to "enable compression" for a runtime.
Wasm compression has been added to the substrate-wasm-builder on April 7th.
Polkadot and Kusama (presently) use the wasm-builder v3.0.0 which was tagged Feb 10th (so no compression).

Compressed WASM is critical for parachains. We should note this in the KB (#1027) and possibly explain in some new content what all build artifacts substrate produces and why.

The scenario:
A pallet has a custom origin used with proposals similar to the collective pallet.
Examples of:
How to use that custom origin to set rules for extrinsics within the same pallet.
How to use that custom origin to set a rule for all extrinsics in another pallet. (eg require a certain threshold of votes)
How to use that custom origin to set a rule to be checked within each extrinsic of another pallet ie check the number of votes within the extrinsic based on logic.

I figured out the above eventually, but it took a while to understand how it was done. It would have been helpful to have a basic description of how it works and what is going on behind the scenes in the macros. I am not yet a Rust expert and the macros make it even harder to understand.