Hi,
I would like to get feedback on a proposed method for Client such that
client.create_raw_payload(AccountId, Call) returns the payload to be signed.

The reason is so that the payload may be signed by other signers (maybe not in the XtBuilder and use this or other tools to create uncheckedExtrinsic.

transaction signing should be decoupled from the client

maybe XtBuilder should take a Sink<Item = (UnsignedTransaction, oneshot::Sender<Result<Transaction, AbortError>>)>

This hasn’t been implemented, but it should be.

should be useful to test things like #103 or #91

should have a plain, map, double map store item and some transactions to deterministically modify them. we should also have a transaction that always returns an error and one that always returns an event.

At the moment events from ExtrinsicSuccess are encoded in the RuntimeEvents struct. The RuntimeEvents struct has no relation to the runtime types. It would be preferable to instead decode the events as the real event type obtained from the runtime. More concretely

pub struct ExtrinsicSuccess<T: System> {
  // ...
  events: srml_system::EventRecord<T::Event, T::Hash>
}

This would require us to add the associated Event type to the System trait.

With this implementation we could do away with the dynamic event parsing which suffers from a couple of issues.

• It needs to fetch the metadata
• It requires substantial manual decoding logic for events
• It requires registering type sizes for custom structs.

I’d be willing to give this a stab.

Runtime errors need to be handled somehow I guess

Could you explain to me the following questions:

• Is it possible to make ClientBuilder for custom runtime?
• Should I implement System for it?

Some signers require asynchronously interacting with external hardware, such as a hardware wallet. Therefore, Signer should return a Future.

All dependencies are now available on crates.io, so we should publish now.

From https://github.com/paritytech/substrate-subxt/pull/99/files#diff-04c6e90faac2675aa89e2176d2eec7d8R9:

If you use #[derive(Call)] without #[module] in the same module, you will get errors
complaining about an undefined method with a name starting with with_.

Many times, we need to fetch all the values in a map. But now subxt only supports fetch one value by key.
Can you support this function?

In original implementation, there needs ModuleCalls to delegate the call of those srml module, I think this is owed to the problem of the project structure.

This design leads to a severe problem: if user of this library wants to introduce a new runtime module, he needs to impl ModuleCalls directly in his module file, and this is prevented by Rust for orphan rule.

So, I just do some changes for it. If we can agree on this approach, I will make a PR on it.

Thank you.

Ref: https://github.com/daogangtang/substrate-subxt/tree/master/src

Treasury::Deposit can't be decoded, see log below:

[2020-06-22T09:16:51Z DEBUG substrate_subxt::events] received event 'Treasury::Deposit'
Error: Events(TypeSizeUnavailable("Balance"))

AFAIK, this could be fixed by register Balance in EventsDecoder::new. Why isn't it?

@ascjones I could be just missing something obvious, but it seems the client's "async fn block_hash" is not completely usable by library users because "mod rpc" is private. The parameter is defined in mod rpc as pub type BlockNumber<T> = sp_rpc::number::NumberOrHex<<T as System>::BlockNumber>

Without reexporting sp_rpc::number::NumberOrHex or including rpc::BlockNumber in pub use, I'm not exactly sure how library users can get a specific block hash using a block number.

Incidentally, sp_rpc::number::NumberOrHex only implements std::convert::From for NumberOrHex<u64>, but kusama, node-template and basically all examples of substrate chains set System::BlockNumber to u32. Therefore,Some(9001u32.into()) does not work either.

Again, it is late and I might be missing something. If this is an oversight in the library, I can do a PR with some basic unit tests. I'm not sure if you would prefer to expose BlockNumber, reexport sp_rpc::number, or add more From's for different integer types in substrate/primitive/rpc.

Edit:

I was able to do sp_core::U256::from(x).into() to take advantage of the from Trait in sp_rpc::number. This works but seems to be a big circle to get to the right type.

Edit 2:

I just ran into the same issue with client.fetch for the StorageKey tuple struct. Can't call the method properly due to unexposed StorageKey. I'm importing my own substrate in Cargo.toml that is pinned to a specific git revision.

Since converting to jsonrpsee, subscriptions are handled with while let msg = sub.next().await which is an infinite loop (see paritytech/jsonrpsee#86).

What if no more messages are received on a subscription (e.g. because connection has closed), or if the specific message we are waiting for doesn't arrive? Should we add a timeout? Otherwise we will wait indefinitely e.g. https://github.com/paritytech/substrate-subxt/blob/253a7d8b0b4ff33744aafc2f4aea6a6d14647681/src/rpc.rs#L286.

To speed up initial sync most light clients distribute the block headers with the binary. The substrate cli supports exporting and importing blocks, but it seems like it exports/imports the entire block.

I am unable to run the example with the following versions:

1. using substrate node version 2.0.0-alpha.5-beff53804-x86_64-macos
2. using release 0.5 of this repo

Error: Rpc(Request(Error { code: ServerError(1002), message: "Verification Error: Execution(ApiError(\"Could not convert parameter `tx` between node and runtime: No such variant in enum MultiSignature\"))", data: Some(String("RuntimeApi(\"Execution(ApiError(\\\"Could not convert parameter `tx` between node and runtime: No such variant in enum MultiSignature\\\"))\")")) }))

Currently when submitting an extrinsic, the future doesn't resolve until the extrinsic is included in a block. The future should return immediately and ExtrinsicSuccess should expose a wait_block and wait_final which return futures.

Consider using https://github.com/paritytech/jsonrpsee for this.

Trying to use the procedural macro for raw storage (that is, storage not in a map) panics with an unwrap failure.

Currently there isn't any no_std support for substrate-subxt making it really difficult to craft extrinsics from within a restricted environment. Is there any planned support for no_std / alloc environments?

Since upgrading to latest signing #33 and Extrinsic version 4, the put_code test fails with Failed to submit extrinsic: Transaction pool error: Invalid transaction validity: InvalidTransaction::BadProof.

Oddly the transfer_balance test succeeds, so not sure why the signature is good for one and not the other.

Hi,

I know there's a module named utility that can send batch transactions.
So if subxt supports it, that would be great.

For our integration tests which require a running substrate node, we should consider running a tagged docker image of substrate: https://hub.docker.com/r/parity/substrate/tags. Also see https://github.com/paritytech/pallet-contracts-waterfall#using-a-docker-image-of-substrate-for-testing

That way we do not need to maintain all modules in the local test node.

Misusing the proc macros results in very unhelpful error messages. I have seen:

• Unwrapping None
• Undefined variables with no explanation
• Probably others

Furthermore, the correct use of the proc macros is poorly documented. I had to figure it out by trial and error.

Now in set_call, we do two things now:

1. set_call
2. build the XtBuilder instance

Should we split it into two methods: set_call() and build(), and use it as:

xt.foo().bar().set_call().build()

At a glance this library seems to assume extrinsics are signed, it would be nice to support unsigned extrinsics.

Consider supporting multiple versions of substrate metadata (currently on v8).

This would allow the library to be used against chains running different versions of substrate.

Rel #25

See #165 (review)

When submitting an extrinsic we occasionally get the following UnknownSubscriptionId WARN/ERROR from jsonrpsee:

https://github.com/paritytech/jsonrpsee/blob/a0bea41c4f37125fa742ec48b12e11cf55c592b5/src/raw/client.rs#L457

This is from the subscription listening for block events. It doesn't stop it working but it is annoying for the user and potentially misleading.

See https://docs.rs/crate/substrate-subxt/0.8.0

Hi, I use subxt-v0.8.0 and subxt-v0.9.0, they both have some bugs as follows:

 Compiling substrate-subxt v0.9.0
error[E0107]: wrong number of type arguments: expected 0, found 1
  --> /Users/lawliet/.cargo/registry/src/crates.rustcc.cn-46a557d00542c05f/substrate-subxt-0.9.0/src/rpc.rs:89:26
   |
89 | impl<T> From<NumberOrHex<<T as System>::BlockNumber>> for BlockNumber<T>
   |                          ^^^^^^^^^^^^^^^^^^^^^^^^^^ unexpected type argument

error[E0107]: wrong number of type arguments: expected 0, found 1
  --> /Users/lawliet/.cargo/registry/src/crates.rustcc.cn-46a557d00542c05f/substrate-subxt-0.9.0/src/rpc.rs:87:47
   |
87 | pub struct BlockNumber<T: System>(NumberOrHex<<T as System>::BlockNumber>);
   |                                               ^^^^^^^^^^^^^^^^^^^^^^^^^^ unexpected type argument

error[E0107]: wrong number of type arguments: expected 0, found 1
  --> /Users/lawliet/.cargo/registry/src/crates.rustcc.cn-46a557d00542c05f/substrate-subxt-0.9.0/src/rpc.rs:93:28
   |
93 |     fn from(x: NumberOrHex<<T as System>::BlockNumber>) -> Self {
   |                            ^^^^^^^^^^^^^^^^^^^^^^^^^^ unexpected type argument

error: aborting due to 3 previous errors

For more information about this error, try `rustc --explain E0107`.
error: could not compile `substrate-subxt`.

My rust toolchain is nightly-2020-06-16

I want extract extrinsic from rpc result, but got invalid transaction version error, please help.

use substrate_subxt::{
    ClientBuilder,
    KusamaRuntime,
    runtimes::*,
    frame::system::*,
    frame::balances::*,
    extra::DefaultExtra
};
use sp_runtime::{
    generic::{
        UncheckedExtrinsic
    },
    MultiSignature,
    AnySignature
};
use codec::{Decode, Encode, EncodeLike, Input, Error};
use sp_core::crypto::Pair;

type Address = <KusamaRuntime as System>::Address;
type Call = Vec<u8>;
type Signature = AnySignature;
type Extra = DefaultExtra<KusamaRuntime>;

type UnEx = UncheckedExtrinsic<Address, Call, Signature, Extra>;


#[async_std::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    env_logger::init();

    let client = ClientBuilder::<KusamaRuntime>::new()
        .set_url("wss://kusama-rpc.polkadot.io")
        .build()
        .await?;

    let block_number = 2852995;

    let block_hash = client.block_hash(Some(block_number.into())).await?;

    if let Some(hash) = block_hash {
        println!("Block hash for block number {}: {:?}", block_number, hash);

        let block = client.block(Some(hash)).await?.unwrap();

        println!("block = {:?}", block);

        let extrinsic: sp_runtime::OpaqueExtrinsic = block.block.clone().extrinsics[2].clone();

        let mut input = extrinsic.0.as_slice();
        println!("input  = {:?}", &input);
        // UncheckedExtrinsic<Address, Call, Signature, Extra>
        let uncheck_extrinsic = UnEx::decode(&mut input);

        println!("block extrinsics = {:?}", uncheck_extrinsic);  // Got  `Invalid transaction version` error

    } else {
        println!("Block number {} not found.", block_number);
    }

    Ok(())
}

Outputs:

Block hash for block number 2852995: 0xdbe3277db35554024f68d947b0f2a08bb2a2bb14b5723a60e44c930648dcbca2
block = SignedBlock { block: Block { header: Header { parent_hash: 0x9685770bfbc3b213b63e59dec3d518b6dab61849cb944b066644b56d94eda6c1, number: 2852995, state_root: 0x458ef43522540efc0b4a7be7f2aef470d229bdf4bad31f0bc38e6c67668f5a46, extrinsics_root: 0xc1d6f1f1216393b81bc155cf9f8163c11b51e3225df5a68febd6d1e9096f409b, digest: Digest { logs: [DigestItem::PreRuntime([66, 65, 66, 69], [2, 121, 0, 0, 0, 244, 178, 210, 15, 0, 0, 0, 0]), DigestItem::Seal([66, 65, 66, 69], [90, 217, 174, 195, 252, 86, 236, 170, 4, 204, 232, 209, 46, 172, 53, 117, 143, 194, 177, 184, 203, 42, 251, 157, 120, 136, 50, 141, 133, 137, 240, 90, 3, 229, 135, 210, 32, 181, 154, 216, 14, 61, 108, 141, 218, 110, 63, 172, 113, 128, 209, 228, 180, 66, 218, 161, 136, 38, 94, 73, 62, 160, 102, 137])] } }, extrinsics: [0402000bc03642da7201, 04140000, 84fe56be5933800b45a21ee8e9817eae9f49099fdf4a20076718497092ed43c62b019a7261d479c4b495cfd1fcffcf9daf35600c391e752e99c805e7951388fb17663a8ab4f9c2241835599c1df443630f0c218b3bdc3e83f5e0390cfced7e580b8ef50374000400a8070648f6e43b7d8a8f10418b696130d2046c10645f10085de80e6098b85f0902286bee] }, justification: None }
input  = [132, 254, 86, 190, 89, 51, 128, 11, 69, 162, 30, 232, 233, 129, 126, 174, 159, 73, 9, 159, 223, 74, 32, 7, 103, 24, 73, 112, 146, 237, 67, 198, 43, 1, 154, 114, 97, 212, 121, 196, 180, 149, 207, 209, 252, 255, 207, 157, 175, 53, 96, 12, 57, 30, 117, 46, 153, 200, 5, 231, 149, 19, 136, 251, 23, 102, 58, 138, 180, 249, 194, 36, 24, 53, 89, 156, 29, 244, 67, 99, 15, 12, 33, 139, 59, 220, 62, 131, 245, 224, 57, 12, 252, 237, 126, 88, 11, 142, 245, 3, 116, 0, 4, 0, 168, 7, 6, 72, 246, 228, 59, 125, 138, 143, 16, 65, 139, 105, 97, 48, 210, 4, 108, 16, 100, 95, 16, 8, 93, 232, 14, 96, 152, 184, 95, 9, 2, 40, 107, 238]
block extrinsics = Err(Error("Invalid transaction version"))

In some scenarios, we need to query unhashed storage directly. Can the new fetch add this support? Or should we simply expose rpc.storage? cc @dvc94ch

I do not know how to get the self-defined module state using this current version of subxt. I can only find the way of sending extrinsics. Will you consider adding this functionality of getting ledger state?

Might be nice to have a higher level api for subscribing to a particular event type and doing the decoding, but no need for this PR

_Originally posted by @ascjones in #126 (comment)

We need an iterator-like API for subscribing to events for @sunshine-protocol CC @dvc94ch

error[E0432]: unresolved import sp_runtime::traits::SimpleArithmetic
--> src/frame/balances.rs:30:5
|
30 | SimpleArithmetic,
| ^^^^^^^^^^^^^^^^ no SimpleArithmetic in traits

error[E0432]: unresolved import sp_runtime::traits::SimpleArithmetic
--> src/frame/system.rs:38:5
|
38 | SimpleArithmetic,
| ^^^^^^^^^^^^^^^^ no SimpleArithmetic in traits

I think this is due to the update of the substrate. Maybe we should add the rev of the substrate's github dependencies.

The substrate_subxt::error::RuntimeError is not exported and error is private so it is impossible to match on RuntimeError. Exporting this error is probably a good idea.

Currently, doing so is not allowed.

Could enable supporting multiple metadata versions (#26) and potentially replace the dynamic event decoding.

Intermittently, all tests pass but the CI job fails with pure virtual method called.

Could be related: https://github.com/openethereum/openethereum/issues/6762, facebook/rocksdb#649.

Perhaps because of the test node, which probably uses rocksdb.

The current git master is not compatible with what is on-chain right now.

Currently, backwards-incompatible metadata changes cause a nasty problem for subxt, and thus Ledgeracio. The old version of subxt supports the old runtime, and the new version of subxt supports the new runtime. However, the on-chain runtime upgrade can happen at any time, so there is no way to know when to switch without race conditions.

To solve this, subxt needs to support multiple incompatible runtime versions. Support for a version will be added before it is proposed on-chain. Support for a version will only be removed when all runtimes have migrated off of it.

• build
• test
• fmt

Integration Tests

If possible run integration tests against a recent substrate node instance (docker).

Currently, all client implementations are written by hand. This works, but is very tedious and won’t scale to large runtimes.

A better option is to generate the client. This is already done by the TypeScript client implementation, so it should be quite doable for the Rust one.

Hi there,

This issue might be quite the same with How could I send a sudo call.

I'm using subxt-0.6, which works fast and as expected. But I find out that this interface module_with_calls become private while I try to upgrade to 0.7. Here is code sample with subxt-0.6

let proposal = client.metadata().module_with_calls("MyModule")
		.and_then(|module| module.call("my_call", my_call_args))
		.map_err(|_| crate::Error::SubxtError("failed to compose a sudo call"))?;
let call = Call::new("Sudo", "sudo", proposal);

So now I cannot upgrade subxt to 0.7 or higher, can that interface change to be public? Or maybe you can give me a better solution instead of changing that interface.

Thanks!

Just pull the latest code, and open tests, run it, display below:

mike@DESKTOP-DC63SDI:~/works/substrate_works/substrate-subxt$ cargo test --release
   Compiling node-runtime v2.0.0 (https://github.com/paritytech/substrate/#337a0b22)
   Compiling substrate-subxt v0.1.0 (/home/mike/works/substrate_works/substrate-subxt)
    Finished release [optimized] target(s) in 1m 12s
     Running ../substrate/target/release/deps/substrate_subxt-ec0abb6820fe6c2b

running 6 tests
test tests::test_tx_contract_put_code ... FAILED
test tests::test_chain_read_metadata ... FAILED
test tests::test_tx_transfer_balance ... FAILED
test tests::test_chain_subscribe_blocks ... FAILED
test tests::test_chain_subscribe_finalized_blocks ... FAILED
test tests::test_state_read_free_balance ... FAILED

failures:

---- tests::test_tx_contract_put_code stdout ----
thread 'tests::test_tx_contract_put_code' panicked at 'called `Result::unwrap()` on an `Err` value: Rpc(ParseError("RuntimeVersion", Error("invalid type: string \"0xdf6acb689907609b\", expected an array of length 8", line: 0, column: 0)))', src/libcore/result.rs:999:5

---- tests::test_chain_read_metadata stdout ----
thread 'tests::test_chain_read_metadata' panicked at 'called `Result::unwrap()` on an `Err` value: Rpc(ParseError("RuntimeVersion", Error("invalid type: string \"0xdf6acb689907609b\", expected an array of length 8", line: 0, column: 0)))', src/libcore/result.rs:999:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.

---- tests::test_tx_transfer_balance stdout ----
thread 'tests::test_tx_transfer_balance' panicked at 'called `Result::unwrap()` on an `Err` value: Rpc(ParseError("RuntimeVersion", Error("invalid type: string \"0xdf6acb689907609b\", expected an array of length 8", line: 0, column: 0)))', src/libcore/result.rs:999:5

---- tests::test_chain_subscribe_blocks stdout ----
thread 'tests::test_chain_subscribe_blocks' panicked at 'called `Result::unwrap()` on an `Err` value: Rpc(ParseError("RuntimeVersion", Error("invalid type: string \"0xdf6acb689907609b\", expected an array of length 8", line: 0, column: 0)))', src/libcore/result.rs:999:5

---- tests::test_chain_subscribe_finalized_blocks stdout ----
thread 'tests::test_chain_subscribe_finalized_blocks' panicked at 'called `Result::unwrap()` on an `Err` value: Rpc(ParseError("RuntimeVersion", Error("invalid type: string \"0xdf6acb689907609b\", expected an array of length 8", line: 0, column: 0)))', src/libcore/result.rs:999:5

---- tests::test_state_read_free_balance stdout ----
thread 'tests::test_state_read_free_balance' panicked at 'called `Result::unwrap()` on an `Err` value: Rpc(ParseError("RuntimeVersion", Error("invalid type: string \"0xdf6acb689907609b\", expected an array of length 8", line: 0, column: 0)))', src/libcore/result.rs:999:5


failures:
    tests::test_chain_read_metadata
    tests::test_chain_subscribe_blocks
    tests::test_chain_subscribe_finalized_blocks
    tests::test_state_read_free_balance
    tests::test_tx_contract_put_code
    tests::test_tx_transfer_balance

test result: FAILED. 0 passed; 6 failed; 0 ignored; 0 measured; 0 filtered out

error: test failed, to rerun pass '--lib'

Hi there,

I just used this crate create a general call for my own module, and it worked as expected.
But while I tried to send a sudo call, it failed, the error message likes

DEBUG runtime  DispatchError
DEBUG runtime  Bad origin

And I checked examples and test cases, it looks there's no sample codes in it.
So my problem is how can I compose a sudo call and send it to a running node.

@tomusdrw is working on a library which should extract the substrate rpc traits, so we will no longer need to depend on the entire substrate-rpc crate.