banteg / multicall.py Goto Github PK

View Code? Open in Web Editor NEW

242.0 5.0 101.0 193 KB

aggregate results of multiple smart contract calls into one

License: MIT License

Python 100.00%

ethereum makerdao multicall web3 web3py daistats

multicall.py's Introduction

multicall.py

python interface for makerdao's multicall and a port of multicall.js

installation

pip install multicall

example

from multicall import Call, Multicall

# assuming you are on kovan
MKR_TOKEN = '0xaaf64bfcc32d0f15873a02163e7e500671a4ffcd'
MKR_WHALE = '0xdb33dfd3d61308c33c63209845dad3e6bfb2c674'
MKR_FISH = '0x2dfcedcb401557354d0cf174876ab17bfd6f4efd'

def from_wei(value):
    return value / 1e18

multi = Multicall([
    Call(MKR_TOKEN, ['balanceOf(address)(uint256)', MKR_WHALE], [('whale', from_wei)]),
    Call(MKR_TOKEN, ['balanceOf(address)(uint256)', MKR_FISH], [('fish', from_wei)]),
    Call(MKR_TOKEN, 'totalSupply()(uint256)', [('supply', from_wei)]),
])

multi()  # {'whale': 566437.0921992733, 'fish': 7005.0, 'supply': 1000003.1220798912}

# seth-style calls
Call(MKR_TOKEN, ['balanceOf(address)(uint256)', MKR_WHALE])()
Call(MKR_TOKEN, 'balanceOf(address)(uint256)')(MKR_WHALE)
# return values processing
Call(MKR_TOKEN, 'totalSupply()(uint256)', [('supply', from_wei)])()

for a full example, see implementation of daistats. original daistats.com made by nanexcool.

api

`Signature(signature)`

signature is a seth-style function signature of function_name(input,types)(output,types). it also supports structs which need to be broken down to basic parts, e.g. (address,bytes)[].

use encode_data(args) with input args to get the calldata. use decode_data(output) with the output to decode the result.

`Call(target, function, returns)`

target is the to address which is supplied to eth_call.
function can be either seth-style signature of method(input,types)(output,types) or a list of [signature, *args].
returns is a list of tuples of (name, handler) for return values. if returns argument is omitted, you get a tuple, otherwise you get a dict. to skip processing of a value, pass None as a handler.

use Call(...)() with predefined args or Call(...)(args) to reuse a prepared call with different args.

use decode_output(output) with to decode the output and process it with returns handlers.

`Multicall(calls)`

calls is a list of calls with prepared values.

use Multicall(...)() to get the result of a prepared multicall.

Environment Variables

GAS_LIMIT: sets overridable default gas limit for Multicall to prevent out of gas errors. Default: 50,000,000
MULTICALL_DEBUG: if set, sets logging level for all library loggers to logging.DEBUG
MULTICALL_PROCESSES: pass an integer > 1 to use multiprocessing for encoding args and decoding results. Default: 1, which executes all code in the main process.
AIOHTTP_TIMEOUT: sets aiohttp timeout period in seconds for async calls to node. Default: 30

test

export WEB3_INFURE_PROJECT_ID=<your_infura_id>
export PYTEST_NETWORK='mainnet'
poetry run python -m pytest

multicall.py's People

Contributors

Stargazers

Watchers

Forkers

wooparadog 57s blockchain-development-resources leadblock pavlovdog konqueror1 joyious jimtje shaodan dirt-nasty flipsidecrypto dautonomous danielfv myofinance pynchmeister yujianfresh iliastsa drbh yanis7774 ccamateur gerrrg btautist jacksonsdean revotu colevs sdkvariant 007vasy codephilanthropist shubbard76 timeconometrics 0xsimulacra zhangrhdx bitrocks playniuniu jimmyx0x endygamie barabazs yangyongcs alesilv marcinspro pseudocodesnipe heki-ru growupinv j-iinn sbtobb silvercondor dedaub 0xinsanity yuntai cleancoindev cheekybastard abhguptads cbsystango jc92 zhaifg pyk defidebauchery kobe-eth aileen5150 coolberwin zekiblue medariox stratum-exchange whonion codybraun p3810cyber nucypher jivin0902 d-e-f-t scottincrypto johnny-vu darkerego 0xketch69 akremstudy joshlam123 0xbasically karuzzzo jerryname2022 haiyanghe kronos-mjolnir undefined6 nemo-nesciam equilibre-finance samhagan kodo-exchange gpsanant isabella232 yangzhijiawork justin-malonson equilibre-finance will9709

multicall.py's Issues

Version discrepancy

Recently Multicall2 support has been added with version upgrade to 0.1.3: #8
But then 0.1.3 tag was added to an older version of the files.

And now the last version (0.1.3) doesn't come with Multicall2 suport

Consider switching to ThreadPoolExecutor

We're running multicall in a background worker and we could see an abnormal usage of the memory. After reviewing the implementation and switching to concurrent.futures.ThreadPoolExecutor, the processes stopped leaking memory.

Relevant line of code:
https://github.com/banteg/multicall.py/blob/master/multicall/utils.py#L27

web3 version 6+ support?

Hi,

Looking to use Multicall in a web3py >6.0 project. Would be great to upgrade to latest web3

Don't work with multi threads and jupyter

running Multicall in main thread works great, but running the function twice in another thread is going to throw an error

"RuntimeError: Timeout context manager should be used inside a task"

and when using Multicall with jupyter, u get this error

File ~/.local/lib/python3.10/site-packages/multicall/multicall.py:57, in Multicall.call(self)
55 def call(self) -> Dict[str,Any]:
56 start = time()
---> 57 response = await_awaitable(self.coroutine())
58 logger.debug(f"Multicall took {time() - start}s")
59 return response

File ~/.local/lib/python3.10/site-packages/multicall/utils.py:67, in await_awaitable(awaitable)
66 def await_awaitable(awaitable: Awaitable) -> Any:
---> 67 return get_event_loop().run_until_complete(awaitable)

File /usr/lib64/python3.10/asyncio/base_events.py:622, in BaseEventLoop.run_until_complete(self, future)
611 """Run until the Future is done.
612
613 If the argument is a coroutine, it is wrapped in a Task.
(...)
619 Return the Future's result, or raise its exception.
620 """
621 self._check_closed()
--> 622 self._check_running()
624 new_task = not futures.isfuture(future)
625 future = tasks.ensure_future(future, loop=self)

File /usr/lib64/python3.10/asyncio/base_events.py:582, in BaseEventLoop._check_running(self)
580 def _check_running(self):
581 if self.is_running():
--> 582 raise RuntimeError('This event loop is already running')
583 if events._get_running_loop() is not None:
584 raise RuntimeError(
585 'Cannot run the event loop while another loop is running')

RuntimeError: This event loop is already running

ValueOutOfBounds when using default-no-parallelism branch

I have read the Issue of #42 and #44 and clone the default-no-parallelism branch

my code is like this

def from_wei(value):
    return value / 1e18

def generate_multicall(call_list: list):
    multi = Multicall(call_list, _w3=w3)
    return multi

fund_vault_proxy_address = "0x9dd3b3471AF147DF6c7E93ff35a5f04eE9342e9C"

asset_data_list = list(Asset.objects.values_list("address", "name"))
test =[1,2,3]
multi = []
for asset_data in asset_data_list:
    # asset_address_list.append(asset_data[0])
    multi.append(
        Call(
            "0xd1Cc87496aF84105699E82D46B6c5Ab6775Afae4",
            ["balanceOf(address)(uint256)", fund_vault_proxy_address],
            [[asset_data[1], from_wei]],
        )
    )
    # asset_name_list.append(asset_data[1])
data = generate_multicall(multi)

run it and return Error

eth_abi.exceptions.ValueOutOfBounds: Value `[[['0xd1Cc87496aF84105699E82D46B6c5Ab6775Afae4', b'p\xa0\x821\x00\x00\x00\x0...` of type <class 'list'> cannot be encoded by TupleEncoder: value has 1 items when 2 were expected

Can anyone assist here?
Thank you in advance.

full error message:

Traceback (most recent call last):
  File "/Users/lj/Documents/Work/NCHU/OnChainFund/Backend/./manage.py", line 22, in <module>
    main()
  File "/Users/lj/Documents/Work/NCHU/OnChainFund/Backend/./manage.py", line 18, in main
    execute_from_command_line(sys.argv)
  File "/Users/lj/Documents/Work/NCHU/OnChainFund/Backend/.venv/lib/python3.10/site-packages/django/core/management/__init__.py", line 446, in execute_from_command_line
    utility.execute()
  File "/Users/lj/Documents/Work/NCHU/OnChainFund/Backend/.venv/lib/python3.10/site-packages/django/core/management/__init__.py", line 440, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/Users/lj/Documents/Work/NCHU/OnChainFund/Backend/.venv/lib/python3.10/site-packages/django/core/management/base.py", line 402, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/Users/lj/Documents/Work/NCHU/OnChainFund/Backend/.venv/lib/python3.10/site-packages/django/core/management/base.py", line 448, in execute
    output = self.handle(*args, **options)
  File "/Users/lj/Documents/Work/NCHU/OnChainFund/Backend/.venv/lib/python3.10/site-packages/django/core/management/commands/shell.py", line 127, in handle
    exec(sys.stdin.read(), globals())
  File "<string>", line 35, in <module>
  File "/Users/lj/Documents/Work/NCHU/OnChainFund/Backend/utils/multicall/multicall.py", line 76, in __call__
    response = await_awaitable(self.coroutine())
  File "/Users/lj/Documents/Work/NCHU/OnChainFund/Backend/utils/multicall/utils.py", line 65, in await_awaitable
    return get_event_loop().run_until_complete(awaitable)
  File "/opt/homebrew/Cellar/[email protected]/3.10.5/Frameworks/Python.framework/Versions/3.10/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
    return future.result()
  File "/Users/lj/Documents/Work/NCHU/OnChainFund/Backend/utils/multicall/multicall.py", line 81, in coroutine
    batches = await gather(
  File "/Users/lj/Documents/Work/NCHU/OnChainFund/Backend/utils/multicall/utils.py", line 82, in gather
    raise_if_exception_in(results)
  File "/Users/lj/Documents/Work/NCHU/OnChainFund/Backend/utils/multicall/utils.py", line 78, in raise_if_exception_in
    raise_if_exception(obj)
  File "/Users/lj/Documents/Work/NCHU/OnChainFund/Backend/utils/multicall/utils.py", line 74, in raise_if_exception
    raise obj
  File "/Users/lj/Documents/Work/NCHU/OnChainFund/Backend/utils/multicall/multicall.py", line 121, in fetch_outputs
    _raise_or_proceed(e, len(calls), ConnErr_retries=ConnErr_retries)
  File "/Users/lj/Documents/Work/NCHU/OnChainFund/Backend/utils/multicall/multicall.py", line 250, in _raise_or_proceed
    raise e
  File "/Users/lj/Documents/Work/NCHU/OnChainFund/Backend/utils/multicall/multicall.py", line 102, in fetch_outputs
    _, outputs = await self.aggregate.coroutine(args)
  File "/Users/lj/Documents/Work/NCHU/OnChainFund/Backend/.venv/lib/python3.10/site-packages/eth_retry/eth_retry.py", line 63, in auto_retry_wrap_async
    return await func(*args,**kwargs)
  File "/Users/lj/Documents/Work/NCHU/OnChainFund/Backend/utils/multicall/call.py", line 111, in coroutine
    args = await run_in_subprocess(
  File "/Users/lj/Documents/Work/NCHU/OnChainFund/Backend/utils/multicall/utils.py", line 69, in run_in_subprocess
    return callable(*args, **kwargs)
  File "/Users/lj/Documents/Work/NCHU/OnChainFund/Backend/utils/multicall/call.py", line 134, in prep_args
    calldata = signature.encode_data(args)
  File "/Users/lj/Documents/Work/NCHU/OnChainFund/Backend/utils/multicall/signature.py", line 39, in encode_data
    return self.fourbyte + encode_single(self.input_types, args) if args else self.fourbyte
  File "/Users/lj/Documents/Work/NCHU/OnChainFund/Backend/.venv/lib/python3.10/site-packages/eth_abi/codec.py", line 72, in encode_single
    return encoder(arg)
  File "/Users/lj/Documents/Work/NCHU/OnChainFund/Backend/.venv/lib/python3.10/site-packages/eth_abi/encoding.py", line 98, in __call__
    return self.encode(value)
  File "/Users/lj/Documents/Work/NCHU/OnChainFund/Backend/.venv/lib/python3.10/site-packages/eth_abi/encoding.py", line 139, in encode
    self.validate_value(values)
  File "/Users/lj/Documents/Work/NCHU/OnChainFund/Backend/.venv/lib/python3.10/site-packages/eth_abi/encoding.py", line 123, in validate_value
    self.invalidate_value(
  File "/Users/lj/Documents/Work/NCHU/OnChainFund/Backend/.venv/lib/python3.10/site-packages/eth_abi/encoding.py", line 88, in invalidate_value
    raise exc(
eth_abi.exceptions.ValueOutOfBounds: Value `[[['0xd1Cc87496aF84105699E82D46B6c5Ab6775Afae4', b'p\xa0\x821\x00\x00\x00\x0...` of type <class 'list'> cannot be encoded by TupleEncoder: value has 1 items when 2 were expected

Example Issue

Hey,

I tried running the example and ended up on the below traceback

Traceback (most recent call last):

  File "<ipython-input-33-9c652de726e4>", line 20, in <module>
    multi()  # {'whale': 566437.0921992733, 'fish': 7005.0, 'supply': 1000003.1220798912}

  File "C:\kaleb\miniconda\envs\skew\lib\site-packages\multicall\multicall.py", line 30, in __call__
    result.update(call.decode_output(output))

  File "C:\kaleb\miniconda\envs\skew\lib\site-packages\multicall\call.py", line 30, in decode_output
    decoded = self.signature.decode_data(output)

  File "C:\kaleb\miniconda\envs\skew\lib\site-packages\multicall\signature.py", line 39, in decode_data
    return decode_single(self.output_types, output)

  File "C:\kaleb\miniconda\envs\skew\lib\site-packages\eth_abi\codec.py", line 155, in decode_single
    return decoder(stream)

  File "C:\kaleb\miniconda\envs\skew\lib\site-packages\eth_abi\decoding.py", line 127, in __call__
    return self.decode(stream)

  File "C:\kaleb\miniconda\envs\skew\lib\site-packages\eth_utils\functional.py", line 45, in inner
    return callback(fn(*args, **kwargs))

  File "C:\kaleb\miniconda\envs\skew\lib\site-packages\eth_abi\decoding.py", line 173, in decode
    yield decoder(stream)

  File "C:\kaleb\miniconda\envs\skew\lib\site-packages\eth_abi\decoding.py", line 127, in __call__
    return self.decode(stream)

  File "C:\kaleb\miniconda\envs\skew\lib\site-packages\eth_abi\decoding.py", line 198, in decode
    raw_data = self.read_data_from_stream(stream)

  File "C:\kaleb\miniconda\envs\skew\lib\site-packages\eth_abi\decoding.py", line 308, in read_data_from_stream
    len(data),

InsufficientDataBytes: Tried to read 32 bytes.  Only got 0 bytes

Note sure what the issue is, tried debugging it, but it seems running the call returns empty b'' which proceeds to fail on the decoder

        for call, output in zip(self.calls, outputs):
            result.update(call.decode_output(output))

Fixing typehinting in Call's `returns` argument

I'm not strongly-versed in python typing (especially in pre-3.9 versions where hints became much more intuitive), otherwise I would have sent a PR directly, but trying to understand.

Call's __init__ method has a argument returns (and deocde_output()), which is defined as such

class Call:
  def __init__(
    ...
    returns: Optional[Iterable[Tuple[str,Callable]]] = None,
    ...
  ) -> None:

Since the second argument to the list can be None, would it be better for the Callable to be replaced with Optional[Callable] or simply Callable | None (or I guess in <3.9 syntax, Union[Callable, None])?

I have never seen the use of Tuples anywhere - both the examples and the tests use lists - but changing Tuple to List throws its own error since it's not expecting more than one argument to the List type. Changing it to Iterable[str, Optional[Callable]], while passing lint check, seems wonky because it definitely doesn't accept dicts.

What's the best way to properly represent the inputs?
This is relatively minor in the grand scope of things, but is an annoyance during development, and being technically correct is the best kind of correct.

Access block number from the result dictionary

Looks like multicall returns the blockNumber during aggregate (or tryBlockAndAggregate) calls.

However, it is omitted from the results dictionary: https://github.com/banteg/multicall.py/blob/master/multicall/multicall.py#L86

In cases where per-block data needs to be accessed and stored, it would be beneficial to know the block number on which the multicall request was performed on, saving the user from doing a separate web3.eth.block_number request just after the multicall request.

Maybe the block number can be returned as '_block_number' in the results dictionary.

Unable to use Multicall with threads in docker.

Docker throws following error when multicall is executed in threads(parallel execution)

The latest version has a deadlock

Because the code is deadlocked！

multicall.py and call.py all use

async with _get_semaphore(), so you must remove call.py

then
it is ok！！！

Python official example：
https://docs.python.org/zh-cn/3/library/asyncio-sync.html?highlight=semaphore#asyncio.Semaphore

sem = asyncio.Semaphore()

await sem.acquire()
try:

work with shared resource

finally:
sem.release()

add multicall v3 support

https://github.com/mds1/multicall

there is a new multicall version (v3) which is deployed over 50+ networks with the same address:
https://github.com/mds1/multicall#multicall3-contract-addresses

Multicall() does not work in pip version 0.7.9

Issue with:
multicall==0.7.9

I create a Call (programmatically) which works if called it by itself.
It calls the token0() function from a UniswapPair-like contract, returns the address of the base token for that pair .

Here are its details:

# on optimism mainnet

>>> c.target
'0xf4d40ebCBf7063D4ff56C6Df0179a86287C648dE'

>>> c.function
'token0()(address)'

>>> c.data
b'\r\xfe\x16\x81'

>>> c.args
[]

>>> c()
returns: [('0xf4d40ebCBf7063D4ff56C6Df0179a86287C648dE_base', None)]
decoded: ('0xb6599bd362120dc70d48409b8a08888807050700',)
{'0xf4d40ebCBf7063D4ff56C6Df0179a86287C648dE_base': '0xb6599bd362120dc70d48409b8a08888807050700'}

But does not work when called as part of a Multicall:

>>> Multicall([c],  _w3=w3)()
coroutine 0 started       
# and just stays here

Look into uvloop as replacement for standard asyncio event loop

Add Support for Polygon ZKEVM

Multicall3 is deployed on polygon ZKEVM:

https://zkevm.polygonscan.com/address/0xcA11bde05977b3631167028862bE2a173976CA11

I have tried adding the corresponding chainId and Network to constants.py:

ZKEVM = 1101
Network.ZKEVM: 0xcA11bde05977b3631167028862bE2a173976CA11

This creates a bug within the manager.py file in web3py that returnsthe following error:

too many arguements want at most two

Functions returning only first return value

Hi!

I was wondering if I'm missing something or functions are not returning all return values using Multicall([Call()]).

Example call:

Multicall(
            [
                Call(
                    address,
                    [
                        "my_function(uint256,uint256)(bytes,bytes,bytes)",
                        argument_1,
                        argument_2,
                    ],
                    [[i, None]],
                )
                for i in range(99)
            ]
)

I was able to fix this for my use case by changing decoded to [decoded] here:

multicall.py/multicall/call.py

Line 27 in 089b390

in zip(self.returns, decoded)

If it's indeed a bug, I would be happy to test my solution and submit a PR.

Ganache BSC Incorrect number of arguments.

Running multi call query to local ganache chain raises

ContractLogicError: execution reverted: Incorrect number of arguments. Method 'eth_call' requires exactly 2 arguments. Request specified 3 arguments:

Common public BSC url works well. Single call works well with local ganache.

Command to run ganache:

ganache-cli -f https://bsc-dataseed.binance.org --chainId 56

At first I suggested the problem is somewhere with ganache, but I've managed to perform multicall with https://github.com/NFEL/eth_muticall repo using local ganache.

403 Client Error when using Multicall on Avalanche Mainnet.

Hello,

I am receiving the following error when attempting to use the avalanche public rpc on mainnet:

403 Client Error: Forbidden for url: https://api.avax.network/ext/bc/C/rpc [0]

I am using the example code as below:

from multicall import Call, Multicall
from web3 import Web3

TOKEN = '0x98e2060F672FD1656a07bc12D7253b5e41bF3876'
WHALE = '0x252332713864B50300d65580decD387cDD957C59'

def from_wei(value):
    return value / 1e18

w3 = Web3(Web3.HTTPProvider('https://api.avax.network/ext/bc/C/rpc'))

multi = Multicall(calls=[
    Call(TOKEN, ['balanceOf(address)(uint256)', WHALE], [('whale', from_wei)]),
], _w3=w3)

multi()

Need help: How do I use multicall with this method?

Hi, I'm trying to use multicall to fetch balancer pool balances.

The method to fetch pool balances seems to be: https://etherscan.io/address/0xba12222222228d8ba445958a75a0704d566bf2c8#readContract#F10

I'm struggling with the parsing of the return values - I could not find an example of decoding return values of type lists. To the best of my understanding, I've been trying to use multicall using the following Call method:

Call('0xBA12222222228d8Ba445958a75a0704d566BF2C8', ['getPoolTokens(bytes32)(((address),(uint256),uint256))', bytes(pool_id_here)],
[["reserves", None]])

however, this returns the following data -

(('0x0000000000000000000000000000000000000060',), (192,), 15238955)

which is of course, incorrect.

What is the correct function signature to use here?

Support switching off state overrides via an environment variable

Hardhat and ganache forks currently dont support state overrides. When running this library with a fork, we get execution revert whenever using the chainId as mainnet.

More details can be found here: https://stackoverflow.com/a/75823265/7655219

Would be nice to control the state overrides behaviour using an environment variable

Stucked at _get_semaphore() while using python to execute the script directly

I am trying to use this package to get the balance of ETH from multiple addresses, below is my code

node_url = 'xxxx'
w3 = Web3(Web3.HTTPProvider(node_url))
addresses = ["0xxxxx", "0xxxxx"]
def from_wei(status, value):
    print(status, value)
    return value / 1e18
multi = Multicall([
    Call(multi_call_address, ['getEthBalance(address)(uint256)', addresses[0]], [(addresses[0], from_wei)]),
    Call(multi_call_address, ['getEthBalance(address)(uint256)', addresses[1]], [(addresses[1], from_wei)]),
], require_success=False, _w3=w3)
results = multi()
print(results)

then I run the script from console,

python test.py

however, the code doesn't output anything and hang, I try to debug it and find the code gets stucked at

@lru_cache(maxsize=1)
def _get_semaphore() -> asyncio.Semaphore:
    return asyncio.Semaphore()

interestingly, the code can run correctly if I remove the decorator @lru_cache(maxsize=1), may I know how to run it corretly without removing the decorator? Thanks!

Update Web3 dependency

Using poetry add [email protected] and [email protected] conflicts happened:

Because multicall (0.8.0) depends on web3 (>=5.27,<5.29.dev0 || >5.31.0,<5.31.1 || >5.31.1,<5.31.2 || >5.31.2,<6.0)
and << poetry_package >> depends on web3 (6.13.0), multicall is forbidden.
So, because << poetry_package >> depends on multicall (0.8.0), version solving failed.

Is it possible to make an update of the package for web3 dep ?

BrokenProcessPool Error when trying to run Multicall

Hi everyone,

I am getting the below BrokenProcessPool runtime error when attempting to run the Multi call example script. I am not sure what is causing this and attempts to debug myself lead me down a low-level concurrent futures path that I am not familiar with. Note this error occurs regardless of whether I pass in w3 instance as parameter or set it as an os environment variable.

Can anyone assist here?

Thank you in advance.

Error:
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.

Example Script:

  w3 = Web3(Web3.HTTPProvider(alchemy_endpoint))
  def from_wei(value):
      return value / 1e18
  
  multi = Multicall([
      Call(MKR_TOKEN, ['balanceOf(address)(uint256)', MKR_WHALE], [['whale', from_wei]]),
      Call(MKR_TOKEN, ['balanceOf(address)(uint256)', MKR_FISH], [['fish', from_wei]]),
      Call(MKR_TOKEN, 'totalSupply()(uint256)', [['supply', from_wei]]),
  ], _w3 = w3)
  multi()

Full Error Message:

  /Users/hamzaalfadel/Desktop/Cred/Code/test_v2/venv/bin/python3.10 /Users/hamzaalfadel/Desktop/Cred/Code/test
  _v2/test_file_2.py
  
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/spawn.py", line 116, in spawn_main
      exitcode = _main(fd, parent_sentinel)
    File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/spawn.py", line 125, in _main
      prepare(preparation_data)
    File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/spawn.py", line 236, in prepare
      _fixup_main_from_path(data['init_main_from_path'])
    File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/spawn.py", line 287, in _fixup_main_from_path
      main_content = runpy.run_path(main_path,
    File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/runpy.py", line 269, in run_path
      return _run_module_code(code, init_globals, run_name,
    File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/runpy.py", line 96, in _run_module_code
      _run_code(code, mod_globals, init_globals,
    File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/runpy.py", line 86, in _run_code
      exec(code, run_globals)
    File "/Users/hamzaalfadel/Desktop/Cred/Code/test_v2/test_file_2.py", line 46, in <module>
      multi()
    File "/Users/hamzaalfadel/Desktop/Cred/Code/test_v2/venv/lib/python3.10/site-packages/multicall/multicall.py", line 57, in __call__
      response = await_awaitable(self.coroutine())
    File "/Users/hamzaalfadel/Desktop/Cred/Code/test_v2/venv/lib/python3.10/site-packages/multicall/utils.py", line 67, in await_awaitable
      return get_event_loop().run_until_complete(awaitable)
    File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
      return future.result()
    File "/Users/hamzaalfadel/Desktop/Cred/Code/test_v2/venv/lib/python3.10/site-packages/multicall/multicall.py", line 62, in coroutine
      batches = await gather([
    File "/Users/hamzaalfadel/Desktop/Cred/Code/test_v2/venv/lib/python3.10/site-packages/multicall/utils.py", line 82, in gather
      raise_if_exception_in(results)
    File "/Users/hamzaalfadel/Desktop/Cred/Code/test_v2/venv/lib/python3.10/site-packages/multicall/utils.py", line 78, in raise_if_exception_in
      raise_if_exception(obj)
    File "/Users/hamzaalfadel/Desktop/Cred/Code/test_v2/venv/lib/python3.10/site-packages/multicall/utils.py", line 74, in raise_if_exception
      raise obj
    File "/Users/hamzaalfadel/Desktop/Cred/Code/test_v2/venv/lib/python3.10/site-packages/multicall/multicall.py", line 97, in fetch_outputs
      _raise_or_proceed(e, len(calls), ConnErr_retries=ConnErr_retries)
    File "/Users/hamzaalfadel/Desktop/Cred/Code/test_v2/venv/lib/python3.10/site-packages/multicall/multicall.py", line 210, in _raise_or_proceed
      raise e
    File "/Users/hamzaalfadel/Desktop/Cred/Code/test_v2/venv/lib/python3.10/site-packages/multicall/multicall.py", line 82, in fetch_outputs
      args = await run_in_subprocess(get_args, calls, self.require_success)
    File "/Users/hamzaalfadel/Desktop/Cred/Code/test_v2/venv/lib/python3.10/site-packages/multicall/utils.py", line 70, in run_in_subprocess
      return await asyncio.get_event_loop().run_in_executor(process_pool_executor, coro, *args, **kwargs)
    File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/asyncio/base_events.py", line 818, in run_in_executor
      executor.submit(func, *args), loop=self)
    File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/concurrent/futures/process.py", line 705, in submit
      self._adjust_process_count()
    File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/concurrent/futures/process.py", line 683, in _adjust_process_count
      p.start()
    File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/process.py", line 121, in start
      self._popen = self._Popen(self)
    File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/context.py", line 284, in _Popen
      return Popen(process_obj)
    File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/popen_spawn_posix.py", line 32, in __init__
      super().__init__(process_obj)
    File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/popen_fork.py", line 19, in __init__
      self._launch(process_obj)
    File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/popen_spawn_posix.py", line 42, in _launch
      prep_data = spawn.get_preparation_data(process_obj._name)
    File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/spawn.py", line 154, in get_preparation_data
      _check_not_importing_main()
    File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/spawn.py", line 134, in _check_not_importing_main
      raise RuntimeError('''
  RuntimeError: 
          An attempt has been made to start a new process before the
          current process has finished its bootstrapping phase.
  
          This probably means that you are not using fork to start your
          child processes and you have forgotten to use the proper idiom
          in the main module:
  
              if __name__ == '__main__':
                  freeze_support()
                  ...
  
          The "freeze_support()" line can be omitted if the program
          is not going to be frozen to produce an executable.
  Traceback (most recent call last):
    File "/Users/hamzaalfadel/Desktop/Cred/Code/test_v2/test_file_2.py", line 46, in <module>
      multi()
    File "/Users/hamzaalfadel/Desktop/Cred/Code/test_v2/venv/lib/python3.10/site-packages/multicall/multicall.py", line 57, in __call__
      response = await_awaitable(self.coroutine())
    File "/Users/hamzaalfadel/Desktop/Cred/Code/test_v2/venv/lib/python3.10/site-packages/multicall/utils.py", line 67, in await_awaitable
      return get_event_loop().run_until_complete(awaitable)
    File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
      return future.result()
    File "/Users/hamzaalfadel/Desktop/Cred/Code/test_v2/venv/lib/python3.10/site-packages/multicall/multicall.py", line 62, in coroutine
      batches = await gather([
    File "/Users/hamzaalfadel/Desktop/Cred/Code/test_v2/venv/lib/python3.10/site-packages/multicall/utils.py", line 82, in gather
      raise_if_exception_in(results)
    File "/Users/hamzaalfadel/Desktop/Cred/Code/test_v2/venv/lib/python3.10/site-packages/multicall/utils.py", line 78, in raise_if_exception_in
      raise_if_exception(obj)
    File "/Users/hamzaalfadel/Desktop/Cred/Code/test_v2/venv/lib/python3.10/site-packages/multicall/utils.py", line 74, in raise_if_exception
      raise obj
    File "/Users/hamzaalfadel/Desktop/Cred/Code/test_v2/venv/lib/python3.10/site-packages/multicall/multicall.py", line 97, in fetch_outputs
      _raise_or_proceed(e, len(calls), ConnErr_retries=ConnErr_retries)
    File "/Users/hamzaalfadel/Desktop/Cred/Code/test_v2/venv/lib/python3.10/site-packages/multicall/multicall.py", line 210, in _raise_or_proceed
      raise e
    File "/Users/hamzaalfadel/Desktop/Cred/Code/test_v2/venv/lib/python3.10/site-packages/multicall/multicall.py", line 82, in fetch_outputs
      args = await run_in_subprocess(get_args, calls, self.require_success)
    File "/Users/hamzaalfadel/Desktop/Cred/Code/test_v2/venv/lib/python3.10/site-packages/multicall/utils.py", line 70, in run_in_subprocess
      return await asyncio.get_event_loop().run_in_executor(process_pool_executor, coro, *args, **kwargs)
  concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.

Strange issues using Multicall with Fuse RPCs

Update: After some investigation, it seems that the Parity client that Fuse uses doesn't handle overrides.
Out of curiosity, what is the reason that entire bytecode is sent with each request?

relevant pip list

multicall 0.7.0
web3 6.0.0b9

Came across a really strange issue. I'm trying to debug it myself, but also wanted to note it here to get more eyes on it.

For some reason, there's something Multicall is doing that is throwing off Fuse network RPCs. I've tried multiple endpoints with the same results. I intend on looping in the Fuse folks, but would like to actually understand what's going on.

Consider this simple script. Note that BSC Multicall() works fine, and Fuse single Call() works fine, but attempting to Multicall() returns an error. Out of 18 networks I routinely use multicall with, only Fuse is currently exhibiting this behavior.

from web3 import Web3
from multicall import Call, Multicall

fuse = Web3(Web3.HTTPProvider('https://rpc.fuse.io'))
bsc = Web3(Web3.HTTPProvider('https://bsc-dataseed.binance.org/'))

print(Multicall([
    Call('0x7C780b8A63eE9B7d0F985E8a922Be38a1F7B2141', 'nonce()(uint256)', [['bsc', None]]),
], _w3=bsc)())
print(Call('0xe26a8aC2936F338Fd4DAebA4BD22a7ec86465fE1', 'nonce()(uint256)', [['call', None]], _w3=fuse)())
print(Multicall([
    Call('0xe26a8aC2936F338Fd4DAebA4BD22a7ec86465fE1', 'nonce()(uint256)', [['multicall', None]]),
], _w3=fuse)())

The error is

ValueError: {'code': -32602, 'message': 'Invalid parameters: Expected from 1 to 2 parameters.', 'data': '"Got: 3"'}

No other flags, like require_success etc have any effect on it. The only thing I do know is that Multicall 0.4.0 does not have this issue. I'll continue to dig through it (including perhaps iterating through releases to determine when this may have started), but hoping some additional eyes could at least give me something to take back to the Fuse team if I need to.

ContractLogicError: execution reverted

Feat req: Allow for custom MULTICALL_ADDRESS and MULTICALL2_ADDRESS

multicall.py should support a custom multicall_map within the multicall.py file. This is trivial to implement and we are more than happy to provide a PR.

Getting 'message': 'invalid opcode: SHR' Error while doing mutlicall with historical block number

Hi,

I have written a script that takes multiple wallet/account address, token address & block number, then it prepares a multicall & call the function. For recent block numbers I get the proper result but for block number which are old like block numbers from 2017 or 2018 it gives invalid opcode SHR error. If I make the same contract function call via web3 library then it returns the result as expected.

I am getting error till block number 7256659 on ethereum.

Below is the sample code for Multicall on ETH chain:

from web3 import Web3
from web3.middleware import geth_poa_middleware
from multicall import Call, Multicall

addresses = [{'walletAddress': '0x536480A12fa7A2934872448752F864Cd0F9ba796', 'tokenAddress': '0xa74476443119a942de498590fe1f2454d7d4ac0d'}]

block_number = 4310318
GET_BALANCE_FN = "balanceOf(address)(uint256)"

def call_back_obj(success, value):
    if success is True:
        return value
    else:
        return False

node_provider = 'eth_node_provider_url'

def get_web3_instance():
    w3 = Web3(Web3.HTTPProvider(node_provider))
    w3.middleware_onion.inject(geth_poa_middleware, layer=0)
    return w3

_w3 = get_web3_instance()

def token_balance_handler(addresses, block_number=None):
    calls = []
    for address in addresses:
        contract_addr = address.get("tokenAddress")
        wallet_addr = address.get("walletAddress")
        calls.append(
            Call(
                contract_addr,
                [GET_BALANCE_FN, (wallet_addr)],
                [[f"{wallet_addr}-{contract_addr}", call_back_obj]],
            )
        )
    return Multicall(
                calls, _w3=_w3, block_id=block_number, require_success=False
            )

print(token_balance_handler(addresses, block_number)())

Sample code for contract function call via web3:

from web3 import Web3

node_provider_url = ""
# connect to Ethereum network
web3 = Web3(Web3.HTTPProvider(node_provider_url))

# set the address and ABI of the contract
contract_address = '0xa74476443119A942dE498590Fe1f2454d7D4aC0d'
contract_abi = [{'constant': True, 'inputs': [{'name': 'account', 'type': 'address'}], 'name': 'balanceOf', 'outputs': [{'name': '', 'type': 'uint256'}], 'payable': False, 'stateMutability': 'view', 'type': 'function'}]

# create a contract object
contract = web3.eth.contract(address=contract_address, abi=contract_abi)

# specify block number
block_number = 4310318

# call the balanceOf function at specific block number
account_address = '0x536480A12fa7A2934872448752F864Cd0F9ba796'
balance = contract.functions.balanceOf(account_address).call(block_identifier=block_number)

print(f'The balance of {account_address} at block {block_number} is {balance} tokens.')

Utilizing `[async_]simple_cache_middleware` to remove chainids mapping

Building off of #62, one thing that web3py supports in both sync and async versions is the caching of chainids by using the simple_cache_middleware.

Would it be an overreach to modify the default middlewares in utils.py and remove the internal mapping, where users will simply have to ensure they're using it (unless we want to check in utils and add it in a similar fashion as ClientTimeout)?

Is there a way to make this work with remote Infura nodes?

I'm just confused about how the provider is detected here--I think the project is great btw!

Making Multicall3 mapping naive

In previous versions of Multicall, when adding local support for new chains, I overrode utils.chainids, Constants.Network, and the MULTICALL_* structs. As Network is an Enum, I have to recreate the entire list -- granted, I only have to specify the chains I care about, this still gets a bit unwieldy since I periodically have to expand support within my own application.

Because Multicall3 doesn't deviate per network, would it make sense to allow for the execution to assume the address without requiring an explicit mapping, leaving it up to the user to determine whether it's available?

Since MC3 is backwards compatible with MC and MC2 and is included in every chain that MC/MC2 is (minus Kava, which we will fund mds to deploy shortly), which is therefore applied in multicall_map in multicall.py#L47-L52, I also feel like this should become the default operation, possibly obsoleting the need for any mapping at all.

Happy to take a stab at simplifying, but wanted to know if this would be a welcome change or not (and perhaps brainstorm a good implementation). Would be interested to hear whether anyone using a recent version of Multicall still relied on the

support python 3.11, 3.12

thank you very much

How to call a function from web3.eth.*

Hello, I need to use Multicall with web3.eth.getCode() but I don't know how to do that, I wanna check a list of addresses to know which is contract address. Could you help me. Thank you.

Error with ThreadPool 'Timeout context manager should be used inside a task'

Using Multicall more than once in a thread throw an error Timeout context manager should be used inside a task.
This error was previously reported here #41 and was fixed there #51
However, I am facing this error while using Multicall in a ThreadPool:

tp = ThreadPool(processes=1)
tp.apply_async(func=execute, args=(task, ))

def execute(task):
    w3 = Web3(Web3.HTTPProvider('http://rpc.url'))
    for v in Multicall(self.calls, _w3=w3)().values():
        print(v)

I am using multicall 0.7.2 and web3 6.4.0
Same results using web3 5.31.3 as recommend in issue #41

Getting Unauthorized while calling multicall with node provider secured with username & password

I am trying to do multicall with node provider which is secured with username & password.
I have full archived node setup for ETHEREUM. Please suggest how to resolve this problem

With this I am getting below error:

aiohttp.client_exceptions.ClientResponseError: 401, message='Unauthorized', url=URL("<node provider url>")

Below is the sample code for ETH chain:

import requests
from web3 import Web3
from web3.middleware import geth_poa_middleware
from multicall import Call, Multicall

addresses = [{'walletAddress': '0xee03B2D7E7903012d5d9238139De657493c69C7F', 'tokenAddress': '0x3819f64f282bf135d62168c1e513280daf905e06'}]

block_number = 16889104
GET_BALANCE_FN = "balanceOf(address)(uint256)"

def call_back_obj(success, value):
if success is True:
return value
else:
return False

node_provider = 'eth_node_provider_url'

username = 'my_username'
password = 'my_password'

session = requests.Session()
session.auth = (username, password)

def get_web3_instance():
w3 = Web3(Web3.HTTPProvider(node_provider, session=session))
w3.middleware_onion.inject(geth_poa_middleware, layer=0)
return w3

_w3 = get_web3_instance()

def token_balance_handler(addresses, block_number=None):
calls = []
for address in addresses:
contract_addr = address.get("tokenAddress")
wallet_addr = address.get("walletAddress")
calls.append(
Call(
contract_addr,
[GET_BALANCE_FN, (wallet_addr)],
[[f"{wallet_addr}-{contract_addr}", call_back_obj]],
)
)
return Multicall(
calls, _w3=_w3, block_id=block_number, require_success=False
)

print(token_balance_handler(addresses, block_number)())

Could not discover provider

I running your code and there was an error:

web3.exceptions.CannotHandleRequest: Could not discover provider while making request: method:eth_chainId
params:()

Thank you

Add support for additional arguments in the call function.

Would it be possible to implement a way to pass additional field into the call function. Right now you are passing {to: address, data: data} into the call but there are some use cases for additional arguments. For example a few of the contracts I interact need a from argument as well.

Example:

self.w3.eth.call({'to': self.target, 'data': calldata, 'from': address})

Support for chain 10 Optimism

Support for Optimism was added in #20 but was later reverted in #22. Apparently there were breaking changes.

I tested multicall on Optimism and it worked well for me. However I'm unsure if I should submit a PR to reintroduce this change.

PS: Only tested with Multicall v1
EDIT: Now also tested with Multicall v2