Code Monkey home page Code Monkey logo

eth-tester's Issues

Tools for testing chain reorgs

What was wrong?

It would be nice to be able to test chain-reorg in a simple manner. I'm not sure where in the stack the fits best but something like

EthereumTester.mine_reorg(reorg_depth=1)

might work.

Main use case would be testing scenarios where state is updated from block contents and has to be updated when the chain changes.

dependency cycle with py-evm backend

  • Version: 0.1.0b23
  • Python: 3.6
  • OS: osx

What was wrong?

I specify eth-tester[py-evm]>=0.1.0-beta.23,<1.0 in my dependencies. pip install -r requirements.txt works fine with that.

However, when trying to run python setup.py develop on my package I get the following error:

error: eth-tester 0.1.0b23 is installed but eth-tester==0.1.0b21 is required by {'py-evm'}

eth-tester requires py-evm 0.2.0a14, but this one in return requires eth-tester 0.1.0b21: https://github.com/ethereum/py-evm/blob/cc877851d4e110eabfba1bfe544bfe187b51ebea/setup.py#L29

How can it be fixed?

Fix the dependency cycle.

Coverage and Gas Profiling

Most likely not the right project for this, but I couldn't think of the best project (between eth-tester, web3.py, and populus) for this feature suggestion when it comes down to implementation.

There should be a standard method to measure meaningful code-coverage metrics (Statement, Branch, and MC/DC ideally) and profile gas costs during runtime execution on EVM bytecode. Doing so on EVM bytecode can be more accurate than against source code (see here), and using a common, low-level implementation we can avoid duplication of efforts between the different langauges. With bytecode-level metrics, one can simply collect the results and figure out the bytecode to source code mapping for a specific language for reporting purposes via whatever collation method makes sense, which might be integrated into a compiler or external tool for presentation to the user.

Gas cost profiling might not make sense in this context, because IIRC they should be more or less standard for a given set of opcodes (basically everything except external calls could be analyzed via static analysis). However, I think it might actually be useful to profile at this level given that the run-time execution would figure out the costs of external calls. Collecting this cost for each transaction made since a baseline (e.g. start of testing) could report min/max/average costs for these external calls.

The best reasoning I can give for why this would be good to include in eth-tester is that it can provide this functionality at the transaction-level (basically "runtime"), which is more appropiate for testing than via static analysis methods or implementations alongside source code (like solidity-coverage).

If there is indeed tools for doing this already that are easy to use and embeddable in the testing process, please let me know. I need them for my current project.

Runtime error

  • Version: 0.1.0b23
  • Python: 3.6.4
  • OS: MacOS Sierra 10.12.16

What was wrong?

When using eth_tester as part of our test suite, I get some runtime errors as exemplified by the following stack trace:

    price,
../../../../.pyenv/versions/3.6.4/lib/python3.6/site-packages/web3/contract.py:1311: in transact_with_contract_function
    txn_hash = web3.eth.sendTransaction(transact_transaction)
../../../../.pyenv/versions/3.6.4/lib/python3.6/site-packages/web3/eth.py:244: in sendTransaction
    [transaction],
../../../../.pyenv/versions/3.6.4/lib/python3.6/site-packages/web3/manager.py:103: in request_blocking
    response = self._make_request(method, params)
../../../../.pyenv/versions/3.6.4/lib/python3.6/site-packages/web3/manager.py:86: in _make_request
    return request_func(method, params)
../../../../.pyenv/versions/3.6.4/lib/python3.6/site-packages/web3/middleware/gas_price_strategy.py:18: in middleware
    return make_request(method, params)
../../../../.pyenv/versions/3.6.4/lib/python3.6/site-packages/web3/middleware/formatting.py:21: in middleware
    response = make_request(method, formatted_params)
../../../../.pyenv/versions/3.6.4/lib/python3.6/site-packages/web3/middleware/attrdict.py:18: in middleware
    response = make_request(method, params)
../../../../.pyenv/versions/3.6.4/lib/python3.6/site-packages/web3/middleware/formatting.py:21: in middleware
    response = make_request(method, formatted_params)
../../../../.pyenv/versions/3.6.4/lib/python3.6/site-packages/web3/middleware/normalize_errors.py:9: in middleware
    result = make_request(method, params)
../../../../.pyenv/versions/3.6.4/lib/python3.6/site-packages/web3/middleware/validation.py:43: in middleware
    return make_request(method, post_validated_params)
../../../../.pyenv/versions/3.6.4/lib/python3.6/site-packages/web3/middleware/formatting.py:21: in middleware
    response = make_request(method, formatted_params)
../../../../.pyenv/versions/3.6.4/lib/python3.6/site-packages/web3/providers/eth_tester/middleware.py:320: in middleware
    return make_request(method, [filled_transaction] + params[1:])
../../../../.pyenv/versions/3.6.4/lib/python3.6/site-packages/web3/middleware/fixture.py:12: in middleware
    return make_request(method, params)
../../../../.pyenv/versions/3.6.4/lib/python3.6/site-packages/web3/middleware/formatting.py:21: in middleware
    response = make_request(method, formatted_params)
../../../../.pyenv/versions/3.6.4/lib/python3.6/site-packages/web3/providers/eth_tester/main.py:46: in make_request
    response = delegator(self.ethereum_tester, params)
cytoolz/functoolz.pyx:232: in cytoolz.functoolz.curry.__call__
    ???
../../../../.pyenv/versions/3.6.4/lib/python3.6/site-packages/web3/providers/eth_tester/defaults.py:36: in call_eth_tester
    return getattr(eth_tester, fn_name)(*fn_args, **fn_kwargs)
../../../../.pyenv/versions/3.6.4/lib/python3.6/site-packages/eth_tester/main.py:78: in func_wrapper
    self.mine_block()
../../../../.pyenv/versions/3.6.4/lib/python3.6/site-packages/eth_tester/main.py:394: in mine_block
    block_hash = self.mine_blocks(1, coinbase=coinbase)[0]
../../../../.pyenv/versions/3.6.4/lib/python3.6/site-packages/eth_tester/main.py:389: in mine_blocks
    self._process_block_logs(block)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <eth_tester.main.EthereumTester object at 0x113774748>, block = {'difficulty': 131200, 'extra_data': '0x0000000000000000000000000000000000000000000000000000000000000000', 'gas_limit': 998051069, 'gas_used': 266681, ...}

    def _process_block_logs(self, block):
>       for _, filter in self._log_filters.items():
E       RuntimeError: dictionary changed size during iteration

../../../../.pyenv/versions/3.6.4/lib/python3.6/site-packages/eth_tester/main.py:401: RuntimeError

Ideas?

README.md contains errors, need check and fix

I have git clone latest code , and found issues for README.md
My dev software is Windows+Python3.5

Issue1

Development
pip install -e . -r requirements-dev.txt
but i can't find requirements-dev.txt, and got below error:
Could not open requirements file: [Errno 2] No such file or directory: 'requirements-dev.txt'

Issue2

below are part of Quick Start in README.md

t.send_transaction({'from': '0x82A978B3f5962A5b0957d9ee9eEf472EE55B42F1', 'to': '0x7d577a597B2742b498Cb5Cf0C26cDCD726d39E6e', 'gas': 21000, 'value': 1})
'0x140c1da1370a908e4c0f7c6e33bb97182011707c6a9aff954bef1084c8a48b25'
t.get_transaction_by_hash(0x140c1da1370a908e4c0f7c6e33bb97182011707c6a9aff954bef1084c8a48b25')
t.t.get_transaction_by_hash(0x...) missing one '

Issue3

t.get_block_by_numbers(1)
this func has been replaced by t.get_block_by_number(1)

I hope anyone can check README.md, and execute all process once, to see if exists any other erros.

Web3 - ValueError: RPC Endpoint has not been implemented: eth_estimateGas

Got a weird error with eth-tester:

$ py.test test_contract.py
test_contract.py:2: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
conftest.py:32: in deploy
    tx_hash = self.factory.deploy(args=args, **kwargs)
../../../../.pyenv/versions/3.6.4/envs/test/lib/python3.6/site-packages/web3/contract.py:268: in deploy
    txn_hash = cls.web3.eth.sendTransaction(deploy_transaction)
../../../../.pyenv/versions/3.6.4/envs/test/lib/python3.6/site-packages/web3/eth.py:212: in sendTransaction
    get_buffered_gas_estimate(self.web3, transaction),
../../../../.pyenv/versions/3.6.4/envs/test/lib/python3.6/site-packages/web3/utils/transactions.py:60: in get_buffered_gas_estimate
    gas_estimate = web3.eth.estimateGas(gas_estimate_transaction)
../../../../.pyenv/versions/3.6.4/envs/test/lib/python3.6/site-packages/web3/eth.py:254: in estimateGas
    [transaction],
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <web3.manager.RequestManager object at 0x7f26a6b66208>, method = 'eth_estimateGas'
params = [{'data': '0x6060604052606060405190810160405280602481526020017f4461707044657673205368696c6c2d612d436f696e20576f726b736...baf09a012e63fb4bce3f65de49a48a38ebdd611cf6d4b7760029000000000000000000000000000000000000000000000000000000000000000a'}]

    def request_blocking(self, method, params):
        """
            Make a synchronous request using the provider
            """
        response = self._make_request(method, params)
    
        if "error" in response:
>           raise ValueError(response["error"])
E           ValueError: RPC Endpoint has not been implemented: eth_estimateGas

../../../../.pyenv/versions/3.6.4/envs/test/lib/python3.6/site-packages/web3/manager.py:102: ValueError

Setup:

  • Version: 0.1.0b18
  • Python: 3.6.4
  • OS: linux
$ pip freeze
attrdict==2.0.0
attrs==17.4.0
certifi==2018.1.18
chardet==3.0.4
cytoolz==0.9.0
eth-abi==1.0.0b0
eth-account==0.1.0a2
eth-hash==0.1.0a3
eth-keyfile==0.5.1
eth-keys==0.2.0b2
eth-rlp==0.1.0a2
eth-tester==0.1.0b18
eth-utils==1.0.0b1
hexbytes==0.1.0b0
idna==2.6
lru-dict==1.1.6
pluggy==0.6.0
py==1.5.2
pycryptodome==3.4.11
pysha3==1.0.2
pytest==3.4.1
requests==2.18.4
rlp==0.6.0
semantic-version==2.6.0
six==1.11.0
toolz==0.9.0
urllib3==1.22
web3==4.0.0b9

Replicate

'../contracts.json' is output of solc --combined-json abi,bin,bin-runtime ../contracts/*.sol

conftest.py:

import pytest

from web3 import Web3
from web3.providers.eth_tester import EthereumTesterProvider
from web3.contract import ImplicitContract
import json

w3 = Web3(EthereumTesterProvider())

@pytest.fixture
def a():
    return w3.personal.listAccounts

@pytest.fixture
def contracts():
    with open('../contracts.json', 'r') as f:
        contracts = json.loads(f.read())
    return contracts['contracts']

@pytest.fixture
def Contract(contracts):

    class Contract:
        def __init__(self, contract_name):
            interface = contracts[contract_name]
            self.abi = interface['abi']
            self.bin = interface['bin']
            self.runtime = interface['bin-runtime']
            self.factory = w3.eth.contract(abi=self.abi, bytecode=self.bin)

        def deploy(self, *args, **kwargs):
            tx_hash = self.factory.deploy(args=args, **kwargs)
            tx_receipt = w3.eth.getTransactionReceipt(tx_hash)
            address = tx_receipt['contractAddress']
            return w3.eth.contract(self.abi, address,
                    ContractFactoryClass=ImplicitContract)


    return Contract

test_contract.py:

def test_contract(Contract):
    Contract('contracts/ContractA.sol:ContractA').deploy()

Accounts are not persisted when using LevelDB

  • py-evm Version: 0.20.a17
  • OS: osx
  • Python Version (python --version): 3.6.5
  • Environment (output of pip freeze):
    ..
    eth-abi==1.1.1
    eth-account==0.2.2
    eth-bloom==1.0.0
    eth-hash==0.1.2
    eth-keyfile==0.5.1
    eth-keys==0.2.0b3
    eth-rlp==0.1.2
    eth-tester==0.1.0b25
    eth-utils==1.0.3
    hexbytes==0.1.0
    ..
    py-ecc==1.4.2
    py-evm==0.2.0a17
    pycparser==2.18
    pycryptodome==3.6.1
    pyethash==0.1.27
    Pygments==2.2.0
    pypandoc==1.4
    pysha3==1.0.2
    pytest==3.5.1
    python-dateutil==2.7.3
    requests==2.18.4
    rlp==1.0.1
    ..

What was wrong?

When using LevelDB account keys are not persisted:

       # LevelDB will set all account balances equal to 0
       #db = ChainDB(LevelDB('./testleveldb.db'))

       # MemoryDB correctly persists account balances to 1000000000000000000000000
        db = ChainDB(MemoryDB())

        self.account_keys = get_default_account_keys()
        genesis_state = generate_genesis_state(self.account_keys)
        self.chain = chain_class.from_genesis(db, genesis_params, genesis_state)

Testing code to retrieve balances:

tester = EthereumTester(backend)
accounts = tester.get_accounts()
print(accounts)
print(list(tester.get_balance(account) for account in accounts))

Feature: Configurable transaction failure handler

Currently, when a transaction in eth-tester throws an exception (at least in pytest), it provides the entire backtrace of the failure, which is mostly showing the layers of calls back to py-evm, most of which is sort of meaningless to the user.

It would be more helpful to a user of eth-tester if there was a handler for this behavior that could given the execution trace or something so tools like @jacqueswww's vdb could handle this exception, or more generally a better trace could be produced (the EVM stack trace instead of the Python trace).

If you make this configurable, different compilers could provide language-specific behavior from giving the source code mapping, which eth-tester could use to retrace the exact line in the source program that caused the issue. This would be MUCH more helpful to debug. The default could just be an EVM opcode printer that shows the raw stack trace.

This is my thinking from sort of a high level user what they would want to see. May make more sense in the pytest-ethereum tool, but I could see this be more broadly useful and enable a general-purpose debugging API for other tools to leverage.

Snapshot state to a genesis file

What was wrong?

Inspired by #88 (comment)

People want to pick up chain state from somewhere they left off in a test chain. It would be cool to use this to set up integration tests for non-eth-tester nodes too!

How can it be fixed?

It would be sweet to have an easy way to snapshot state into a genesis file (which can then be reimported using #129).

I think a good API is: genesis_dict = tester.genesis_snapshot(). First step is consensus on that.

Update gas limit

What was wrong?

Current default block gas limit is 3.14m gas. Mainchain gas limit has been around 8m for a while now, which means it is possible to deploy larger contracts than what is currently testable. Lowest testnet block gas limit is 7m gas.

How can it be fixed?

Until #129 is implemented to permanently solve the configuration issue, I would suggest that the arbitrarily-chosen default be at least 2x'd to 6.28m gas (2 Pi). This would ensure a more realistic block gas limit for testing, while retaining a sufficient margin of safety against the current limits (which can fluctuate slightly)

py-evm backend ignores fork setup

What was wrong?

As of alpha 26, there is no clean way to define the VM configuration at runtime. MainnetTesterChain.configure_forks is broken: it sets the configuration on the vm instance, and then get_vm_class_for_block_number ignores vm_configuration on the instance, using the class's vm_configuration instead.

How can it be fixed?

No one noticed that fork choice stuff isn't working. Maybe just neuter the API to officially make it do nothing, and we can re-add the functionality when it's requested.

Also, probably remove MainnetTesterChain.configure_forks from py-evm since it's described as "footgun" and isn't working anyway.

Calling Filter.remove(*values) with no values produces unhandled error

  • Version: 0.1.0b27
  • Python: 3.6.5
  • OS: linux

What was wrong?

I am using the snapshot functionality to catch and revert a TransactionFailed error when automining transactions. I often (70% of the time in my test suite) run into an issue when reverting to a snapshot where it catches the following unhandled exception in this function call:

eth-tester/eth_tester/utils/filters.py

Lines 47 to 60 in c51beb5

def remove(self, *values):
try:
values_to_remove = set(values)
except TypeError:
# log filters are dicts which are not hashable
values_to_remove = values
queued_values = self.get_changes()
self.values = [
value
for value
in self.get_all()
if value not in values_to_remove
] 

    self._t.revert_to_snapshot(self._snapshot_id)
eth_tester/main.py:543: in revert_to_snapshot
    self._revert_log_filter(log_filter)
eth_tester/main.py:590: in _revert_log_filter
    filter.remove(*values_to_remove)
eth_tester/utils/filters.py:62: in remove
    in self.get_all()

---------------------------------------------------------------------------------------------

.0 = <tuple_iterator object at 0x7f2585dfdef0>

    value
    for value
    in self.get_all()
>   if value not in values_to_remove
            ]
E           TypeError: unhashable type: 'dict'

eth_tester/utils/filters.py:63: TypeError

When this error occurs, the following variables of interest are set:

values = ()  # Empty tuple

# Tuple of 1 dict
self.get_all() = ({'type': 'mined', 'log_index': 0, 'transaction_index': 0, 'transaction_hash': b'_\x0f;\xd1n/@pq\xcaL\xea\xd1\x11\xe5\x86\x03\xc3\xe9.\xd4\x88\x8b9\xfaHD\xc3\xa3;V\x1d', 'block_hash': b'\xcf\x9b\xec\xcaD[\x7fm\xc0KJ\x1c{\x80.Z\t\xa8\xb7fV\xe5\xb9X\x05\xe0N\x85E\xb0\xc0.', 'block_number': 2, 'address': b'\xf2\xe2F\xbbv\xdf\x87l\xef\x8b8\xae\x84\x13\x0fOU\xde9[', 'data': b'', 'topics': (b'\x8b\xe0\x07\x9cS\x16Y\x14\x13D\xcd\x1f\xd0\xa4\xf2\x84\x19I\x7f\x97"\xa3\xda\xaf\xe3\xb4\x18okdW\xe0', b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00+Z\xd5\xc4y\\\x02e\x14\xf81|z!^!\x8d\xcc\xd6\xcf', b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00+Z\xd5\xc4y\\\x02e\x14\xf81|z!^!\x8d\xcc\xd6\xcf')},)

values_to_remove = set()  # Empty set

How can it be fixed?

I haven't been able to replicate this issue as it is inconsistent for me during runtime in my testing tool, but I think handling when values is unset is fairly obvious, since this basic Python also raises the same error:

[v for v in ({},) if v not in set()]

TypeError: unhashable type: 'dict'

I can submit a quick fix for this, but I need a little more instruction on how to handle this scenario

I am thinking of suggesting the following:

    queued_values = self.get_changes()
    if values_to_remove:
        self.values = [
            value
            for value
            in self.get_all()
            if value not in values_to_remove
        ]
        for value in queued_values:
            if value in values_to_remove:
                continue
           self.queue.put_nowait(value)
    else:
        self.values = self.get_all()
        for value in queued_values:
           self.queue.put_nowait(value)

Something smarter with filtering probably would work too.

PyEthereum2.1 backend tests need a different environment

  • Version: 0.1.0-beta.26
  • OS: CI

What was wrong?

See https://circleci.com/gh/ethereum/eth-tester/891?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-build-link

==================================== ERRORS ====================================
_____________ ERROR collecting tests/backends/test_pyethereum21.py _____________
tests/backends/test_pyethereum21.py:5: in <module>
    from eth_tester import (
eth_tester/__init__.py:4: in <module>
    from .main import (  # noqa: F401
eth_tester/main.py:29: in <module>
    from eth_tester.backends import (
eth_tester/backends/__init__.py:18: in <module>
    from .pyethereum.v20 import (
eth_tester/backends/pyethereum/v20/__init__.py:6: in <module>
    from .main import (  # noqa: F401
eth_tester/backends/pyethereum/v20/main.py:56: in <module>
    from ethereum.tools.tester import (
.tox/py36-pyethereum21/lib/python3.6/site-packages/ethereum/tools/__init__.py:1: in <module>
    from ethereum.tools import keys, new_statetest_utils, testutils, tester, _solidity
.tox/py36-pyethereum21/lib/python3.6/site-packages/ethereum/tools/keys.py:9: in <module>
    scrypt = __import__('scrypt')
.tox/py36-pyethereum21/lib/python3.6/site-packages/scrypt/__init__.py:1: in <module>
    from .scrypt import *
.tox/py36-pyethereum21/lib/python3.6/site-packages/scrypt/scrypt.py:15: in <module>
    _scrypt = cdll.LoadLibrary(imp.find_module('_scrypt')[1])
/usr/local/lib/python3.6/ctypes/__init__.py:426: in LoadLibrary
    return self._dlltype(name)
/usr/local/lib/python3.6/ctypes/__init__.py:348: in __init__
    self._handle = _dlopen(self._name, mode)
E   OSError: libcrypto.so.1.0.0: cannot open shared object file: No such file or directory

How can it be fixed?

Force an environment that has the libcrypto.so.1.0.0 library available. See some similar (but not exactly the same) environment issues in web3.py: ethereum/web3.py@8327480

Make block gas limit configurable

  • Version: 0.1.0-beta.24
  • Python: 3.6
  • OS: osx

What was wrong?

Currently the block gas limit is hard coded:

eth-tester/eth_tester/backends/pyevm/main.py

Line 66 in b9ad39a

GENESIS_GAS_LIMIT = 3141592

This fails some tests with larger contracts and should be made configurable. Also increasing the default might make sense, the mainnet blocks have been bigger than 6 million gas for some time.

How can it be fixed?

Make the block gas limit configurable and think about increasing the default value.

Handle pending transactions with a tx pool

  • Version: 0.1.0-beta.16
  • Python: all
  • OS: all

What was wrong?

Eth tester does not handle pending transactions correctly when disabling auto mining transactions:

Sending a transaction with the same nonce causes an invalid nonce error:

from eth_tester import (
    EthereumTester,
    PyEVMBackend,
)

backend = PyEVMBackend()
t = EthereumTester(backend=backend)
t.disable_auto_mine_transactions()
accounts = t.get_accounts()
tx = t.send_transaction({'from': accounts[0], 'to': accounts[1], 'gas': 21000, 'value': 1, 'nonce': 0})
tx2 = t.send_transaction({'from': accounts[0], 'to': accounts[1], 'gas': 21000, 'value': 1, 'nonce': 0})
>>> evm.exceptions.ValidationError: Invalid transaction nonce

How can it be fixed?

The issue is that send_transaction and send_raw_transaction immediately execute on the vm's state and thereby increment the valid nonce.

Things to consider for the fix:

  • A tx pool needs to be managed at the eth tester level
  • It should consider the rules for replacing transactions. e.g. transactions with existing nonce should overwrite #57
  • Will the tx pool work for every backend?
  • eth-tester exposes both send_transaction and send_raw_transaction both of which have different code paths. We need to ensure that we treat them the same as far as the tx pool is concerned.
  • eth-tester does not know about the outcome of a transaction (e.g. for figuring out what nonce a transaction will execute at). Could circumvent this by using get_transaction_by_hash after executing the transaction.

A very rudimentary fix:

if self.auto_mine_transactions:
    self.mine_block()
else:
    snapshot = self.take_snapshot()
    # If a pending transaction with the same nonce exists, overwrite it?
    self._pending_transactions.append(transaction)
    self.revert_to_snapshot(snapshot)

def enable_auto_mine_transactions(self):
    self.auto_mine_transactions = True
    return [self.send_transaction(tx) for tx in self._pending_transactions]

@carver @pipermerriam feel free to modify the issue to clarify things 🙂

MoackBackend returned hash value can't found valid trans

  • Version: 0.1.0b32
  • Python: 3.5
  • OS: win

What was wrong?

I got below error:

Traceback (most recent call last):
  File "D:\eth-tester\eth_tester\backends\mock\main.py", line 244, in get_transaction_receipt
    receipt = self.receipts[transaction_hash]
KeyError: b'\xf3\xbc\xde#\xb2U\x05H\xe0\x98.\xae\xb5}\xb3\x80!\xd4\xca\x0f6\xd8\xc9\xb1\x05\xa3<&\x9eR\x91\x9b'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:/eth-tester/sample.py", line 10, in <module>
    print(t.get_transaction_receipt('0xf3bcde23b2550548e0982eaeb57db38021d4ca0f36d8c9b105a33c269e52919b'))
  File "D:\eth-tester\eth_tester\main.py", line 307, in get_transaction_receipt
    raw_receipt = self.backend.get_transaction_receipt(raw_transaction_hash)
  File "D:\eth-tester\eth_tester\backends\mock\main.py", line 247, in get_transaction_receipt
    "No transaction found for hash: {0}".format(transaction_hash)
eth_tester.exceptions.TransactionNotFound: No transaction found for hash: b'\xf3\xbc\xde#\xb2U\x05H\xe0\x98.\xae\xb5}\xb3\x80!\xd4\xca\x0f6\xd8\xc9\xb1\x05\xa3<&\x9eR\x91\x9b'

Process finished with exit code 1

My code is below:

from eth_tester import EthereumTester, MockBackend


t = EthereumTester(MockBackend())
print(t.get_accounts())
print(t.get_balance('0xaBbACadABa000000000000000000000000000000'))
print(t.send_transaction({'from': '0x82A978B3f5962A5b0957d9ee9eEf472EE55B42F1', \
                          'to': '0x7d577a597B2742b498Cb5Cf0C26cDCD726d39E6e', 'gas': 21000, 'value': 1}))
print(t.get_transaction_receipt('0xf3bcde23b2550548e0982eaeb57db38021d4ca0f36d8c9b105a33c269e52919b'))

How can it be fixed?

I wonder if this issue only exist on MockBackend or on PyEVM both?

Support nonce in transactions

What was wrong?

When sending transactions from web3.py buildTransaction() with nonce specified, I got got an error from validate_transaction():

    def validate_transaction(value, txn_type):
        if txn_type not in ALLOWED_TRANSACTION_TYPES:
            raise TypeError("the `txn_type` parameter must be one of send/call/estimate")
        if not is_dict(value):
            raise ValidationError("Transaction must be a dictionary.  Got: {0}".format(type(value)))
    
        unknown_keys = tuple(sorted(set(value.keys()).difference(
            TRANSACTION_TYPE_INFO[txn_type],
        )))
        if unknown_keys:
            raise ValidationError(
                "Only the keys '{0}' are allowed.  Got extra keys: '{1}'".format(
                    "/".join(tuple(sorted(TRANSACTION_TYPE_INFO[txn_type]))),
>                   "/".join(unknown_keys),
                )
            )
E           eth_tester.exceptions.ValidationError: Only the keys 'data/from/gas/gas_price/to/value' are allowed.  Got extra keys: 'nonce'

How can it be fixed?

Add support for nonce in transaction.

This commit should be reverted when the issue is fixed.
ethereum/web3.py@07c446e

Write a web3 backend

Write a backend which uses web3. This sets up the ability to test against real nodes (with limited functionality since we can't time travel a real node).

Remove pyethereum backends

What was wrong?

The pyethereum codebase is unmaintained.

How can it be fixed?

Drop both of the pyethereum based backends.

Transaction not mined when running multiple unit tests (PyEthereum21Backend)

  • Version: 0.1.0b26
  • Python: 3.6
  • OS: linux

Suppose the following django project structure:

  • api
    • tests.py
  • client
    • tests.py

Also suppose the following TestCase setup for both test files:

    def setUp(self):
        # Retrieve a fresh client copy
        self.client = MyClient(
            provider=self.provider, ...
        )

        super().setUp()

    def tearDown(self):
        # Revert to the clean state for every test case
        self.eth_tester.revert_to_snapshot(self.clean_state_snapshot)
        super().tearDown()

    @classmethod
    def setUpTestData(cls):
        """Only once setup"""
        cls.eth_tester = EthereumTester(backend=PyEthereum21Backend())
        cls.provider = EthereumTesterProvider(cls.eth_tester)
        cls.web3 = Web3(cls.provider)

        # deploying smart contracts etc ...

        cls.clean_state_snapshot = cls.eth_tester.take_snapshot()

Below is the output of running each TestCase on its own:

# client.tests output
Failed to import bitcoin. This is not a fatal error but does
mean that you will not be able to determine the address from
your wallet file.
Using existing test database for alias 'default'...
System check identified no issues (0 silenced).
Initializing chain from provided state
Initializing chain from provided state
INFO:eth.block	Block pre-sealed, 21000 gas used 
INFO:eth.chain	Adding to head head=c6745cf3
Saved 3 address change logs
INFO:eth.chain	Added block 1 (5f5f608a) with 1 txs and 21000 gas 
INFO:eth.block	Block pre-sealed, 470791 gas used 
INFO:eth.chain	Adding to head head=5f5f608a
Saved 2 address change logs
INFO:eth.chain	Added block 2 (3c04531f) with 1 txs and 470791 gas 
INFO:eth.block	Block pre-sealed, 1641161 gas used 
INFO:eth.chain	Adding to head head=3c04531f
Saved 2 address change logs
INFO:eth.chain	Added block 3 (94a8e4d5) with 1 txs and 1641161 gas 
INFO:eth.block	Block pre-sealed, 505467 gas used 
INFO:eth.chain	Adding to head head=94a8e4d5
Saved 2 address change logs
INFO:eth.chain	Added block 4 (92e1a399) with 1 txs and 505467 gas 
INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
.INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
.INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
INFO:eth.block	Block pre-sealed, 98148 gas used 
INFO:eth.chain	Adding to head head=08aa138d
Saved 7 address change logs
INFO:eth.chain	Added block 6 (c1174bae) with 1 txs and 98148 gas 
.INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
.INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
INFO:eth.block	Block pre-sealed, 98148 gas used 
INFO:eth.chain	Adding to head head=08aa138d
Saved 7 address change logs
INFO:eth.chain	Added block 6 (c1174bae) with 1 txs and 98148 gas 
.INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
INFO:eth.block	Block pre-sealed, 98020 gas used 
INFO:eth.chain	Adding to head head=08aa138d
Saved 7 address change logs
INFO:eth.chain	Added block 6 (d4d6425a) with 1 txs and 98020 gas 
.
----------------------------------------------------------------------
Ran 6 tests in 1.762s

OK
Preserving test database for alias 'default'..

and for the api tests:

Failed to import bitcoin. This is not a fatal error but does
mean that you will not be able to determine the address from
your wallet file.
Using existing test database for alias 'default'...
System check identified no issues (0 silenced).
Initializing chain from provided state
Initializing chain from provided state
INFO:eth.block	Block pre-sealed, 21000 gas used 
INFO:eth.chain	Adding to head head=c6745cf3
Saved 3 address change logs
INFO:eth.chain	Added block 1 (5f5f608a) with 1 txs and 21000 gas 
INFO:eth.block	Block pre-sealed, 470791 gas used 
INFO:eth.chain	Adding to head head=5f5f608a
Saved 2 address change logs
INFO:eth.chain	Added block 2 (3c04531f) with 1 txs and 470791 gas 
INFO:eth.block	Block pre-sealed, 1641161 gas used 
INFO:eth.chain	Adding to head head=3c04531f
Saved 2 address change logs
INFO:eth.chain	Added block 3 (94a8e4d5) with 1 txs and 1641161 gas 
INFO:eth.block	Block pre-sealed, 505467 gas used 
INFO:eth.chain	Adding to head head=94a8e4d5
Saved 2 address change logs
INFO:eth.chain	Added block 4 (92e1a399) with 1 txs and 505467 gas 
INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
.INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
.INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
.INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
INFO:eth.block	Block pre-sealed, 98148 gas used 
INFO:eth.chain	Adding to head head=08aa138d
Saved 7 address change logs
INFO:eth.chain	Added block 6 (2d599c9a) with 1 txs and 98148 gas 
.INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
.INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
INFO:eth.block	Block pre-sealed, 98148 gas used 
INFO:eth.chain	Adding to head head=08aa138d
Saved 7 address change logs
INFO:eth.chain	Added block 6 (2d599c9a) with 1 txs and 98148 gas 
.
----------------------------------------------------------------------
Ran 6 tests in 1.652s

OK
Preserving test database for alias 'default'...

Everything looks ok.
The problem appears when I try to run both test cases at once, using a single python3 manage.py test command.

What will happen is that inside my code, a call to web3.eth.waitForTransactionReceipt will always timeout or, a call to EthereumTester.get_transaction_receipt will always raise a TransactionNotFound

Below is the output of executing both test cases together, and me pressing ctrl+c to terminate when the call to web3.eth.waitForTransactionReceipt blocks ( setting the timeout to 1 second for example will just make all my unit tests fail )

Failed to import bitcoin. This is not a fatal error but does
mean that you will not be able to determine the address from
your wallet file.
Using existing test database for alias 'default'...
System check identified no issues (0 silenced).
Initializing chain from provided state
Initializing chain from provided state
INFO:eth.block	Block pre-sealed, 21000 gas used 
INFO:eth.chain	Adding to head head=c6745cf3
Saved 3 address change logs
INFO:eth.chain	Added block 1 (5f5f608a) with 1 txs and 21000 gas 
INFO:eth.block	Block pre-sealed, 470791 gas used 
INFO:eth.chain	Adding to head head=5f5f608a
Saved 2 address change logs
INFO:eth.chain	Added block 2 (3c04531f) with 1 txs and 470791 gas 
INFO:eth.block	Block pre-sealed, 1641161 gas used 
INFO:eth.chain	Adding to head head=3c04531f
Saved 2 address change logs
INFO:eth.chain	Added block 3 (94a8e4d5) with 1 txs and 1641161 gas 
INFO:eth.block	Block pre-sealed, 505467 gas used 
INFO:eth.chain	Adding to head head=94a8e4d5
Saved 2 address change logs
INFO:eth.chain	Added block 4 (92e1a399) with 1 txs and 505467 gas 
INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
.INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
.INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
.INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
INFO:eth.block	Block pre-sealed, 98148 gas used 
INFO:eth.chain	Adding to head head=08aa138d
Saved 7 address change logs
INFO:eth.chain	Added block 6 (0fa9f473) with 1 txs and 98148 gas 
.INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
.INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
INFO:eth.block	Block pre-sealed, 98148 gas used 
INFO:eth.chain	Adding to head head=08aa138d
Saved 7 address change logs
INFO:eth.chain	Added block 6 (0fa9f473) with 1 txs and 98148 gas 
.Initializing chain from provided state
Initializing chain from provided state
INFO:eth.block	Block pre-sealed, 21000 gas used 
INFO:eth.chain	Adding to head head=c6745cf3
Saved 3 address change logs
INFO:eth.chain	Added block 1 (5f5f608a) with 1 txs and 21000 gas 
INFO:eth.block	Block pre-sealed, 470791 gas used 
INFO:eth.chain	Adding to head head=5f5f608a
Saved 2 address change logs
INFO:eth.chain	Added block 2 (3c04531f) with 1 txs and 470791 gas 
INFO:eth.block	Block pre-sealed, 1641161 gas used 
INFO:eth.chain	Adding to head head=3c04531f
Saved 2 address change logs
INFO:eth.chain	Added block 3 (94a8e4d5) with 1 txs and 1641161 gas 
INFO:eth.block	Block pre-sealed, 505467 gas used 
INFO:eth.chain	Adding to head head=94a8e4d5
Saved 2 address change logs
INFO:eth.chain	Added block 4 (92e1a399) with 1 txs and 505467 gas 
INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
^C^CTraceback (most recent call last):
  File "manage.py", line 15, in <module>
    execute_from_command_line(sys.argv)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/django/core/management/__init__.py", line 371, in execute_from_command_line
    utility.execute()
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/django/core/management/__init__.py", line 365, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/django/core/management/commands/test.py", line 26, in run_from_argv
    super().run_from_argv(argv)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/django/core/management/base.py", line 288, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/django/core/management/base.py", line 335, in execute
    output = self.handle(*args, **options)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/django/core/management/commands/test.py", line 59, in handle
    failures = test_runner.run_tests(test_labels)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/django/test/runner.py", line 603, in run_tests
    result = self.run_suite(suite)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/django/test/runner.py", line 569, in run_suite
    return runner.run(suite)
  File "/usr/lib/python3.6/unittest/runner.py", line 176, in run
    test(result)
  File "/usr/lib/python3.6/unittest/suite.py", line 84, in __call__
    return self.run(*args, **kwds)
  File "/usr/lib/python3.6/unittest/suite.py", line 122, in run
    test(result)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/django/test/testcases.py", line 206, in __call__
    super().__call__(result)
  File "/usr/lib/python3.6/unittest/case.py", line 653, in __call__
    return self.run(*args, **kwds)
  File "/usr/lib/python3.6/unittest/case.py", line 605, in run
    testMethod()
  File "/home/artemis/PycharmProjects/uport_relay_server/client/tests/test_client.py", line 110, in test_create_identity
    log = self._create_identity(owner, owner)
  File "/home/artemis/PycharmProjects/uport_relay_server/client/tests/test_client.py", line 91, in _create_identity
    tx_receipt = self.web3.eth.waitForTransactionReceipt(tx_hash)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/eth.py", line 199, in waitForTransactionReceipt
    return wait_for_transaction_receipt(self.web3, transaction_hash, timeout)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/utils/transactions.py", line 54, in wait_for_transaction_receipt
    txn_receipt = web3.eth.getTransactionReceipt(txn_hash)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/eth.py", line 204, in getTransactionReceipt
    [transaction_hash],
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/manager.py", line 103, in request_blocking
    response = self._make_request(method, params)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/manager.py", line 86, in _make_request
    return request_func(method, params)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/middleware/gas_price_strategy.py", line 18, in middleware
    return make_request(method, params)
  File "cytoolz/functoolz.pyx", line 232, in cytoolz.functoolz.curry.__call__
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/middleware/formatting.py", line 48, in apply_formatters
    response = make_request(method, formatted_params)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/middleware/attrdict.py", line 18, in middleware
    response = make_request(method, params)
  File "cytoolz/functoolz.pyx", line 232, in cytoolz.functoolz.curry.__call__
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/middleware/formatting.py", line 50, in apply_formatters
    response = make_request(method, params)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/middleware/normalize_errors.py", line 9, in middleware
    result = make_request(method, params)
  File "cytoolz/functoolz.pyx", line 232, in cytoolz.functoolz.curry.__call__
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/middleware/formatting.py", line 50, in apply_formatters
    response = make_request(method, params)
  File "cytoolz/functoolz.pyx", line 232, in cytoolz.functoolz.curry.__call__
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/middleware/formatting.py", line 48, in apply_formatters
    response = make_request(method, formatted_params)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/providers/eth_tester/middleware.py", line 322, in middleware
    return make_request(method, params)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/middleware/fixture.py", line 12, in middleware
    return make_request(method, params)
  File "cytoolz/functoolz.pyx", line 232, in cytoolz.functoolz.curry.__call__
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/middleware/formatting.py", line 50, in apply_formatters
    response = make_request(method, params)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/providers/eth_tester/main.py", line 46, in make_request
    response = delegator(self.ethereum_tester, params)
  File "cytoolz/functoolz.pyx", line 758, in cytoolz.functoolz.excepts.__call__
  File "cytoolz/functoolz.pyx", line 491, in cytoolz.functoolz.Compose.__call__
  File "cytoolz/functoolz.pyx", line 232, in cytoolz.functoolz.curry.__call__
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/providers/eth_tester/defaults.py", line 36, in call_eth_tester
    return getattr(eth_tester, fn_name)(*fn_args, **fn_kwargs)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/eth_tester/main.py", line 336, in get_transaction_receipt
    raw_receipt = self.backend.get_transaction_receipt(raw_transaction_hash)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/eth_tester/backends/pyethereum/v20/main.py", line 376, in get_transaction_receipt
    transaction_hash,
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/eth_tester/backends/pyethereum/v20/main.py", line 155, in _get_transaction_by_hash
    if transaction.hash == transaction_hash:
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/ethereum/transactions.py", line 141, in hash
    return utils.sha3(rlp.encode(self))
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/ethereum/utils.py", line 192, in sha3
    return sha3_256(to_string(seed))
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/ethereum/utils.py", line 4, in sha3_256
    def sha3_256(x): return keccak.new(digest_bits=256, data=x).digest()
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/Crypto/Hash/keccak.py", line 173, in new
    return Keccak_Hash(data, digest_bytes, update_after_digest)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/Crypto/Hash/keccak.py", line 70, in __init__
    state = VoidPointer()
  File "/usr/lib/python3.6/unittest/signals.py", line 36, in __call__
    self.default_handler(signum, frame)
KeyboardInterrupt

nested topic arrays should be interpreted as match options for the topic at the nested array position

  • Version:0.1.0b29
  • Python: 3.6

What was wrong?

See review comment here: https://github.com/ethereum/web3.py/pull/976/files/bec2e4e082c2ac58fd38063dec5f5a22641bc3cc

To summarize: we were interpreting the nested arrays in topic lists as full variants of the topic list, rather than options for the topic at the nested array position.

How can it be fixed?

Several methods, validators etc. need to be updated to follow the spec here:

https://github.com/ethereum/wiki/wiki/JSON-RPC#a-note-on-specifying-topic-filters

Transaction Logs parameters naming

I am using this package in conjunction with:
web3==3.15.0
ethereum==1.6.1

When I create a transaction that has a log, the log has the following structure:
[AttributeDict({'block_hash': ... , transaction_hash: ..., log_index: ... })]

When I retrieve a transaction log from an existing HTTPProvider instance, the log has this structure:
[AttributeDict({'blockHash': ... , transactionHash: ..., logIndex: ... })]

Therefore, when I try to call from web3.utils.events.get_event_data it fails with the following error:
Traceback (most recent call last): File "/home/artemis/Documents/LiClipse_Workspace/django-ethereum-events/django_ethereum_events/decoder.py", line 39, in decode_logs decoded.append(self.decode_log(log)) File "/home/artemis/Documents/LiClipse_Workspace/django-ethereum-events/django_ethereum_events/decoder.py", line 33, in decode_log return log_topic, get_event_data(self.topics_map[log_topic]['EVENT_ABI'], log) File "/home/artemis/.virtualenvs/django_ethereum_events/lib/python3.5/site-packages/eth_utils/string.py", line 85, in inner return force_obj_to_text(fn(*args, **kwargs)) File "/home/artemis/.virtualenvs/django_ethereum_events/lib/python3.5/site-packages/web3/utils/events.py", line 201, in get_event_data 'logIndex': log_entry['logIndex'], File "/home/artemis/.virtualenvs/django_ethereum_events/lib/python3.5/site-packages/web3/utils/datastructures.py", line 25, in __getitem__ return self.__dict__[key] KeyError: 'logIndex'

Overwritten transactions are not removed

  • Version: 0.1.0b15
  • Python: 3.5
  • OS: all

What was wrong?

When submitting a transaction that overwrites another transaction, the original transaction is not removed and calls to getTransaction() still return the old transaction instead of null.

In geth, the transaction would be removed and returned as None from getTransaction().

Example:

tester = EthereumTester(auto_mine_transactions=False)

web3 = Web3(EthereumTesterProvider(tester))

original_tx = web3.eth.sendTransaction({
    'from': '...',
    'to': '...',
    'value': 1,
    'gas': 23500,
    'gasPrice': 10,
    'nonce': 0
})

print(web3.eth.getTransaction(original_tx))

replacement_tx = web3.eth.sendTransaction({
    'from': '...',
    'to': '...',
    'value': 2,
    'gas': 23500,
    'gasPrice': 20,
    'nonce': 0
})

print(web3.eth.getTransaction(replacement_tx))
print(web3.eth.getTransaction(original_tx))  # Should be null but still returns transaction

How can it be fixed?

Unsure whether this is a backend problem or not but i imagine there needs to be some tx pool managing functionality added.

Convert documentation to ReadTheDocs

What was wrong?

Docs are currently only in the README.

How can it be fixed?

Convert documentation to be sphinx based and host on readthedocs.

Mismatch on the acceptable input/output for contract creation transactions

  • Version: 0.1.0-beta.20
  • Python: 3.6

What was wrong?

This is most noticeable when posting a pending transaction and then mining it later:

  • You post the pending transaction that creates a contract (with no to key)
  • When retrieving the pending transaction to add to the block, the normalizer returns the transaction with to set to b''
  • The inbound normalizer/validator chokes on a to value that is not a normal address

cc @mhchia

How can it be fixed?

Since eth-tester generally follows the practice of strict requirements on both inputs and outputs, one way to solve this would be:

  1. Specify an empty to field by either:
    a. leaving it out of the transaction
    b. including a 'to' key in the transaction dict, with value: ''
  2. Normalization should update to include
    a. inbound normalization should normalize '' to b''
    a. inbound normalization should normalize a missing key to b''
    b. outbound normalization should normalize from b'' to ''
  3. When saving pending transactions to mine, do not apply outbound normalization first It's ok as is, because the transaction gets inbound normalized when applying to the block

Cleanup address and account utils

What is wrong?

  • There is a duplicate util in both eth_tester.utils.account and eth_tester.utils.address for generating contract addresses.
  • The private_key_to_address function should be removed in favor of using eth-keys API

Mock storage variables in contracts?

I was curious as to whether it is theoretically possible to somehow add some functionality to the eth-tester library to mock state variables in contracts? I don't actually know too much about how data is stored in the EVM, and I'm coming from a Solidity mind-set so maybe I'm thinking about storage incorrectly. I was hoping for a reason why, if this is not possible, and maybe a hint of the right direction to start exploring this idea if it should be possible.

I would basically like functionality to do something like:

contract Example {
   function getVal() view public returns (uint256) {...}

   function doSomething() view public returns (bool) {
     uint256 x = getVal();
     require(x > 10);
     ...
  }
}
def test_contract(contract):
    EthereumTester.mock(contract, 'getVal').return_value = 11
    output = contract.functions.doSomething().call()
    assert output

Validate Transaction Function Doesnt allow for chainId

  • Version: 0.1.0b13
  • Python: 3.5
  • OS: linux

What was wrong?

The validate_transaction function does not allow for chainId as a parameter. When I ran a test using web3.py to send a transaction that included the chainId as a parameter, I received the following error.

eth_tester.exceptions.ValidationError: Only the keys 'data/from/gas/gas_price/nonce/to/value' are allowed. Got extra keys: 'chainId'

ethereum/web3.py/pull/595

How can it be fixed?

Add chainId to the inbound.py and oubound.py validation functions.

Better API for initializing with genesis parameters

What was wrong?

As of #123 there will be a way to specify genesis parameters, but it will take some detailed documentation to understand, and the variety of options will probably confuse beginners. (Thanks to @voith for offering to write the docs)

Let's reduce the ways to initialize the backend with custom genesis parameters, and do it in a way that leans more on the global standards (like the genesis file, which may only be a "de facto" standard, but still better than a custom web3.py thing).

How can it be fixed?

Let's can coalesce around this as the preferred single mechanism for initializing a custom genesis state:

backend = PyEVMBackend.from_genesis(dict(gas_limit=etc))

# also accepts the equivalent json-encoded string:
backend = PyEVMBackend.from_genesis('{"gas_limit": etc}')

We should further explore what use cases people have for setting up custom genesis, but I think we can handle them all cleanly by creating tools to easily generate a genesis dict.

The following APIs are shooting from the hip, I'm not saying that any specific one is a good idea. Just some examples...

from eth_utils import make_genesis_accounts

genesis_accounts = make_genesis_accounts(num_accounts=3, init_state=dict(balance=10**18))

PyEVMBackend.from_genesis(dict(gas_limit=7_000_000, accounts=genesis_accounts))

or

# some silly keys, where int(key) in range(10)
keys = eth_utils.get_test_keys(10)

genesis = eth_utils.make_funded_genesis(dict(gas_limit=7_000_000), keys=keys)

tester_backend = PyEVMBackend.from_genesis(genesis)

for key in keys:
  tester_backend.add_account(keys)

How cool is it that the same tools could be used to generate a genesis file for any other node?! Speaking of which, maybe these generation tools already exist somewhere else?

Fix toolz and cytoolz code

What was wrong?

Right now all imports come from cytoolz. However, our setup.py module is setup to conditionally install either cytoolz or toolz depending on whether we're on Cython or pypy. This means that a pypy installation (which we aren't currently testing against) will fail because cytoolz won't be installed.

How can it be fixed?

Update all toolz/cytoolz imports to import from eth_utils.toolz.

Vyper dependency causing testing errors

  • Version: 0.1.0b24/0.1.0b26
  • Python: 3.6.5
  • OS: osx

What was wrong?

Testing errors when bumping version of vyper

Reproduce:

  • pull down casper master
  • in requirements.txt, bump vyper version to most recent commit — 8113f79f7bce362e7a0f803b32736cb460c78640
  • install via pip install -r requirements.txt
  • run pytest tests

The above will produce the following error:

 TypeError: 'ChainDB' object does not support item assignment

If the version of eth-tester is bumped to 0.1.0b26 in addition to the vyper commit bump, running pytest tests will produce:

evm.exceptions.ValidationError: mix hash mismatch; 0x351006e4ee3e28295bf07ff25dd9fd91d23751e0036fdfc39a51635e0e2edd69 != 0x0000000000000000000000000000000000000000000000000000000000000000

How can it be fixed?

Not sure yet.

Ability to generate blocks with uncles

What was wrong?

Generating a block with an uncle isn't something that can really be done in any reasonable way.

How can it be fixed?

Add an API for generating uncles.

Accept empty array of addresses in filter, match any address

What was wrong?

The spec is unclear about what to do with a filter that has an empty list of addresses. At least one reference implementation (geth) chooses to interpret an empty list of addresses as matching all addresses: https://github.com/ethereum/go-ethereum/blob/461291882edce0ac4a28f64c4e8725b7f57cbeae/eth/filters/filter.go#L254-L256

How can it be fixed?

Given that the spec is unclear, let's just match geth's implementation. (Let's re-open discussion if parity does something different)

eth-tester should accept an empty array of addresses in a filter, and treat it the same as if no addresses key was supplied.

eth-tester[pyethereum16] filter 'toBlock' is exclusive

  • Version: eth-tester==0.1.0b21
  • Python: 3.6
  • OS: osx

What was wrong?

I'm not sure where to report this best, but as it happens on a eth-tester based chain I think this is a good place.

The issue is that creating a filter with the toBlockparameter set to a block number behaves differently when eth-tester with the pyethereum1.6 backend than with geth.

Geth's toBlock is inclusive while the eth-tester filter excludes it. Even though this is a backend related problem this should probably be handled by eth-tester or the backend plugins and tested.

How can it be fixed?

Handle filter behaviour and test that toBlock is inclusive.

Allow call method to query state of past blocks

What was wrong?

Currently the eth-tester library doesn't support using the eth_call method to query the state of past blocks in the blockchain. Given that the new version of web3.py will support querying past blocks in the chain it makes sense to have that functionality available in eth-tester as well.

eth-tester 0.1.0b5 has requirement cytoolz==0.8.2, but you'll have cytoolz 0.9.0.1 which is incompatible.

could this strict requirement be made more generous?

pip install -r requirements.txt
...
eth-tester 0.1.0b5 has requirement cytoolz==0.8.2, but you'll have cytoolz 0.9.0.1 which is incompatible.
...

also related to the same issue as yesterday --> energywebfoundation/ew-link-origin#6 "dependencies: eth-tester 0.1.0b5 has requirement cytoolz==0.8.2, but you'll have cytoolz 0.9.0.1 which is incompatible. "

thanks

Creating a filter generates duplicate events in other filters

  • Version: 0.1.0b27
  • Python: 3.5
  • OS: linux

What was wrong?

When creating a new filter using create_log_filter(), _process_blocks_logs() is called to populate filter with past events.
This method however calls add() method for all existing filters:

    def _process_block_logs(self, block):
        for _, filter in self._log_filters.items():
            for transaction_hash in block['transactions']:
                receipt = self.get_transaction_receipt(transaction_hash)
                for log_entry in receipt['logs']:
                    raw_log_entry = self.normalizer.normalize_inbound_log_entry(log_entry)
                    filter.add(raw_log_entry)

How can it be fixed?

Filter::add() method should check if the log entry already exists in Filter::values before appending/processing it.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.