Code Monkey home page Code Monkey logo

eth-gas-reporter's Introduction

eth-gas-reporter

npm version Build Status Codechecks buidler

A Mocha reporter for Ethereum test suites:

  • Gas usage per unit test.
  • Metrics for method calls and deployments.
  • National currency costs of deploying and using your contract system.
  • CI integration with codechecks
  • Simple installation for Truffle and Buidler
  • Use ETH, BNB, MATIC, AVAX, HT or MOVR price to calculate the gas price.

Example output

Screen Shot 2019-06-24 at 4 54 47 PM

Installation and Config

Truffle

npm install --save-dev eth-gas-reporter
/* truffle-config.js */
module.exports = {
  networks: { ... },
  mocha: {
    reporter: 'eth-gas-reporter',
    reporterOptions : { ... } // See options below
  }
};

Buidler

npm install --save-dev buidler-gas-reporter
/* buidler.config.js */
usePlugin('buidler-gas-reporter');

module.exports = {
  networks: { ... },
  gasReporter: { ... } // See options below
};

Other

This reporter should work with any build platform that uses Mocha and connects to an Ethereum client running as a separate process. There's more on advanced use cases here.

Continuous Integration (Travis and CircleCI)

This reporter comes with a codechecks CI integration that displays a pull request's gas consumption changes relative to its target branch in the Github UI. It's like coveralls for gas. The codechecks service is free for open source and maintained by MakerDao engineer @krzkaczor.

Complete set-up guide here (it's easy).

Screen Shot 2019-06-18 at 12 25 49 PM

Options

⚠️ CoinMarketCap API change ⚠️

Beginning March 2020, CoinMarketCap requires an API key to access currency market price data. The reporter uses an unprotected . You can get your own API key here and set it with the coinmarketcap option. (This service's free tier allows 10k reqs/mo)

In order to retrieve the gas price of a particular blockchain, you can configure the token and gasPriceApi (API key rate limit may apply).

NOTE: HardhatEVM and ganache-cli implement the Ethereum blockchain. To get accurate gas measurements for other chains you may need to run your tests against development clients developed specifically for those networks.

Option Type Default Description
currency String 'EUR' National currency to represent gas costs in. Exchange rates loaded at runtime from the coinmarketcap api. Available currency codes can be found here.
coinmarketcap String (unprotected API key) API key to use when fetching current market price data. (Use this if you stop seeing price data)
gasPrice Number (varies) Denominated in gwei. Default is loaded at runtime from the eth gas station api
token String 'ETH' The reference token for gas price
gasPriceApi String Etherscan The API endpoint to retrieve the gas price. Find below other networks.
outputFile String stdout File path to write report output to
forceConsoleOutput Boolean false Print report output on console
noColors Boolean false Suppress report color. Useful if you are printing to file b/c terminal colorization corrupts the text.
onlyCalledMethods Boolean true Omit methods that are never called from report.
rst Boolean false Output with a reStructured text code-block directive. Useful if you want to include report in RTD
rstTitle String "" Title for reStructured text header (See Travis for example output)
showTimeSpent Boolean false Show the amount of time spent as well as the gas consumed
excludeContracts String[] [] Contract names to exclude from report. Ex: ['Migrations']
src String "contracts" Folder in root directory to begin search for .sol files. This can also be a path to a subfolder relative to the root, e.g. "planets/annares/contracts"
url String web3.currentProvider.host RPC client url (ex: "http://localhost:8545")
proxyResolver Function none Custom method to resolve identity of methods managed by a proxy contract.
artifactType Function or String "truffle-v5" Compilation artifact format to consume. (See advanced use.)
showMethodSig Boolean false Display complete method signatures. Useful when you have overloaded methods you can't tell apart.
maxMethodDiff Number undefined Codechecks failure threshold, triggered when the % diff for any method is greater than number (integer)
maxDeploymentDiff Number undefined Codechecks failure threshold, triggered when the % diff for any deployment is greater than number (integer)

token and gasPriceApi options example

Network token gasPriceApi
Ethereum (default) ETH https://api.etherscan.io/api?module=proxy&action=eth_gasPrice
Binance BNB https://api.bscscan.com/api?module=proxy&action=eth_gasPrice
Polygon MATIC https://api.polygonscan.com/api?module=proxy&action=eth_gasPrice
Avalanche AVAX https://api.snowtrace.io/api?module=proxy&action=eth_gasPrice
Heco HT https://api.hecoinfo.com/api?module=proxy&action=eth_gasPrice
Moonriver MOVR https://api-moonriver.moonscan.io/api?module=proxy&action=eth_gasPrice

These APIs have rate limits. Depending on the usage, it might require an API Key.

NB: Any gas price API call which returns a JSON-RPC response formatted like this is supported: {"jsonrpc":"2.0","id":73,"result":"0x6fc23ac00"}.

Advanced Use

An advanced use guide is available here. Topics include:

  • Getting accurate gas data when using proxy contracts like EtherRouter or ZeppelinOS.
  • Configuring the reporter to work with non-truffle, non-buidler projects.

Example Reports

Usage Notes

  • Requires Node >= 8.
  • You cannot use ganache-core as an in-process provider for your test suite. The reporter makes sync RPC calls while collecting data and your tests will hang unless the client is launched as a separate process.
  • Method calls that throw are filtered from the stats.
  • Contracts that are only ever created by other contracts within Solidity are not shown in the deployments table.

Troubleshooting

Contributions

Feel free to open PRs or issues. There is an integration test and one of the mock test cases is expected to fail. If you're adding an option, you can vaildate it in CI by adding it to the mock options config located here.

Credits

All the ideas in this utility have been borrowed from elsewhere. Many thanks to:

  • @maurelian - Mocha reporting gas instead of time is his idea.
  • @cag - The table borrows from / is based his gas statistics work for the Gnosis contracts.
  • Neufund - Block limit size ratios for contract deployments and euro pricing are borrowed from their ico-contracts test suite.

Contributors

eth-gas-reporter's People

Contributors

area avatar ben-kaufman avatar bmmpxf avatar cgewecke avatar christopherdedominici avatar clemsos avatar dependabot[bot] avatar fodisi avatar frangio avatar gnidan avatar iamchrissmith avatar itsnickbarry avatar julien51 avatar krzkaczor avatar ldub avatar lucaperret avatar martincik avatar ppoliani avatar rmuslimov avatar wighawag avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

eth-gas-reporter's Issues

Optional output to file

May we allow user to save table output to file? That may be useful for automation of this reports.

Internal functions are not estimated.

Hi dear @cgewecke,
first of all thanks for the great tool!

Maybe I'm wrong or missing something but I've noticed that internal functions are not estimated in the report but they are in the inline test report.

I've been working on this little project TokenLocker contract where I've implemented this pattern:

contract TokenLocker is Ownable {
....
function release(address _to) internal ... { ... }
function() public {
        release(msg.sender);
    }
}

The tests are running cool
(e.g ✓ should release the tokens to the first beneficiary expecting the LogRelease event (47139 gas))

but in the resume table at the end I got 0 estimation of the release function
| TokenTimelock · release · - · - · - · 0 · - │

It could be my lack of experience.
Thanks in advance!

Config

  • gas price and eth price
  • if these are present, calculate cost per test. This is useful if you want to approximate costs of contract deployment in a dedicated sub-suite. Or show these values in CI.
  • currency code (list the available codes in a doc).

Blockcypher `safe low` gas price has no relationship to reality.

Stuck at 5 gwei while eth gas station reports 36 gwei due to cryptokitty insanity.

Fix by using: https://ethgasstation.info/json/ethgasAPI.json

{
 "speed": 0.9603672373790222, 
 "safeLow": 360.0, 
 "safeLowWait": 9.2, 
 "block_time": 13.806122448979592, 
 "average": 400.0, 
 "avgWait": 1.7, 
 "fastestWait": 0.5, 
 "blockNum": 4704355, 
 "fast": 510.0, 
 "fastest": 550.0, 
 "fastWait": 0.6
}

Byzantium upgrades

  • testrpc dev dep to v6
  • Pure / View
  • Check tx status to filter thrown tx's

Add gasStats

Alan Lu wrote these really cool gasStats methods for Gnosis that show average gas usage for each method across a full test run.

They won't really work for the reporter because the execution context is different. They also require installation on a per test basis in the before blocks. However, truffle's artifacts is exposed as a global in the reporter and there's another way to do this with abi-encoder.

  • List all the .sols in contracts
  • artifact.require all the contracts
  • create a mapping of the method hashes using abi-encoder
  • for every block, get the tx(s) & get the method hash from the code + gas spent.
  • add to map, as array of expenditures.
  • process the averages using Alan's technique here
  • publish as a table.

Filtering out unused contract methods

It is possible that a lot of methods implemented in contracts will be not covered (for different reasons: small coverage, optional tests called, etc.). For those cases, I think having only called Methods in table desirable.
Let's have this as an option in .ethgas.js

Test stop at string.match(regex) when a certain contract is in the test

Hey there,

I am trying to connect the reporter to daostak/arc (https://github.com/daostack/arc) but for some reason, the test just stops.
We tried to debug it and found that this line causes the reporter operation to stop:
const regExp = new RegExp(`^${binary}`); at getStats.js line 76.
This only happens when the UController.sol contract is in the project when we remove this contract the reporter starts to work again.

Would appreciate any help on the matter.

Enabling this reporter severely decreases the ability to debug failing tests

Without this reporter, a failing test will print out the error messages as soon as the run finishes. Was it a revert, a opcode or invalid jump, etc. Stuff that really helps figuring out the issue.

With the reporter, it does no longer print out these errors. Which means that I can use it as long as my tests all pass, but as soon as one fails, I have to disable it to fix it.

Support Web3 1.0

The gas reporter inherits web3 from the Truffle test environment and it's likely this will be upgraded to Web3 1.0 soon. Sync web3 methods will be async.

(Suggested via email by Sai)

Collect gas data for specified tests only

This would useful for speed improvements alone. It's common to want to collect data for a discrete simulation that sits within a larger test corpus. Might be convenient to be able to use Mocha's @test flagging to be more selective.

(Suggested via email by Sai)

Add deployment subsection to table

Algo for this:

  • At mapMethodsToContracts get all the artifact instances and return them to the runner.
  • At hook start get a deployments specific startBlock
  • At hook end
    • for each block find contract creation transactions and get their address
    • for each artifact try to generate an instance at that address
    • for each successfully created instance produce { contract: , gasUsed: }
  • At generateGasStatsReport print a separate alphabetically sorted deployments section

Only supports unique method signatures

@iamchrissmith raised this. They're trying to compare two erc20 token implementations side by side. At the moment we track method data by signature hash (using abi-decoder) and if there are duplicates, all invocations are collected under it, even if they belong to different contracts.

Output test duration

hey, firstly: thanks for this tool!

was wondering whether there was a way to output the test duration using this reporter.
trying to identify our bottlenecks in a quite large test suite

Start deployments data search at block zero?

Maybe this should be an option for CI or something. Otherwise lots of old data in a long running testrpc instance might corrupt the vals. On the other hand, lots of data is missing if you deploy in the migrations and use those instances.

Error: Content needs to be a primitive, got: function

// ^^ a bunch of tests passing, and then...  ^^
     ✓ should be able to withdraw via core as COO (45404 gas)
Gas spent on test suite: 44334524000000879275

/Users/primary/Projects/cryptokitties-bounty/node_modules/truffle/build/cli.bundled.js:320098
        throw reason;
        ^

Error: Content needs to be a primitive, got: function
    at Cell.setOptions (/Users/primary/Projects/cryptokitties-bounty/node_modules/cli-table2/src/cell.js:27:11)
    at new Cell (/Users/primary/Projects/cryptokitties-bounty/node_modules/cli-table2/src/cell.js:12:8)
    at /Users/primary/Projects/cryptokitties-bounty/node_modules/cli-table2/src/layout-manager.js:157:16
    at arrayMap (/Users/primary/Projects/cryptokitties-bounty/node_modules/cli-table2/node_modules/lodash/index.js:1406:25)
    at Function.map (/Users/primary/Projects/cryptokitties-bounty/node_modules/cli-table2/node_modules/lodash/index.js:6710:14)
    at /Users/primary/Projects/cryptokitties-bounty/node_modules/cli-table2/src/layout-manager.js:156:16
    at /Users/primary/Projects/cryptokitties-bounty/node_modules/cli-table2/node_modules/lodash/index.js:2275:27
    at /Users/primary/Projects/cryptokitties-bounty/node_modules/cli-table2/node_modules/lodash/index.js:3049:15
    at baseMap (/Users/primary/Projects/cryptokitties-bounty/node_modules/cli-table2/node_modules/lodash/index.js:2274:7)
    at Function.map (/Users/primary/Projects/cryptokitties-bounty/node_modules/cli-table2/node_modules/lodash/index.js:6710:14)
npm ERR! Test failed.  See above for more details.```

Add median stat, use min gas value to calculate the currency cost

  1. In any well written test suite a substantial number (if not most) of the tests will be about ensuring that error conditions throw.
  2. We're at a turning point with testrpc where shortly it will begin refunding unused gas for most of the transactions that fail.
  3. Averages are useless for both of these cases - right now the vals are getting skewed way high, going forward they will skew way low.
  4. Atm we should foreground the min as the most plausible 'real' cost?
  5. In future we should foreground the max?

Doesn't work with truffle develop

On 4.0.0-beta.2 the tests don't execute and an empty table is printed.

On 4.0.1 mocha reports reporter not found:

truffle(develop)> test
"eth-gas-reporter" reporter not found
Error: invalid reporter "eth-gas-reporter"

Tried installing the reporter globally and got the same error.

Error: Synchronous requests are not supported.

Seen at Neufund, when using the provider in truffle.js. . . the pattern looks like this:

    inprocess: {
      network_id: "*",
      provider: TestRPC.provider({
        accounts: Array(10).fill({ balance: "12300000000000000000000000" })
      })
    },

The errors then come from our use of sync web3 methods like web3.eth.getBlock('latest)

Empty test cases report gas consumed

Consider the following diff creating an empty test case:

diff --git a/mock/test/metacoin.js b/mock/test/metacoin.js
index f7e6397..b146518 100644
--- a/mock/test/metacoin.js
+++ b/mock/test/metacoin.js
@@ -34,6 +34,7 @@ contract('MetaCoin', function (accounts) {
       assert.equal(metaCoinEthBalance, 2 * metaCoinBalance, 'Library function returned unexpected function, linkage may be broken')
     })
   })
+  it('should not consume any gas when an empty test case runs', function() {});
   it('should send coin correctly', function () {
     var meta

We should not expect this test case to report gas being consumed. However, it does:

  Contract: MetaCoin
    ✓ should put 10000 MetaCoin in the first account (347432 gas)
    ✓ should call a function that depends on a linked library (347432 gas)
    ✓ should not consume any gas when an empty test case runs (347432 gas)
    ✓ should send coin correctly (398425 gas)

You can also see that the previous test which only uses calls and doesn't send any transactions also reports gas usage even though the calls do not consume gas.

Tested on macOS 10.13.5, truffle v4.1.11, eth-gas-reporter v0.1.17 (and v0.1.8)

Neufund's % of limit

They have a nice gas logging utility in their tests that looks like this:

screen shot 2017-10-02 at 7 41 48 pm

387% of the limit. The source data for the gasPrice and ether price is static.

Report table doesn't count calls for other contracts

I have a bug with the report table, it displays incorrect gas usage. I have a contract which in one of its functions calls the decreaseApproval function of a StandardToken contract (https://github.com/OpenZeppelin/openzeppelin-solidity/blob/master/contracts/token/ERC20/StandardToken.sol) and fires an event.
The cost which the table displayed for calling that function was 22,204, this is about the cost of the basic tx fee and a single event, it looks like it didn't count the call to the other contract at all.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.