Comments (24)
Just an update here that as of foundry-rs/foundry#2576, foundry now automatically etches a single byte to empty accounts as part of vm.mockCall
, so currently I'm uncertain of the value of adding a mocking library.
from forge-std.
Nice, I didn't know Solidity had higher order functions!
I like this API, the downside is that you'd need all the permutations of input/output param types, so not sure how feasible it is.
from forge-std.
One thing I did not realize it that people need out of gas simulation. I just created this issue in my mocking provider repo.
I will keep following this thread and I will try to contribute.
from forge-std.
Another feature I'd like a lot, but probably needs to be done in foundry, is the ability to capture values and/or do better assertions on verify.
Right now, the only way to check that a dependency was called is to do
vm.expectCall(address(dep), abi.encodeWithSelector(dep.foo.selector, param1, paramN));
dep.foo();
There are a couple of issues with this approach
- If at least one of the params doesn't exactly match, you get something like "method call was expected with
calldata
but got none", and the only way to find out what happened is to try to compare that with thecalldata
that you see in the execution trace. - It only allows for strict equality of the params
- It doesn't allow to use the value later in the test, i.e, it requires to hardcode a value that may only stay the same as long as the setup doesn't change (like an address of a contract deployed internally)
So what I'd love to have is something like this
dep.foo();
bytes[] memory captures = vm.capure(address(dep), dep.foo.selector);
assertEq(captures.length, 1, "Method wasn't called the expected number of times!");
(int param1, address param2, ... , paramX) = abi.decode(captures[0], (int, address, ...);
assertGt(param1, 0); // Can do non-exact match of the param
// Can call something else on param2 now that I know it's address
// etc
In a way it's kinda similar to the existing function accesses(address) external returns (bytes32[] memory reads, bytes32[] memory writes);
, but for contract interactions.
Thoughts?
from forge-std.
@bbonanno You should have a closer look at my library because, as I understand, it already supports the features you're requesting.
For example you can mock a method, without the need to specify the exact parameters. As long as the msg.sig
matches, it can reply with an expected result.
An example:
// Make it return false whenever calling .isEven(anything)
provider.givenSelectorReturnResponse(
// Respond to `.isEven()`
abi.encodePacked(IOddEven.isEven.selector),
// Encode the response
MockProvider.ReturnData({
success: true,
data: abi.encodePacked(bool(false))
}),
// Log the event
false
);
That means that you could call the mocked method .isEven()
with any argument and it will reply with false
.
provider.isEven(1) == false
provider.isEven(42) == false
Another way to achieve even more flexibility is to set a default response for ANY kind of request.
Code example:
// Make the provider successfully respond with 42 for any request
provider.setDefaultResponse(
MockProvider.ReturnData({
success: true,
data: abi.encode(uint256(42))
})
);
If you want to inspect the requests being made you can even enable logging and check the logged calls afterward.
provider.getCallData(0); // 0 is the index of the call, it logs all requests.
A caveat to enabling default responses is if there is any call that uses the opcode STATICCALL
, logging won't work, since it will try to write to the provider's state, thus it will fail.
from forge-std.
@cleanunicorn the partial mocking is also possible with the standard mockCall, that's not an issue.
For the capturing, looking at the logs is what I do now, but is not ideal, and it doesn't solve the issue of not doing exact comparison with the values.
I'm trying to do something like an ArgumentCaptor in mockito (if you ever used it)
from forge-std.
I spent some time trying to understand Mockito, but I am not a Java developer but the idiosyncratic approach doesn't make a lot of sense to me.
This is the article I spent most time with https://www.baeldung.com/mockito-argumentcaptor
And I did not understand the advantages of using Mockito vs normal mocking.
Going back to your "ideal" code example, it still doesn't make a lot of sense. I guess this is because you didn't have enough time to explain how everything should work.
I am going to split that into multiple sections and try to explain what I understand and ask additional questions.
dep.foo();
I assume this just calls the method that you want to mock or test. Unfortunately, it's not what what is mocked and what is tested, what is the implementation being tested and what is the mocked code connected to the implementation.
bytes[] memory captures = vm.capure(address(dep), dep.foo.selector);
I assume this captures the previous calls to the dep
contract instance, specifically the method identified by dep.foo.selector
. This should be possible after the call was made since it is called after dep.foo()
.
assertEq(captures.length, 1, "Method wasn't called the expected number of times!");
This checks how many calls were made to the method identified by dep.foo()
, however it's not clear why this is a test since the first line of code in the snippet calls dep.foo()
exactly once. Maybe I don't understand where the code ends and the test begins.
(int param1, address param2, ... , paramX) = abi.decode(captures[0], (int, address, ...);
This takes the first logged call and decodes something. It's not clear what it decodes because the method dep.foo()
does not accept any arguments.
assertGt(param1, 0); // Can do non-exact match of the param
// Can call something else on param2 now that I know it's address
// etc
This check makes sense to me, I understand the non-exact match you want to do.
Coming back to my mocker, it seems this is possible. Here is a snippet that mocks a contract that is expecting a call from Implementation
.
Drop this in Remix and play with it.
import "https://github.com/cleanunicorn/mockprovider/blob/master/src/MockProvider.sol";
contract Implementation {
address callInto;
constructor(address callInto_) {
callInto = callInto_;
}
function foo() public {
callInto.call(
abi.encodeWithSelector(0x11223344, 1, 2, 3, 4)
);
}
}
contract TestImplementation {
function test_foo() public {
MockProvider provider = new MockProvider();
provider.setDefaultResponse(MockProvider.ReturnData({
success: true,
data: ""
}));
Implementation implementation = new Implementation(provider);
implementation.foo();
MockProvider.ReturnData memory rd = provider.getCallData(0);
(uint a, address b, uint c, uint d) = abi.decode(rd.data, (uint, address, uint, uint));
// You can check each item individually
// a > 0
// b == 2
// c not even
// ...
}
}
You can test each item (a, b, c, d) individually with your own rules.
from forge-std.
Yeah, now I realised my example is a bit shit, but you totally nailed it, that's exactly what I'm talking about.
Having a look at the code and looks cool, my noob self didn't know that fallback
could return stuff, hence my LenientMock being so limited 😅
For your first question, given the way the Java compiler and the JVM works, mockito allows you to create mocks without having to write specific classes for each one, pretty much what your lib does, albeit is waaaay simpler given the tools (or lack of) that solidity has.
I think your lib is an amazing starting point, but I also think is missing a few features, I'm all up for adding a few PRs if you like :)
Also not sure if the path is to merge that with forge-std or to keep it as a separate lib
Thoughts?
from forge-std.
I think it's great that you want to contribute. I am not sure if the current interaction API format should be simplified.
In terms of being merged with forge-std, currently I am not sure. The ability to mock is something really useful.
If I start adding forge-specific features it could be merged, bit not required to. Otherwise it can be framework agnostic.
from forge-std.
Thanks for the discussion @bbonanno and @cleanunicorn! 🙂
Here's one more mocking library, by @PARABRAHMAN0 (h/t @gakonst in the foundry telegram)
Would love to also get some thoughts from @maurelian and @PaulRBerg. Either way, I'll try to share my own thoughts / propose a path forward tomorrow in the next few days
from forge-std.
yeah the UX with this mocking library is quite nice: https://github.com/OlympusDAO/test-utils/blob/master/src/test/mocking.t.sol#L36-L61
from forge-std.
I've just caught up on everything that has been discussed here. My thoughts:
- Mocking calls to contracts that do not exist is much more useful than mocking calls to deployed contracts.
- As per my test classification, a unit test is a unit test only if all other contracts are mocked, and writing unit tests is a good thing (I personally think that Ethereum devs focus too much on integration tests - lack of good mocking UX may play a role in that)
- If I put in the work to deploy a contract, I could just inherit from the original contract and write a test-specific version of it, rather than mock it.
- I don't think that mocking OOG errors is that useful and/ or common. When entrusting third-party contracts to perform some work, a good rule of thumb is to reserve a gas stipend (this is what I do in PRBProxy). I don't see why I'd be interested in throwing an out-of-gas in a third-party mock instead of honing in on my stipend calculations.
- An explicit reason (with function name, selector, and all) for why a mock call reverted would certainly be nice if it was provided by forge.
In terms of what mock contract to choose, my votes go to @cleanunicorn's MockProvider
and OlympusDAO's mocking.sol
. The latter's ability to compose .mock
with the mocked function selector is awesome, certainly much better DX than creating a mock provider. But I'm not sure how robust and extensible it is - it looks like a testing gizmo they built for their internal needs at Olympus.
- Mock contract size matters because it is going to be deployed a great many times and this may impact test runtime efficiency - this is why I wouldn't go with Gnosis' implementation.
- @maurelian's
UniMock
is nice but looks very similar toMockProvider
. I chose the latter simply because it seems better packaged as an open-source tool (dedicated repo + @cleanunicorn even wrote tests for it). - Waffle's
Doppelganger.sol
is okay, but Waffle has received little maintenance over the last year or so and it's unlikely that they will respond to any of our feature requests in timely manner.
from forge-std.
- Just catching up here myself. I really appreciate the interest in unimock, it was fun to build and something I thought there was a need for.
That said I don't plan to maintain it, so anyone is welcome to copy the code into a new repo and run with it if they wish.
Otherwise it feels to me like @cleanunicorn's MockProvider
is the natural choice, as it is already more mature, and he is actively supporting and improving it.
from forge-std.
Awesome, yea I tend to agree with the last two responses in that:
- The OlympusDAO lib is great UX, but it won't generalize well unfortunately since you need a lot of overloads
- @cleanunicorn's lib is the way to go, especially appreciate his ongoing maintenance and improvements.
Admittedly I'm not the biggest user of mocks in forge so I'm not the best person to answer this, but here's my next two questions:
- How much of this belongs in forge-std, and what should we improve in foundry? For example, I can imagine updating
vm.mockCall
to (1) place code at an address if it doesn't exist, and (2) have an overload orvm.mockRevert
cheatcode to to support mocking reverts. If that's done, is there still value in adding a mocking library to forge-std? My main concern around asking this is that too much stuff in forge-std will slow down compilation times since solidity compilation is slower than rust cheat codes. - If we do integrate @cleanunicorn's lib:
- Where does the
vm.mockCall
cheatcode fit into dev workflows? - Should the lib be a dependency or upstreamed here? Even if upstreamed here, I don't think it needs to have anything forge specific, so e.g. hardhat users could still use it by installing forge-std
- Where does the
from forge-std.
Does it make sense to add a vm.mockCallForward
cheatcode that lets you forward the call to a function in the local test class so that mocks can have some level of logic? So that instead of mocking a full contract, you could just forward a single call? That would've helped me in foundry-rs/foundry#2740 too
from forge-std.
Does it make sense to add a
vm.mockCallForward
cheatcode that lets you forward the call to a function in the local test class so that mocks can have some level of logic? So that instead of mocking a full contract, you could just forward a single call? That would've helped me in foundry-rs/foundry#2740 too
I like it, it'd be the equivalent of mocking an answer, so I guess the param would be a function type reference to a public function in the same test? (not sure if the compiler would like that though)
from forge-std.
Does it make sense to add a
vm.mockCallForward
cheatcode that lets you forward the call to a function in the local test class so that mocks can have some level of logic? So that instead of mocking a full contract, you could just forward a single call? That would've helped me in foundry-rs/foundry#2740 too
Sorry do you mind clarifying? Not sure I totally follow the idea here. Maybe an example would help.
from forge-std.
Does it make sense to add a
vm.mockCallForward
cheatcode that lets you forward the call to a function in the local test class so that mocks can have some level of logic? So that instead of mocking a full contract, you could just forward a single call? That would've helped me in foundry-rs/foundry#2740 tooSorry do you mind clarifying? Not sure I totally follow the idea here. Maybe an example would help.
Yeah... Something like this:
contract MyTest is Test {
// This would have the same i/o signature as the mocked function.
function myMockFunction(address foo) public returns(string memory) {
return "hello world!";
}
function testSomething() public {
// Redirect calls to `123.foo` to `this.myMockFunction`
vm.mockCall(address(123), "foo(address)", address(this), this.myMockFunction.selector);
...
}
}
from forge-std.
Ah I see. Though I don't follow see the use case for it / when would you need to do that? FWIW you can also vm.etch
any code you want at address(123)
to have methods return different data
from forge-std.
I had a case just now in an integration test where it was rather difficult to figure out exactly what values the function I wanted to mock would be called with and I wanted to return something based on the input. So yeah, add some minimal amount of dynamic logic to the test. I generally try to avoid that and normally would consider that a bad idea but it was hard to get around in this instance.
vm.etch
overrides the entire code at the address. I just wanted to override a single function on the contract at that address (one that's called internally by the other contract).
from forge-std.
Seems like a niche use case so I'm not sure it's worth a cheatcode, but an issue in the foundry repo would be the right place to discuss that.
vm.etch
overrides the entire code at the address. I just wanted to override a single function on the contract at that address (one that's called internally by the other contract).
Yep, so you can take the original source, modify it, then compile and use deployCode
/ etch
. (Admittedly that's a bit tedious and requires having the original source code)
from forge-std.
Hey @mds1 ! (always fun to see you around here :)
Question regarding these mocks: Is it somehow possible to introspect the mocked function, similar to how jest
does mocks?
Let's say I mock a specific contract/function and want to test that it was called N
times, each time with xyz
parameters, is this currently possible?
from forge-std.
Hey! I'm not too familiar with how jest mocks work, but from that description I don't think it's currently possible, though there is a feature request for it: foundry-rs/foundry#4513.
from forge-std.
Going to close this now that we have a mockCallRevert
cheat (ref foundry-rs/foundry#4343 and #337), since that covers most use cases
from forge-std.
Related Issues (20)
- Add `vm.blobhashes`
- "ds-test/test.sol" Not Found HOT 4
- JSON Parsing with Default Fallbacks HOT 3
- ENV var incorrectly takes precedence over CLI flag HOT 1
- [FAIL. Reason: revert: stdStorage find(StdStorage): No storage use detected for target.] HOT 5
- Native String Cheatcodes Not Present HOT 2
- contract creation code storage out of gas HOT 1
- `DSTest` removal in v1.8.0 seems to break the `forge` contract sizer filter HOT 1
- assertEq "Invalid data" HOT 2
- Add prompt cheatcode
- Change `vm` visibility to internal in `StdAssertions` and `StdUtils` HOT 2
- Package.json version is not updated. HOT 4
- Feature request: block gas limits for most popular chains HOT 2
- 1.7.1 -> 1.8.1 breaking change: why is DSTest no longer inherited by Test? HOT 1
- Add Vm.prevrandao as uint256 overloaded function
- Add envExists cheatcode function
- Add isContext cheatcode function HOT 1
- add new blobhash cheatcodes HOT 2
- Support returning unimplemented cheatcode HOT 1
- Can't seem to read `CallerMode` from `vm.readCallers()` HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from forge-std.