federicobond / solidity-parser-antlr Goto Github PK
View Code? Open in Web Editor NEWA Solidity parser for JS built on top of a robust ANTLR4 grammar
License: MIT License
A Solidity parser for JS built on top of a robust ANTLR4 grammar
License: MIT License
Can VariableDeclaration
be simplified? Currently, VariableDeclaration
contains many redundant information.
For example, isStateVar
, visibility
, and isDeclaredConst
are only used when the variable is a state variable. Can we move the information to StateVariableDeclaration
? Besides, StateVariableDeclaration
will always have one VariableDeclaration
object.
I guess we can refactor VariableDeclaration
to:
typeName
, name
, and storageLocation
.
Another question is why eventParameter, functionParameter, and variableDeclaration share the same variableDeclaration object? Can we separate them?
pragma solidity 0.4.0;
pragma solidity ^0.4.0;
pragma solidity >= 0.4.0;
pragma solidity <= 0.4.0;
pragma solidity < 0.4.0;
pragma solidity > 0.4.0;
pragma solidity != 0.4.0;
pragma solidity >=0.4.0 <0.4.8; // from https://github.com/ethereum/solidity/releases/tag/v0.4.0
pragma solidity 0.4;
pragma solidity ^0.4;
pragma solidity >= 0.4;
pragma solidity <= 0.4;
pragma solidity < 0.5;
pragma solidity > 0.4;
pragma solidity != 0.4;
pragma solidity >=0.4 <=0.4;
pragma solidity 0;
pragma solidity v0;
pragma solidity ^0;
pragma solidity >= 0;
pragma solidity <= 0;
pragma solidity < 1;
pragma solidity > 0;
pragma solidity != 0;
pragma solidity >=0 <=1;
All of the above are syntactically correct pragma statements (feel free to test in remix). But parser only recognises version a.b.c
. We need to make the b
& c
part optional, not mandatory.
A contract like this can be compiled:
contract C {
event e(bytes calldata);
}
but the parser fails:
ParserError: extraneous input 'calldata' expecting {',', ')'}
I guess it's related with calldata
being a reserved word in other contexts.
The parser seems to work fine when a function parameter is called calldata
.
Hi, many contracts are using 0XABC as a hex literal but the grammar only allows for 0xABC, because of this parsing of a lot of contracts is failing. Remix IDE allows 0X, so I guess it should be allowed here as well, right?
Howdy,
Sorry for the noise, but I'm just wondering if we could we get the latest version of the package that's ES5 transpiled published up to NPM? I'm depending on my fork still in my project and would love to depend on the vanilla NPM package.
Hi Federico. I noticed the parser throws an error when parsing a return statement written like this "return();". Again, I've seen contracts running on Ethereum that use this statement, so I think it's syntactically valid. But again, I'm a newbie in Solidity, so I might be wrong. Would you mind looking into this? Thanks!
Hi,
Is there any way to release the current master branch of the parser to npm?
The lack of address payable is blocking https://github.com/prettier-solidity/prettier-plugin-solidity
a = abi.decode(b, (uint256[]));
breaks the parser.
Any array inside the tupple breaks the parser. It is valid solidity code though.
Full sample code
pragma solidity ^0.5.0;
contract hello {
function world(bytes memory b) public {
uint256[] memory a;
a = abi.decode(b, (uint256[]));
}
}
First of all thanks for creating this fine tool. It's been helping us a lot already!
In our effort to generate documentation from Ethereum NatSpec comments I decided to take a crack at adding the ability to parse those to this tool. I have to admit I am quite new to Antlr and for quite some time it felt like I was just messing around with it. But I got to a state where I am able to successfully parse single line and multiline NatSpec comments and add them to the AST. That didn't come without any caveats though:
@
in them)These are things I couldn't figure out yet (also sadly due to the sparse JS runtime documentation) and if this is a thing you might want to include I'd highly appreciate a little push in the correct direction.
Here's my attempt so far:
The antlr grammar:
solidityj/solidity-antlr4@master...chmanie:feature/add-natspec
The AST builder:
master...chmanie:feature/add-natspec
Since the AST format of this library is customized, I feel exporting the function _isASTNode
is generally favored as it guarantees that when the format of output of this library changes, this check will/should always be up-to-date.
@federicobond do you agree?
we have a parser error if in the contract there is a require()
function
const parser = require('solidity-parser-antlr');
const input = `
pragma solidity ^0.4.23;
contract Lottery {
address[] public players;
address public manager;
constructor() public {
manager = msg.sender;
}
function enter() public payable {
require(msg.value > .01 ether); // if you comment or remove this line, the parser works
players.push(msg.sender);
}
function random() private view returns (uint) {
return uint(keccak256(block.difficulty, now, players));
}
}
`;
try {
console.log(parser.parse(input));
} catch (error) {
if (error instanceof parser.ParserError) {
console.log(error.errors);
}
}
The parser doesn't seem to differentiate between the left side of this expression:
(x,) = (1,2);
and this one:
(x) = 1;
In both cases I get an ExpressionStatement
whose left side is:
{
type: "TupleExpression",
components: [{ type: "Identifier", name: "x" }],
isArray: false
}
Is this expected?
According to https://solidity.readthedocs.io/en/v0.5.8/contracts.html#libraries we can call a library method like Set.insert(knownValues, value)
. We may want to call something like Set.calculate(value1, value2)
, which would return another value.
The problem is, solidity-parser-antlr is returning this as a MemberAccess
, which is correct, but then it does not give the parameters.
It does appear when using FunctionCall, but only under an expression of type MemberAccess
.
I could make a workaround but it could also be interesting to get the parameters. Will it be possible?
EDIT
Same happens with super._transferFrom(from, to, tokenId);
The AST builder does not include a definition of return parameters, which means that function definitions are lacking the return parameters as well.
First of all, thank you for the awesome project!
I suggest enhancing transformAST to return not only the parsed one but also the original one.
Similar to Javaparser project, if we want to support Transform
and Generate
the solidity code, returning original node is essential.
We could use the changed version to reconstruct specific granularity.
For example,
var source = "contract test { uint a; function c() public { uint f = 3; uint e = 1+4;}}"
var ast = parser.parse(source);
parser.visit(ast, {
FunctionDefinition: (node) => {
console.log(node.ctx.getText()); // "functionc()public{uintf=3;uinte=1+4;}"
}
});
By the way, I'm still thinking about how to keep whitespace and newlines.
Could you give me some advice? (It may be related to #32)
Hi Fede,
I'm opening this issue to track the different parsing errors which I found when creating this PR.
These files can't be parsed:
I'm afraid I don't know how to fix this issue, so no PR this time :(
Some invalid sources make the parser fail when building the ast, because this method throws.
For example, this code contract { function a() return bool {} }
causes that problem.
Here's a stack trace of it failing:
TypeError: Cannot read property 'getText' of null
at ASTBuilder.StateVariableDeclaration (node_modules/solidity-parser-antlr/src/ASTBuilder.js:628:23)
at ASTBuilder.visit (node_modules/solidity-parser-antlr/src/ASTBuilder.js:1133:40)
at ASTBuilder.ContractPart (node_modules/solidity-parser-antlr/src/ASTBuilder.js:89:17)
at ASTBuilder.visit (node_modules/solidity-parser-antlr/src/ASTBuilder.js:1133:40)
at ASTBuilder.<anonymous> (node_modules/solidity-parser-antlr/src/ASTBuilder.js:1121:19)
at Array.map (<anonymous>)
at ASTBuilder.visit (node_modules/solidity-parser-antlr/src/ASTBuilder.js:1120:16)
at ASTBuilder.ContractDefinition (node_modules/solidity-parser-antlr/src/ASTBuilder.js:76:22)
at ASTBuilder.visit (node_modules/solidity-parser-antlr/src/ASTBuilder.js:1133:40)
at ASTBuilder.<anonymous> (node_modules/solidity-parser-antlr/src/ASTBuilder.js:1121:19)
at Array.map (<anonymous>)
at ASTBuilder.visit (node_modules/solidity-parser-antlr/src/ASTBuilder.js:1120:16)
at ASTBuilder.SourceUnit (node_modules/solidity-parser-antlr/src/ASTBuilder.js:33:22)
at ASTBuilder.visit (node_modules/solidity-parser-antlr/src/ASTBuilder.js:1133:40)
at Object.parse (node_modules/solidity-parser-antlr/src/index.js:65:23)
at getImports (src/solidity/imports.js:5:22)
at Context.it (test/solidity/imports.js:15:21)
Hi there!
I'm trying to use this module in a Create React App application. It works fine when I run it in development mode, but when I try to create a production build I get this:
$ react-scripts build
Creating an optimized production build...
Failed to compile.
Failed to minify the code from this file:
./node_modules/solidity-parser-antlr/src/index.js:2
Read more here: http://bit.ly/2tRViJ9
There are 2 main approaches given on that bit.ly link to fix it.
I tried to fix the problem myself with Rollup and Babel, but I'm getting a lot of problems with that approach that are deep in Antlr land, which is not my wheelhouse. This is where I'm up to:
https://github.com/CoinageCrypto/solidity-parser-antlr
It builds a dist/index.js
file, but when I try to use that from the browser I keep getting "can't reference _____ of undefined" errors and similar.
So I shelved that approach and went off to the second fix:
So I added this as a submodule. I then get this:
Failed to compile.
./src/lib/solidity-parser-antlr/antlr4/dfa/DFA.js
Line 107: 'DFAStatesSet' is not defined no-undef
Search for the keywords to learn more about each error.
Looking at that file, I don't actually see anywhere that DFAStatesSet
is defined, and there are no other mentions of that anywhere in the project that I can find. Where should this be coming from?
I think this is actually an eslint error, which Create React App enforces quite strongly and I can't change the configuration without ejecting. So since it's working when it runs as a node module, I gave adding an // eslint-disable-next-line no-undef
a try, but then just hit another file with the same situation.
I decided peppering what looks like a library's source code with eslint comments was probably a road to pain, I tried a more configuration-based approach. I looked if there's a way to disable eslint for that folder, but again, not without ejecting (discussed here: facebook/create-react-app#3886), which I could do, but seriously would like to avoid it.
I've used GatsbyJS a lot in the past, so I threw this module at that build system and couldn't get it to run. I'm not sure why. I think again it's about it being ES6 packaged, but I figured it'd be best to start a discussion here before I go too deep into this.
So that brings me here:
dist
folder?antlr4
directory a library? Can we use an NPM dependency for that rather than packaging it with the source itself?My contract has the following constructor
import "./Negotiator.sol";
constructor(Negotiator _negotiator) public {
owner = msg.sender;
negotiator = _negotiator;
}
function() public payable {
revert("Cannot send ether");
}
I'd like to use web3.eth.abi.encodeParameters
to encode my constructor parameters.
With solidity-parser-antlr, I'm getting the following result for my contract's constructor parameters however:
[ { type: 'Parameter',
typeName: { type: 'UserDefinedTypeName', namePath: 'Negotiator' },
name: '_negotiator',
storageLocation: null,
isStateVar: false,
isIndexed: false } ]
The solidity documentation states that when a fallback function is available on the contract, then the Contract Type should be converted to an address payable
.
This issue is to discuss the possibility of having the typing for the output of the parser automatically generated. Currently, the typing file at https://github.com/DefinitelyTyped/DefinitelyTyped/blob/master/types/solidity-parser-antlr/index.d.ts is more or less manually written and is very incomplete. Since the output is very structured, I feel like this is doable. However, I haven't figured out a way yet. Any thoughts?
Hi Federico. I'm trying to parse a solidity file and I'm getting an error at the following line:
minContribution = .1 ether;
From what I could dig, I think this statement violates the DecimalNumber element of the grammar, as it needs to start with a number ([0-9]+):
DecimalNumber
: [0-9]+ ( '.' [0-9]* )? ( [eE] [0-9]+ )? ;
This statement is part of the source code of a contract running on Ethereum (https://etherscan.io/address/0x009ef15c147ff4c0eb373e1abd2f4d184e5cb916#code), which makes me think that the .X decimal format is valid syntax. However, I'm a total newbie in Solidy, so I could be very wrong.
Thoughts on this? Thanks!
There are some properties of ASTNode
types declared as optional via adding ?
to ``PROPERTY_NAME (
PROPERTY_NAME?`). However, marking optional in this way results in saying such property having value `TYPE | undefined | null` while they are in fact of type `TYPE | null` (or correct me if wrong). e.g.
solidity-parser-antlr/index.d.ts
Line 201 in f0ef770
CC: @Leeleo3x
Give a very simple example, if I add a //
followed by a newline to a correctly written contract, current version gives ParseError
with message extraneous input \'/\' expecting {<EOF>, \'pragma\', \'import\', \'contract\', \'interface\', \'library\'} (1:0)
. This happens regardless where the comment is put.
From aragon-core here
function transitionToVotingState(Vote storage vote) internal {
vote.state = VoteState.Voting;
OwnershipApp ownershipApp = getOwnershipApp();
etc. . .
Error is:
"message": "mismatched input 'storage' expecting {',', ')'}",
Possible fix at this rule, and it's management here?
Federico, I don't want to hound you with PRs of questionable quality over here but as you know, I'm also perfectly happy to submit them. Somewhat curious about how this parser works. Whatever is easiest, though.
Solidity version 0.4.24 introduced this syntax:
(uint256 a, bytes memory b) = f();
Parser currently doesn't support it.
no viable alternative at input '(uint256a'. mismatched input ',' expecting {';', '='}. extraneous input ')' expecting {';', '='}
After upgrading contracts to solc 0.4.23
parsing them fails with:
/node_modules/solidity-parser-antlr/src/index.js:53
throw new ParserError({ errors: listener.getErrors() })
^
Error
at new ParserError (/Users/Elena/colonyNetwork/node_modules/solidity-parser-antlr/src/index.js:12:16)
at Object.parse (/Users/Elena/colonyNetwork/node_modules/solidity-parser-antlr/src/index.js:53:11)
at /Users/Elena/colonyNetwork/scripts/check-storage.js:23:25
at Array.forEach (<anonymous>)
at Object.<anonymous> (/Users/Elena/colonyNetwork/scripts/check-storage.js:5:32)
at Module._compile (module.js:641:30)
at loader (/Users/Elena/colonyNetwork/node_modules/babel-register/lib/node.js:144:5)
at Object.require.extensions.(anonymous function) [as .js] (/Users/Elena/colonyNetwork/node_modules/babel-register/lib/node.js:154:7)
at Module.load (module.js:560:32)
at tryModuleLoad (module.js:503:12)
Suspecting the culprit is the newly introduced constructor
keyword.
code:
contract A {
function baz(){
uint foo;
uint bar;
(foo, , ,, bar,) = (1,2,3,4,5,6);
}
}
> const ast = sp.parse(code);
> const expression = ast.children[0].subNodes[0].body.statements[1].expression
In above scenario, accessing the node expression.left
gives info about foo
& bar
as 2 array items.
The problem is that after parsing, we loose the comma info in the tuple and therefore the ability to map the Identifier with its corresponding value.
Is there a work-around for this (I'm trying to avoid using my custom RegExp on the code string)?
(I'm not sure if this is a problem within the scope of a parser to solve, but it just feels like this information should be present).
It might be nice to have a single grammar of truth rather than maintain two copies here and in solidity-antlr4.
Parser errors on this line in Zeppelin's ERC20.sol
function transferFrom(address from, address to, uint256 value);
Triggered by the Identifier rule?
(I'm installing directly from the repo).
The grammar rules for inline assembly changed with cef1d0b.
We should update the AST builder accordingly.
The loop expression of the for statement gest the type ExpressionStatement
added:
solidity-parser-antlr/src/ASTBuilder.js
Lines 690 to 693 in ded9c70
This is causing issues in prettier-solidity, where is difficult to know if an ExpressionStatement should have a semicolon or not (see prettier-solidity/prettier-plugin-solidity#46 and prettier-solidity/prettier-plugin-solidity#64 for the details). Since the loop expression of a for doesn't have a semicolon, calling it an ExpressionStatement feels wrong.
My question is if there's a good reason to do this, and if there's a chance of modifying it.
Thanks!
AST nodes that are built manually inside ASTBuilder functions do not contain location information.
Just one more:
uint256 constant FIXED_ONE = uint256(1) << PRECISION;
errors with:
{}
FWIW I ran Zeppelin, Melonport, Chronobank, Bancor, & Gnosis through the parser and issues 5,6,7 are the only things that came up. Looks good!
Typescript is getting popular nowadays. In order for projects making use of this project getting benefits from the typing, a type definition is necessary for this library. This issue is to track the progress of such effort. There are already efforts on creating types for this library at https://github.com/DefinitelyTyped/DefinitelyTyped/tree/master/types/solidity-parser-antlr . However, after discussion at #47 , we think that having type definition in this repo is more preferable.
CC all authors of the created types at @ type: @LogvinovLeon @albrow @yxliang01
Hi @federicobond!
I'm interested in replicating this lib for vyper. These are the steps I think that would require, would appreciate if you can point out any big pieces I'm missing, or pitfalls I might not be seeing?
Thanks again for maintaining this awesome lib.
Hi Federico.
Thank you for creating this awesome tool! Itβs been helping me a lot.
When I tried to parse a contract like the following, the parser parsed the code including an error:
contract Owned is temp {
modifier onlyOwner {
assert(msg.sender == owner);
_
}
}
"errors": [
{
"message": "no viable alternative at input '_}'",
"line": 5,
"column": 4
}
]
This error is occurred because of underscore (indeed, if we remove the underscore, the parser would not report the error).
The underscore is a part of the source code of a contract running on Ethereum (a function modifier).
Thoughts on this? Thank you!
This simplifies the development of automatic code formatters, linters and other tools that rely on concrete representations.
See estree/estree#41 and the https://github.com/cst/cst package for more information.
Is it formally defined as an expression which is also a statement?
There seems to be an issue with Assembly Function Declarations. All of the following are valid according to remix, but this parser seems to only successfully parse the first one:
pragma solidity ^0.5.0;
contract JoinSplit {
function() external payable {
assembly {
function validateJoinSplit(y) -> x {
}
}
}
}
pragma solidity ^0.5.0;
contract JoinSplit {
function() external payable {
assembly {
function validateJoinSplit(y) {
}
}
}
}
pragma solidity ^0.5.0;
contract JoinSplit {
function() external payable {
assembly {
function validateJoinSplit() -> x {
}
}
}
}
The next code (version 0.2.12):
var parser = require('solidity-parser-antlr')
var input = `
contract test {
uint[] a;
}
`
try {
const ast = parser.parse(input);
console.log(JSON.stringify(ast, null, 4));
} catch (e) {
if (e instanceof parser.ParserError) {
console.log(e.errors)
}
}
generates the output:
{
"type": "SourceUnit",
"children": [
{
"type": "ContractDefinition",
"name": "test",
"baseContracts": [],
"subNodes": [
{
"type": "StateVariableDeclaration",
"variables": [
{
"type": "VariableDeclaration",
"typeName": {
"type": "ElementaryTypeName",
"name": "uint"
},
"name": "a",
"expression": null,
"visibility": "default",
"isStateVar": true,
"isDeclaredConst": false,
"isIndexed": false
}
],
"initialValue": null
}
],
"kind": "contract"
}
]
}
The type of the variable 'a' is 'ElementaryTypeName' instead of 'ArrayTypeName'.
contract Foo {
function(address) external returns (address) myFunc;
}
This code compiles (notice that the function's name is right before semicolon).
Unfortunately I haven't been able to find any literature on the exact rules the compiler follows in this situation. Couldn't find anything helpful in the solidity grammar either.
(This pattern is not used very often, so not a big problem IMO)
Hi,
Sorry If I was not clear with the title, I don't know how to explain the problem, TBH.
So, going directly into the problem, this https://github.com/OpenZeppelin/openzeppelin-solidity/blob/master/contracts/token/ERC721/ERC721.sol#L273 using the code below, returns the result shown in the end. I was looking at the solidity-antlr4 repository and it seems to not be catching that use case and in fact, it's just missed. Am I right?
parser.visit(fLink, {
FunctionCall: (functionCallNode) => {
if (functionCallNode.expression.name === 'IERC721Receiver') {
console.log(functionCallNode);
}
},
});
{ type: 'FunctionCall',
expression: { type: 'Identifier', name: 'IERC721Receiver' },
arguments: [ { type: 'Identifier', name: 'to' } ],
names: [] }
Cheers
A declarative, efficient, and flexible JavaScript library for building user interfaces.
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. πππ
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google β€οΈ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.