yanagiba / swift-ast Goto Github PK
View Code? Open in Web Editor NEWA Swift AST parser and visitor that is written in Swift.
License: Apache License 2.0
A Swift AST parser and visitor that is written in Swift.
License: Apache License 2.0
I have seen files fail to parse with the error expected '(' in parameter
. From what I can tell, these failures seem to be related to 1) using generics and 2) defining operators like ==
and +=
, for example:
public func ==<A: Equatable, B: Equatable>(
I am not well versed in swift so apologies if this is not a detailed enough description. I have linked several OSS code samples below that cause this failure and seem related. The included snippet is from the airbnb/lona project and runs successfully in swift 4. Thanks!
infix operator ?=
func ?=<T> (left: inout T?, right: T) {
if left == nil {
left = right
}
}
Command to run swift-ast
with the code above:
swift-ast /Users/chrishulton/code/testing/swift_qa/failing_cases/expected_paren.swift
The code should parse successfully.
There is a parse failure: expected '(' in parameter
/Users/chrishulton/code/testing/swift_qa/failing_cases/expected_paren.swift:3:9 fatal: expectedParameterOpenParenthesis
Here are some additional OSS examples that cause this failure:
Hi - I tried using make xcodegen
to generate an Xcode project. It gives the following error:
error: could not find target(s): Source; use the 'path' property in the Swift 4 manifest to set a custom target path
Any idea what the problem is?
Thanks!
Hello,
I've discovered an issue causing a file to not be parsed that imports a constant
i.e. import let Foundation.NSNotFound
Best Regards,
Jovan
Semicolons not handled as expected.
Every line which ends with a semicolon is treated as an error.
class SemicolonIssue {
private let foo = "";
}
Command to run swift-ast
with the code above:
swift-ast SemicolonIssue.swift
A successfully parsed TopLevelDeclaration.
The parser failed with a badDeclaration error.
/Users/adfontes/Development/TDSoftware-GmbH/Apptracr/ios-monitoring-modul/ApptracrDemo/ApptracrDemo/swift-ast-issues/SemicolonIssue.swift:13:25 fatal: badDeclaration
See the above sample class.
The variable defined as optional fails when parsed because of a failure to detect an expected pattern. However, as I don't see anything wrong with this code.
Side-note, the Swift 4 compiler parses this file (swiftc -parse <FILE>
), I updated swift-ast to the latest version, and this file is not my own.
/// Various types of fields
/// that can be used in a Schema.
public struct Field {
public let name: String
public let type: DataType
public let optional: Bool
public let unique: Bool
public let `default`: Node?
public let primaryKey: Bool
public enum DataType {
case id(type: IdentifierType)
case int
case string(length: Int?)
case double
case bool
case bytes
case date
case custom(type: String)
}
public init(
name: String,
type: DataType,
optional: Bool = false,
unique: Bool = false,
default: Node? = nil,
primaryKey: Bool = false
) {
self.name = name
self.type = type
self.optional = optional
self.unique = unique
self.default = `default`
self.primaryKey = primaryKey
}
public init(
name: String,
type: DataType,
optional: Bool = false,
unique: Bool = false,
default: NodeRepresentable? = nil,
primaryKey: Bool = false
) {
let node: Node?
if let d = `default` {
node = try? d.makeNode(in: rowContext)
} else {
node = nil
}
self.init(
name: name,
type: type,
optional: optional,
unique: unique,
default: node,
primaryKey: primaryKey
)
}
}
extension Field: Equatable {
public static func ==(lhs: Field, rhs: Field) -> Bool {
return lhs.name == rhs.name
&& lhs.type == rhs.type
&& lhs.optional == rhs.optional
&& lhs.unique == rhs.unique
&& lhs.default == rhs.default
&& lhs.primaryKey == rhs.primaryKey
}
}
extension Field.DataType: Equatable {
public static func ==(lhs: Field.DataType, rhs: Field.DataType) -> Bool {
switch (lhs, rhs) {
case (.id(let a), .id(let b)):
return a == b
case (.int, .int),
(.string, .string),
(.double, .double),
(.bool, .bool),
(.bytes, .bytes),
(.date, .date):
return true
case (.custom(let a), .custom(let b)):
return a == b
default:
return false
}
}
}
extension IdentifierType: Equatable {
public static func ==(lhs: IdentifierType, rhs: IdentifierType) -> Bool {
switch (lhs, rhs) {
case (.int, .int),
(.uuid, .uuid):
return true
case (.custom(let a), .custom(let b)):
return a == b
default:
return false
}
}
}
Command to run swift-ast
with the code above:
swift-ast /mnt/d/swift_repos/72896077/fluent/Sources/Fluent/Schema/Field.swift
Parser fails on line 7:16
/mnt/d/swift_repos/72896077/fluent/Sources/Fluent/Schema/Field.swift:7:16 fatal: expectedPattern
How can I get an AST Node starting position(number of characters elapsed already before this node) in the input swift file ?
Do you plan to add support for Swift 4.2?
The code below is complete. Feel free to run it with swift foo.swift
to see that it's valid and works. However, the parser is not able to deal with the closure. Specifically the arguments to the closure seem to be confusing the parser.
[Detailed steps to reproduce the issue.]
let isPositive = { (_ value: Int) -> Bool in
return value > 0
}
print(isPositive(100))
Command to run swift-ast
with the code above:
swift-ast /Users/rudro/Downloads/foo.swift
What do you expect to happen as a result of the reproduction steps?
What currently happens as a result of the reproduction steps?
/Users/rudro/Downloads/foo.swift:1:23 fatal: expectedCloseParenTuple
Is your project open sourced? If yes, can you point us to your repository?
If not, is it possible to make a small project that fails the Travis CI?
If not, can you create a gist with your sample code for us?
Hello,
I've discovered an issue causing a file to not be parsed that contains an Xcode defined color literal
i.e. #colorLiteral(red: 0, green: 0, blue: 1, alpha: 1)
Best Regards,
Jovan
Hi, I've followed the installation steps for "standalone tool", but when I tried the command line usage I got: "-bash: swift-ast: command not found".
I'm running swift-as on MacOSX High Sierra, with a swift version of 4.1.2
Hi, it appears that swift comments (and also whitespace) are just ignored by the various visitors, e.g. swift-ast -print-ast filename
. This makes it unusable as a simple swift-to-other-language translator, which is a stated goal of the project.
How can I enable comments to be included in the output?
Thanks.
Hi, I played with swift-ast
, but I don't see any way to somehow parse AST output. Is there an option to dump e.g. json of AST? Thanks
swift-ast -github-issue /path/to/file.swift
might help me generate this file[Insert a brief but thorough description of the issue]
[Detailed steps to reproduce the issue.]
[Insert sample source code here]
Command to run swift-ast
with the code above:
swift-ast sample.swift
What do you expect to happen as a result of the reproduction steps?
What currently happens as a result of the reproduction steps?
[
Insert the current outputs and
maybe other information that
could help us understand the issue better
]
Is your project open sourced? If yes, can you point us to your repository?
If not, is it possible to make a small project that fails the Travis CI?
If not, can you create a gist with your sample code for us?
Semicolons not handled as expected.
Every const_decl/var_decl which ends with a semicolon is treated as an error.
struct SemicolonIssue {
private let issue = "";
}
Command to run swift-ast
with the code above:
swift-ast SemicolonIssue.swift
A successfully parsed TopLevelDeclaration.
The parser failed with a badDeclaration error.
SemicolonIssue.swift:13:25 fatal: badDeclaration
swift-ast -github-issue /path/to/file.swift
might help me generate this file[Insert a brief but thorough description of the issue]
UInt64 literals such as 0xffffffffffffffff, cause runtime overflow errors at
[Detailed steps to reproduce the issue.]
[Insert sample source code here]
Command to run swift-ast
with the code above:
swift-ast sample.swift
What do you expect to happen as a result of the reproduction steps?
What currently happens as a result of the reproduction steps?
[
Insert the current outputs and
maybe other information that
could help us understand the issue better
]
Is your project open sourced? If yes, can you point us to your repository?
If not, is it possible to make a small project that fails the Travis CI?
If not, can you create a gist with your sample code for us?
When parsing sources, which are using .~
operator form https://github.com/kickstarter/Kickstarter-Prelude/blob/master/Prelude/Operators.swift
Parser returns fatal expected member name following '.
Examples of sources:
https://github.com/kickstarter/ios-oss/blob/master/Kickstarter-iOS/Views/Cells/CommentCell.swift#L28 and https://github.com/kickstarter/ios-oss/blob/master/KsApi/models/templates/CategoryTemplates.swift#L26
Gets confused by extension on Integer. What's interesting is the if you change 0.foo
to 1.foo
or some other number, it seems to work.
[Detailed steps to reproduce the issue.]
extension Int {
var foo: Int {
return 10
}
}
print(0.foo)
Command to run swift-ast
with the code above:
swift-ast /Users/rudro/Downloads/foo.swift
What do you expect to happen as a result of the reproduction steps?
What currently happens as a result of the reproduction steps?
/Users/rudro/Downloads/foo.swift:7:10 fatal: expectedCloseParenFuncCall
Is your project open sourced? If yes, can you point us to your repository?
If not, is it possible to make a small project that fails the Travis CI?
If not, can you create a gist with your sample code for us?
The parser fails with expected expression
for syntax like the following:
return uti.flatMap(mimeType(uti:))
I included a sample of code from kickstarter/ios-oss, and have a few additional examples included below.
The sample code does seem to run successfully with swift 4.
import Foundation
private func mimeType(extension: String, where: CFString? = nil) -> String? {
let uti = UTTypeCreatePreferredIdentifierForTag(kUTTagClassFilenameExtension,
`extension` as CFString,
`where`)?.takeRetainedValue()
return uti.flatMap(mimeType(uti:))
}
private func mimeType(uti: CFString) -> String? {
return UTTypeCopyPreferredTagWithClass(uti, kUTTagClassMIMEType)?.takeRetainedValue() as String?
}
Command to run swift-ast
with the code above:
swift-ast /Users/chrishulton/code/testing/swift_qa/failing_cases/expected_expression.swift
The code should be able to be parsed.
There is a fatal error: expected expression
/Users/chrishulton/code/testing/swift_qa/failing_cases/expected_expression.swift:7:35 fatal: expectedExpr
Here are a few additional failing files for this case:
Let me know if I can provide any more information! Thanks!
The sample code included is complete. Run with swift foo.swift
to verify this is valid code. Looks like the typealias
line inside the protocol
causes the error as it does not appear to be a "typical" protocol member.
[Detailed steps to reproduce the issue.]
protocol MyProtocol {
typealias Number = Int
func update(value: Number)
}
print("hello")
Command to run swift-ast
with the code above:
swift-ast /Users/rudro/Downloads/foo.swift
What do you expect to happen as a result of the reproduction steps?
What currently happens as a result of the reproduction steps?
/Users/rudro/Downloads/foo.swift:2:5 fatal: badProtocolMember
Is your project open sourced? If yes, can you point us to your repository?
If not, is it possible to make a small project that fails the Travis CI?
If not, can you create a gist with your sample code for us?
swift-ast -github-issue /path/to/file.swift
might help me generate this filethe ast output of a protocol method name comes out as same as protocol name
protocol TableUpdatable {
func reloadTable()
func errorLoadingData()
}
Command to run swift-ast
with the code above:
swift-ast sample.swift
proto_decl
name: TableUpdatable
0: kind: method
name: reloadTable
1: kind: method
name: errorLoadingData
proto_decl
name: TableUpdatable
0: kind: method
name: TableUpdatable
1: kind: method
name: TableUpdatable
I get the following:
unable to invoke subcommand: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift-build (No such file or directory)
$ swift-ast test.swift
swift-ast/test.swift:11:15 fatal: expected ']' in expression list
let b = a[safe: i] ?? 0
^~~~
If you think this is a bug, please run
swift-ast -github-issue test.swift
and file a GitHub issue.
$ swift test.swift
b is 1
This is test.swift
extension Collection {
/// Returns the element at the specified index iff it is within bounds, otherwise nil.
subscript (safe index: Index) -> Element? {
return indices.contains(index) ? self[index] : nil
}
}
let a = [0, 1, 2]
let i = 1
let b = a[safe: i] ?? 0
print("b is \(b)")
swift-ast -github-issue /path/to/file.swift
might help me generate this fileWhen running Swift-AST tests with Xcode10, there is an infinite loop in the Token.Kind
equatable extension. In concrete, it gets stuck in Sources
-> Lexer
-> TokenKind+Equatable.swift
lines 19, 23 and 145
Build, install and run tests. The infinite loop happens just by running tests. (Better to do it inside Xcode GUI because it stops and lets to inspect. From console it just finishes with an Exited with signal code 11
Run tests normally
Exited with signal code 11
/ infinite loop
Detected from https://github.com/angelolloqui/SwiftKotlin when upgrading to Xcode10
A long decimal number like -0.5773502691896257310588680411456152796745
causes an EXC_BAD_INSTRUCTION
in the lexer at this line:
let decimal = (fractionalPart?.decimal ?? 0) * rdx + value
See the sample code, it's complete and can be run with a simple swift foo.swift
to see the output. Note that the case statement confuses the parser.
[Detailed steps to reproduce the issue.]
import Foundation
func foo(_ anchorPoint: CGPoint) {
switch anchorPoint {
// Top Left Corner
case CGPoint(x: 1, y: 1):
print("top left")
default:
print("other")
}
}
foo(CGPoint(x: 1, y: 1))
Command to run swift-ast
with the code above:
swift-ast /Users/rudro/Downloads/foo.swift
What do you expect to happen as a result of the reproduction steps?
What currently happens as a result of the reproduction steps?
/Users/rudro/Downloads/foo.swift:6:17 fatal: expectedCaseColon
Is your project open sourced? If yes, can you point us to your repository?
If not, is it possible to make a small project that fails the Travis CI?
If not, can you create a gist with your sample code for us?
swift-ast -github-issue /path/to/file.swift
might help me generate this fileThe file attached to this issue can be successfully built and executed using swiftc
with no additional parameters.
Parse the included file using swift-ast -dump-ast <file>
and observe that it fails to parse the value matching in the catch clause.
enum VendingMachineError: Error {
case insufficientFunds(coinsNeeded: Int)
}
func throwsSomething() throws -> String {
return "This doesn't matter"
}
do {
try throwsSomething()
}
catch VendingMachineError.insufficientFunds(3) {
print("3 coins missing")
}
catch VendingMachineError.insufficientFunds(let coinsNeeded) {
print("Not 3 coins missing")
}
catch {
print("No coins missing")
}
Command to run swift-ast
with the code above:
swift-ast /home/nathan/value_ex.swift
A valid ast dump for the test file, it appears that swift-ast does not correctly parse the type instantiation for the value matching.
/home/nathan/value_ex.swift:17:45 fatal: expectedPattern
The ANTLR-Swift-Target was merged into antlr https://github.com/antlr/antlr4 official repo.
https://github.com/johndpope/Antlr-Swift-Runtime
I built the java / golang / proto parser here
https://github.com/johndpope/Antlr-Swift-Runtime/tree/master/Test/Test
other languages here
https://github.com/johndpope/ANTLR-Swift-Target/tree/master/gen/grammars-v4
I'm interested to do machine learning language translation DRAGGNN with tensorflow by training on generated languages from one -> then translating the AST in SWIFT. eg. give me a golang class -> spit out the corresponding swift file.
let me know if your interested
made some progress here - https://github.com/nubbel/swift-tensorflow/
Dear people, I want to get the data type of swift source code, but it's seems difficult if don't use swift intermediate code. If exists a tool can convieniently get data type or some solutions to get data type of swift source code
The issue has same issue.
#19 @yanagiba @ryuichis
Thank you
$ cat test.swift
let a = 10
let zero = 0
switch a {
case zero..<5:
print("less than 5\n")
default:
print("more than 5")
}
$ swift test.swift
more than 5
$ swift-ast test.swift
/Users/obonilla/o/swift-ast/test.swift:4:12 fatal: expected ':' after case items
case zero..<5:
^~~~
Hello,
I've discovered an issue causing a file to not be parsed that contains an Xcode defined image literal
i.e. #imageLiteral(resourceName: "SomeResource")
Best Regards,
Jovan
swift-ast -github-issue /path/to/file.swift
might help me generate this fileThe parser fails with expected expression for syntax like the following:
guard let self = self else { return }
Sample code
networkRequest.fetchData() { [weak self] result in
guard let self = self else { return }
switch result {
case .Succeeded(let data):
self.processData(data)
case .Failed(let err):
self.handleError(err)
}
}
It could be really nice if the parser could parse vaild Swift code - upgrade self from weak to strong.
https://github.com/apple/swift-evolution/blob/master/proposals/0079-upgrade-self-from-weak-to-strong.md
The code should be able to be parsed.
There is a fatal error: expected expression
swift-ast -github-issue /path/to/file.swift
might help me generate this fileThe parsing of type matching in the catch clause pattern fails, while swift itself can compile the file.
Parse the included code example.
enum someErr: Error {
case woops
}
func badstuffhappens() throws -> String {
throw someErr.woops
}
try? badstuffhappens()
try! badstuffhappens()
do {
try badstuffhappens()
} catch someErr.woops {
}
catch is someErr {
}
Command to run swift-ast
with the code above:
swift-ast /home/nathan/hw.swift
Parsing fails because the is
keyword is not expected.
/home/nathan/hw.swift:26:7 fatal: expectedPattern
Shebangs are supported in swift files.
$ cat test.swift
#!/usr/bin/swift
print("hello world!")
$ chmod +x test.swift
$ ./test.swift
hello world!
$ swift test.swift
hello world!
However, swift-ast
fails to parse a file with shebangs.
$ swift-ast test.swift
/Users/obonilla/o/swift-ast/test.swift:1:2 fatal: expected a valid keyword after '#' in compiler-control statement
#!/usr/bin/swift
^~~~
Do you know if we can currently do type resolutions for the AST using the Standard Library (perhaps by providing some sort of path to it) to allow for type resolutions beyond primitive type?
swift-ast -github-issue /path/to/file.swift
might help me generate this fileI am using swift-ast
to parse Swift source code, and for that, it works great. However, I also need to be able to parse *.swiftinterface
files, as that is what I can generate from Objective-C headers. Since I only care about the public interface, this is ideal. However, I can't parse these files with swift-ast
because it exits with a DiagnosticStopper
error.
I've had to try modifying the swiftinterface files to make them "look" like compilable Swift, for example, by adding empty brackets {} after function declarations. In this way I've been able to hackily use swift-ast
for my purposes, but it's much too brittle to be effective.
I tried using SourceKitten directly to examine the structure, which was very promising as it works the same for swift and swiftinterface, but unfortunately it seems to be missing a bunch of stuff that I am currently able to get from swift-ast.
Run swift-ast
on any *.swiftinterface
file
Code is parsed and I can navigate the tree, albeit with less detail.
DiagnosticStopped exit.
'postfix' as constant/variable name doesn't work.
Every constant/variable named 'postfix' is treated as an error.
struct PostfixIssue {
private let postfix = ""
}
Command to run swift-ast
with the code above:
swift-ast PostfixIssue.swift
A successfully parsed TopLevelDeclaration.
The parser failed with an expectedPattern error.
PostfixIssue.swift:13:17 fatal: expectedPattern
I'm trying to parse tests form upstream swift repository and with swift-ast and found a crash of files https://github.com/apple/swift/blob/master/test/SILOptimizer/diagnostic_constant_propagation_int.swift and https://github.com/apple/swift/blob/master/test/SILOptimizer/diagnostic_constant_propagation.swift
This files are also attached in the archive.
swift-ast -github-issue /path/to/file.swift
might help me generate this fileswift-ast cannot parse function declarations with Swift 5.5's async
keyword.
Try to parse:
func foo() async
What do you expect to happen as a result of the reproduction steps?
Should parse successfully.
What currently happens as a result of the reproduction steps?
Diagnostic Stopper Error due to bad declaration
swift-ast -github-issue /path/to/file.swift
might help me generate this fileTry to parse:
class SomeClass: @unchecked Sendable
It will fail due to the @unchecked
attribute. It doesn't seem the swift-ast type inheritance parser accepts attributes.
Command to run swift-ast
with the code above:
swift-ast sample.swift
What do you expect to happen as a result of the reproduction steps?
What currently happens as a result of the reproduction steps?
[
Insert the current outputs and
maybe other information that
could help us understand the issue better
]
Is your project open sourced? If yes, can you point us to your repository?
If not, is it possible to make a small project that fails the Travis CI?
If not, can you create a gist with your sample code for us?
swift-ast -github-issue /path/to/file.swift
might help me generate this file[Insert a brief but thorough description of the issue]
[Detailed steps to reproduce the issue.]
[Insert sample source code here]
Command to run swift-ast
with the code above:
swift-ast sample.swift
What do you expect to happen as a result of the reproduction steps?
What currently happens as a result of the reproduction steps?
[
Insert the current outputs and
maybe other information that
could help us understand the issue better
]
Is your project open sourced? If yes, can you point us to your repository?
If not, is it possible to make a small project that fails the Travis CI?
If not, can you create a gist with your sample code for us?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.