Code Monkey home page Code Monkey logo

Comments (7)

davidanthoff avatar davidanthoff commented on May 18, 2024

I completely agree, there seem to be a lot of false positives from Lint.jl...

I do think there is value in having a package that can lint a file outside of an IDE, eventually folks might want to integrate that e.g. in the build process. So if you end up writing a new linter that is based on a new JuliaParser, it might make sense to just keep that in mind and offer an option that this new linter can also be called and used independently from the LS?

I guess another option would be to improve Lint.jl and try to at least get rid of all these false positives, which I find the most annoying. Maybe we should just open issues over there and hope someone picks it up in the short run.

But completely on board with the medium term goal here.

from languageserver.jl.

oxinabox avatar oxinabox commented on May 18, 2024

@ZacLN For your interest I have a CST prototype based on Tokenizer.jl here: https://github.com/oxinabox/Expr2LaTeX.jl/blob/master/JuliaCST.ipynb

Reason I didn't start by stripping JuliaParser, is that it immediately discards all whitespace.

I'm not sure my prototype is directly useful to you, but poke me if you want to talk CST stuff.

from languageserver.jl.

ZacLN avatar ZacLN commented on May 18, 2024

Thanks @oxinabox, I'm pretty much done with the core parser now (just a matter of trawling through packages to pick up the odd parsing mismatch). I've ended up using pretty much the same patterns as JuliaParser (I think a recursive descent parser?).

Am I right in reading your approach as finding all boundaries of expressions before parsing them, i.e. top down in some sense? Does this make things easier in terms of incremental parsing and eliding syntax errors?

from languageserver.jl.

oxinabox avatar oxinabox commented on May 18, 2024

Yes, the approach was to get expression boundries, because what got me started was wanting to redisplay code as LaTeX, which can only be done on a per expression basis.
Expressions seems the logical building block, since you it is a "whole" thing.
Any expression can be evaled.
Any expression can be parsed to get an AST.

I am excited to hear that your CST Parser is basically done.
Where is the code for that? (I feel like is should be in its own package, no?)

from languageserver.jl.

ZacLN avatar ZacLN commented on May 18, 2024

https://github.com/ZacLN/Parser.jl is the repo.
Some basic functionality:

using Parser
str = readstring(joinpath(Pkg.dir("Tokenize"),"src","lexer.jl"))
x, ps = Parser.parse(ParseState(str), true) 

# convert to normal AST
x0 = Expr(x)

# Formatting hints
ps.hints
str2 = Parser.Hints.apply(ps.hints, str)

# formatting hints have no effect on AST
Expr(Parser.parse(str2, true)) == x0

# returns: the object, y, at byte 434
#              the stack of expressions that contain it, h
#              the sequence of nodes within these expressions, inds
y, h, inds = find(x, 434);

# Gets the scope at byte position 22514
S = Parser._get_full_scope(x, 22514);

# convert the names/signatures to standard AST, repetitions are where something is overwritten. 
(x->Expr(x.id)).(S)

EDIT: that example was broken, had to push a quick update

from languageserver.jl.

ZacLN avatar ZacLN commented on May 18, 2024

The internals are essentially the same as with normal AST except that we have IDENTIFIERs instead of symbols, LITERALS to capture all numbers/strings,etc so that all positional info can be stores. You can iterate across an EXPR in terms of the sub-expressions/lexemes in the order that they visually appear

from languageserver.jl.

ZacLN avatar ZacLN commented on May 18, 2024

#48

from languageserver.jl.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.