gleam-lang / tree-sitter-gleam Goto Github PK

When editing a source file, using attributes can cause odd syntax highlighting failures. For instance, using the @external attribute to mark an external function will cause the rest of the source file to be parsed incorrectly unless the function defines a fallback implementation.

A solution to this problem would likely be to attach attributes to legal parent nodes, such as functions and statement blocks. This would allow tree-sitter to more consistently "complete" a tree as soon as it matches a syntax node.

Hello!

v0.25 introduces a new feature called use. It looks like this:

use <- expression
use a <- expression
use a, b, c <- expression

With some code like so

<<code:int-size(8)-unit(2), reason:utf8>>

which gives

(bit_string
  (bit_string_segment
    value: (identifier
    options: (bit_string_segment_options
      (bit_string_segment_option_int)
      (bit_string_segment_option_size
        (integer))
      (bit_string_segment_option_unit)))
  (bit_string_segment
    value: (identifier)
    options: (bit_string_segment_options
      (bit_string_segment_option_utf8)))))))

I'd like the named nodes within the options field of bit_string_segment to be a little more generic so I can capture them in a highlight query and give them a special highlight. Something like

; captures unit, "size", "utf8", "int", etc.
(bit_string_segment_option) @function.builtin

so it might be parsed like so instead:

(bit_string
  (bit_string_segment
    value: (identifier
    options: (bit_string_segment_options
      (bit_string_segment_option)
      (bit_string_segment_option
        (integer))
      (bit_string_segment_option)))
  (bit_string_segment
    value: (identifier)
    options: (bit_string_segment_options
      (bit_string_segment_option)))))))

The nodes come out to be less specific than they are currently, but I think that's ok because you could do something like:

((bit_string_segment_option) @utf8
 (#eq? @utf8 "utf8"))

What do you think, would you be open to a PR that makes a refactor like this?

Hello!

This is a feature added in v0.26.0

An example of it can be found here:

https://github.com/gleam-lang/gleam/blob/72756d5e123853fc66c8810f783cc137be05a5f4/test/language/src/main.gleam#L114-L125

When I have code like:

    http.Get, [".well-known", "webfinger"] ->
      case host {
        value if value == config.connection.host -> web_finger_page(req, config)
        _ -> wisp.not_found()
      }

in a case statement, the syntax highlighting fails on the second . in config.connection.host.

Louis was able to reproduce it and I did my best to check with the latest commit from this repo via an entry in the Helix languages.toml.

Grateful for Gleam, this project and Helix :)

Hello there!

Gleam's parser has been changed to only permit let and use directly within functions and block. Code such as this will now be rejected:

let x = let y = 1

This is a subset of the previously accepted syntax so the current tree sitter will still parse all valid programs, but perhaps perhaps it wants to be restricted also?

use #(a, b) <- blah

:)

I noticed some strange behavior with the syntax highlighting of records vs records with functions. Consequently, I tried to introduce this test:

fn thing() {
  let local_record = Bar(thing: fn(x) { x + 1 })
  local_record.thing(5)
  // ^ variable
  //           ^ property
}

However, this test fails with this error:

  ✗ functions.gleam
    Failure - row: 44, column: 15, expected highlight 'property', actual highlights: 'function'

So the syntax highlighter is able to determine that local_record is a variable, not a module, but it is not able to extend that inference to imply that therefore the second part of the field_access must be a property instead of a function.

Hello!

The upcoming Gleam release as 2 syntax changes. panic (which can go anywhere an expression goes, like todo but without the string argument), and let assert <pattern> = <expression>, which is the new syntax for assert <pattern> = <expression>.

Thanks!

Hello!

A new syntax has been added to Gleam, the attribute syntax

@target(erlang)
pub fn main() { todo }

Attributes proceed imports or definitions in modules. The compiler only accepts @target(erlang) and @target(javascript) but I believe the tree sitter should accept any name that would be a valid function name in place of target and any comma-delimited sequence of optionally labelled constant expressions within the ().

@deprecated(since: "1.2.0", replacement: wobble)
pub fn wibble() { todo }

Hello!

Gleam has some reserved keywords. Would it make sense to add these to the grammar? So they get highlighted appropriately if used.

https://github.com/gleam-lang/gleam/blob/0008b79d2eedfa2dd4d13efd67fa063e7ada285d/compiler-core/src/parse.rs#L2792-L2793

Thanks,
Louis

Hello there!

Starting with Gleam v0.32 the syntax for unqualified importing a type will be import modulename.{type TypeName}.

The existing import modulename.{Name} will refer to the value constructor, not the type.

Thanks!

Howdy! As you know, I am working on integrating this grammar with neovim https://github.com/J3RN/tree-sitter-gleam/issues/1, during which I realized there is no node for string escape sequences. Without this node, I cannot highlight escape sequences similar to what I did with the nvim-treesitter Elixir highlight queries.

For example, tree-sitter-elixir parses:

"foo\nbar"

as the following tree of nodes:

(string
  quoted_start: """
  (quoted_content)
  (escape_sequence)
  (quoted_content)
  quoted_end: """)

I'm curious what you think!

Hello! I found an instance in which a function name does not get highlighted. It's not clear to me why this is right away

https://github.com/gleam-lang/stdlib/blob/f9d57db847ff2a8471c9136bd309c870b610d04d/src/gleam/uri.gleam#L325

When opening a gleam file with the latest commit, I get the following error:

This query could not be parsed: QueryError { row: 96, column: 3, offset: 2086, message: "try", kind: NodeType }

The Version 0.32.0 works fine, it has broken after that commit.
Example file: https://github.com/philipgiuliani/glubs/blob/55edb1e95d6b8ec66daa0e8aa2a13a008aeafad2/src/glubs/webvtt.gleam
Editor: helix 23.10 (f6021dd0)

It looks like it stopped working at this commit: b7ae2c9

The new updates seem to be working really well! One spot where I'm seeing a parsing error is with @external.

It seems like the fn ... block is failing to parse, I'm guessing due to missing curly braces. If I add those, it parses successfully, but I obviously then get a syntax error from gleam.

It seems like the function isn't nested below the attribute so I'm not sure how easy this would be to change.

Thank you!

Assuming wrapper accepts a callback with a generic argument (eg fn (event) -> Nil):

I can do this:

fn test() {
  wrapper(fn (event : ClickEvent) {  do_something(event) })
}

but I also can do this:

fn test() {
  use event : ClickEvent <- wrapper()
  do_something(event)
}

TS parsing will break down on such code - removal of : ClickEvent fixes it, but it's necessary.

https://github.com/gleam-lang/stdlib/blob/c3f4f1b46443d70f0919d5646b3d4a4cc93d34d8/src/gleam/float.gleam#L247

After this float literal there is no more syntax highlighting.

Hello!

The latest Gleam release has added a new syntax, the <> operator.

Could we update this library to support it please? 🙏

https://github.com/gleam-lang/gleam/blob/6202f1b7b2aa15767b5a45410e870060befb4fa3/compiler-core/src/parse.rs#L847

https://github.com/gleam-lang/gleam/blob/6202f1b7b2aa15767b5a45410e870060befb4fa3/compiler-core/src/parse.rs#L2735

Thanks!

Highlighting seems to be broken in the gleam core language tests:

https://github.com/gleam-lang/gleam/blob/0f20f6f46fa9c9e4baa6f264dd49e59ecbd9b0a5/test/language/src/main.gleam#L84

It looks like it stops after 0., so it could be that floats without values after the decimal aren't being parsed properly

Hello!

The syntax for adding a message to panic and todo is now this

todo as "handle the case when..."
panic as "this is unreachable as..."

Currently only a string is permitted on the right hand side, but perhaps in future an expression would be supported. It may be good to have the tree sitter support parsing an expressions there to be more permissive than the compiler parser.

Hello! I'm curious if there are any plans to integrate this grammar with Neovim via nvim-treesitter. If not, I'm happy to help out with this 😄 Thanks!

I'll preface this with "I don't really know if there will be a resolution here". Also far from an expert in tree-sitter.

This is neovim specifically, but I imagine any indent query will run into the same issue. The grammar defines repeat1 for case_clauses, which I am guessing means 1 or more? That's the node used to determine the beginning of indentation. Before any clauses have been added, there is nothing to match on to signify that we should start indentation.

It's not valid Gleam to have an empty case, so changing that doesn't seem correct?

I guess this is more of a question/discussion issue. This functionality does work in VSCode, but obviously that is completely different. Just noting that it's not particularly unusual behavior.

Is there anything that can be changed in the grammar to handle this? If it's also a specific indent query issue, I can try to look more into that.

This might also end up just being a shortcoming of the grammar and indentation, but was just curious to get some info from more knowledgeable people :)

Gleam features a pub modifier for constants, types, functions (internal and external), type definitions, and type aliases. Additionally, it also features an opaque modifier for type definitions and type aliases.

Presently, an construct with a modifier is represented as a different base AST node than the same construct without a modifier. For example:

const foo = "bar"

is represented as

(constant
 name: (identifier)
 value: (string
  (quoted_content)))

and the version with the public modifier:

pub const foo = "bar"

is represented as

(public_constant
 name: (identifier)
 value: (string
  (quoted_content)))

In the interest of usability, we should move to make constructs with and without a modifier share the same base AST node while the construct with the modifier should have an additional child node for the modifier.

e.g.

pub const foo = "bar"

should be represented as

(constant
 (visiblity_modifier)
 name: (identifier)
 value: (string
  (quoted_content)))

This behavior also exists in tree-sitter-rust.

Hello!

The external type syntax has been reworked.

// Previous
pub external type One

// New
pub type One

Thank you

Looks like we may be able to add this to GitHub to get better support for Gleam.

https://twitter.com/importantshock/status/1496884996317011968

Create an issue on your repo and CC me (/patrickt on GitHub) or /dcreager.

Is this something the grammar and queries are ready for?