getgrit / gritql Goto Github PK

Similar to the only_in_json flag, we should have a only-in-diff flag that can point to a git diff to limit the apply to.

The contributing guide covers from Grit-specific things, but doesn't provide a brief introduction to building the repo and making contributions.

We should provide a step by step guide for new contributors.

We would like to support PHP as a language for matching against.

Review this guide and join us on Discord to discuss.

Acceptance criteria

1. Working grammar + language support
2. At least 10 test cases, including rewrites and metavariables.

thanks for putting this together; it looks awesome!

unfortunately, the model of downloading binaries and running them is a moderately incompatible with nixos; it would be really nice if there was an alternative way of getting all the dependencies

some ideas:

• add a flake.nix that builds the projects and all the deps
• create a docker image with all the necessary deps so that can be a workaround
• ???

thanks :)

Summary

Invalid rewrites when trying to remove a node.

Repro

https://app.grit.io/studio?key=Mtz14P9InB2-2vdJYtxLk

engine marzano(0.1)
language python


pattern test_func($name, $parameters, $return_type, $body) {
    function_definition($name, $parameters, $return_type, $body) as $function where {
        $name <: r"test_.*",
    }
}

pattern test_class($name, $superclasses, $body) {
    class_definition($name, $superclasses, $body) as $cls where {
        $superclasses <: contains `TestCase`,
    }
}

test_func($return_type) as $function where {
    $function <: within test_class(),
    $return_type => .
}

And target:

from unittest import TestCase

class TestFirst(TestCase):
  def test_typed(self) -> None:
    pass

  def test_another(self):
    pass

Produces the following:

The dangling -> : is invalid, the -> should be removed as well.

I also noticed this in some cases while removing some kwarg from a list of kwargs -- unfortunately I don't have a repro on that though. Removing a kwarg in some cases left a dangling comma, i.e. Something(a=1, b=2, c=3) -> "remove kwarg a" -> Something(, b=2, c=3) invalid.

CI currently only ensures that we build wasm. We should have a few javascript tests that run a simple patterns for each language that is supported by wasm.

Example query to find functions that have a first arg of string

`function ($arg) { $_ }` where {
  $arg <: type `string`
}

$next_name should be passed fine here: https://app.grit.io/studio?key=AX1KMHI0HZRz1FYUGB99E

Instead it somehow is not resolved.

CSS patterns should be able to match <style> tags inside .vue files, similar to how we implemented it for JS:

Summary

Blank lines break function rewrite.

Repro

With gritql query:

engine marzano(0.1)
language python

pattern resolver_func($name, $parameters, $return_type, $body) {
    function_definition($name, $parameters, $return_type, $body) as $function where {
        $name <: "resolve",
        $return_type <: "Out1 | None"
    }
}

pattern change_signature() {
    resolver_func($return_type, $body, $name, $parameters) => 
    `
def $name($parameters) -> Out2 | None:
    # TODO: fix body
    $body
    `
}

change_signature()

And test Python code:

from dataclasses import dataclass


@dataclass(frozen=True)
class Out1:
    res: str


@dataclass(frozen=True)
class Out2:
    res: str


@dataclass(frozen=True)
class A:
    a: str

    def resolve(self, a: str, b: str) -> Out1 | None:
        1 + 1
        print("hi")

        1 + 1

        return Out1("hi")


@dataclass(frozen=True)
class B:
    a: str

    def resolve(self, a: str, b: str) -> Out1 | None:
        return Out1("hi")

I get the following error message:

Error expected line to start with 8 spaces, code is either not indented with spaces, or does not consistently indent code blocks

Rewriting A as:

@dataclass(frozen=True)
class A:
    a: str

    def resolve(self, a: str, b: str) -> Out1 | None:
        1 + 1
        print("hi")
        1 + 1
        return Out1("hi")

Then it works as expected.

Seems to be coming from here ish:

We've had pull requests that if merged would have broken patterns used in the standard library, we should add a CI check that runs grit patterns test in CI.

blocked by: #126

Hi,

I want to try GritQL to transform all possible elements to be self closed. I've checked https://app.grit.io/studio?preset=prefer_self_closing_tag_jsx&key=OR1LvWpzVk9vLiS6tgCZ9

but it seems like it's stripping *, any advice ?

    <ion-spinner *ngIf="isLoading" color="white"></ion-spinner>

It appears that pattern markdown files can't start with a number, and can't contain a hyphen. If they do it creates an error when you try to run it.

To load files from remote locations, we likely need to do it asynchronously. This call will need to return a future:

We would like to support C++ as a language for matching against.

Review this guide and join us on Discord to discuss.

Acceptance criteria

1. Working grammar + language support
2. At least 10 test cases, including rewrites and metavariables.

Currently, if you have a syntax error in a query the error you receive is not very helpful - just a location.

Error Pattern syntax error at 13:28. If you hit this error while running grit apply on a pattern from the Grit standard library

We would like to have much better error messages for syntax errors.

For example:

Syntax error at 13:28, received [ but expected one of {, {.

There's some prior discussion here:

• tree-sitter/tree-sitter#255

When you try the following pattern described in the docs for sequential it doesn't work in the studio.

You also get the following warning.

Log Warning: sequential matches at the top of the file. If a pattern matched outside of a 
sequential, but no longer matches, it is likely because naked patterns are 
automatically wrapped with `contains bubble <pattern>`

This warning is somewhat unclear, as it doesn't really tell the user how they should proceed to make things work the way they expect it to.

In JSON, empty strings in snippets become wildcard matches:

engine marzano(0.1)
language json

`"x": ""`

This will match "x": "" but also "x": "foo". When we find an empty node in a snippet we probably want to restrict it to only match other empty nodes.

engine marzano(0.1)
language js

pattern traverse($accumulated_name) {
    or {
        `describe($name, $describe)` where {
            $next_name=`alf`,
            $describe <: contains traverse($accumulated_name) until `describe($_, $_)`
        },
        `it($final, $_)` where {
            $final => `$accumulated_name`
        }
    }
}

file($body) where {
    $body <: contains bubble traverse(accumulated_name=`d`) until `describe($_, $_)`
}

Input file

describe("My describe", () => {
  const input = "5";

  describe("foo", () => {
    it("stops here")
  });

  describe("nested more", () => {
    describe("something...", () => {
      it('branch 2', () => {});
      it('branch 3', () => {});
    })
  });
});

Currently, if a field is null we can't rewrite/insert into it (because we don't have an associated range for a field that wasn't found).

Workaround

Rewrite the entire parent node that contained the null field.

Example

engine marzano(0.1)
language python

pattern test_func($name, $parameters, $return_type, $body) {
    function_definition($name, $parameters, $return_type, $body) as $function where {
        $name <: r"test_.*",
    }
}

pattern test_class($name, $superclasses, $body) {
    class_definition($name, $superclasses, $body) as $cls where {
        $superclasses <: contains `TestCase`,
    }
}

test_func($return_type) as $function where {
    $function <: within test_class(),
    $return_type => `None`
}

I'm setting up grit for the first time and trying to test a pattern I've created.
When I run the grit, I get an error

$ grit apply props_use_swagger

Error: Failed to fetch standard library grit module getgrit/stdlib: Failed to clone repo getgrit/stdlib: authentication required but no callback set; class=Ssh (23); code=Auth (-16)

I've also tried logging in through grit auth to see if it fixes it, but it hasn't.

Note that I've installed and am using grit with nushell, but I seem to have the same issue when running through zsh.

This is the only pattern I have

// ~/.grit/patterns/<redacted>/common.grit

language js

// Find cases where a member of a prop is imported from swagger types
pattern props_use_swagger() {
  `interface $interface { $entries }` where { and {
    `interface` <: r`Props$`,
    `entries` <: contains `$name: $type` where {
      $type <: ensure_import_from("@/types/swagger")
    }
  } }
}

this test:

    #[test]
    fn print_sexp() {
        let code = r#"<Hello attr="foo"></Hello>"#;
        let mut parser = tree_sitter::Parser::new().unwrap();
        let lang = tree_sitter_html::language().into();
        parser.set_language(&lang).unwrap();
        let tree = parser.parse(code, None).unwrap().unwrap();
        let root = tree.root_node();
        let sexp = root.to_sexp();
        assert!(!sexp.contains("ERROR"));
        println!("{sexp}");
    }

fails even though the snippet is valid html, and parses correctly through the tree-sitter cli.
commenting out the vue module in in language/src/lib.rs fixes this which is odd as the test does not make use of any functionality outside of tree-sitter

We would like to support HTML as a language for matching against.

Review this guide and join us on Discord to discuss.

Acceptance criteria

1. Working grammar + language support
2. At least 10 test cases, including rewrites and metavariables.

The standard library has patterns that use ai, but ai functionality is not open source, so running grit patterns test using an open source build fails to run, as the patterns which make use of ai fail to compile. easiest fix (although a bit hacky) would be to filter out tests that don't compile, and list them as skipped due to compilation failures at the end of the report.

We should test the Grit standard library before merging.

By default .grit the cli is looking up the .grit in your current folder or in the root of your repo. I think this is a sane default. However I also think it would be nice to define a custom path for your .grit. Perhaps you want your .grit committed in a repo of it's own, or perhaps you would always like to look up patterns from ~/.grit.

I discovered a dependency on the embeddings crate, which seems to be an empty placeholder: https://github.com/getgrit/gritql/blob/main/crates/core/Cargo.toml#L17

Is this something that is still supposed to be initialized, or something that should be cleaned up?

This should find many results in our monorepo:

`$client.$_($_)` as $call where {
  $client <: imported_from($source),
  $source <: includes "client"
}

People inexplicably like to write their DSLs in yaml, compile yaml down to grit

NB please do not edit the editor from this link. Copy a forked version to experiment.

https://app.grit.io/studio?key=hWu-nA-6hu6_esJtvoXag

See playground here: https://app.grit.io/studio?key=SagGy0msuq3fQLmcDXA8z

As you can see trying to match a non-anonymous default export function is currently matching the same as export function.

The following pattern

engine marzano(0.1)
language python

`def $method($args): $body` => `def $method($args): $body`

Should return the exact same thing, but that is not the case as you can see in the following example. https://app.grit.io/studio?key=m44jIoo2l3JuT1lcAXkEm

For simple inline patterns, needing to specify the language in the declaration is a hindrance.

For example:

$ grit apply 'language css
r"--red-(.+)"($var)'

This would be more ergonomic:

$ grit apply 'r"--red-(.+)"($var)'' --language css

It would be good to have support for Dart to allow migrating Flutter apps.

grit check is slow, investigate why

In the following example https://app.grit.io/studio?key=jdcyl3I8UCOr1F_WxhQ-z we have some valid python code that runs perfectly fine. But because the editor has been set to remove trailing whitespace grit complains that the code is not properly indented, when in fact this is valid python and runs fine as is.

running edit_grammars twice should be the same as running it once, but currently running repeated causes files to be changed.

This valid TypeScript code is interpreted as an error node. Error recovery creates comments where none exist:

type RouteResponse = `${infer Key extends keyof RouteResponses}:${string}`
        ? RouteResponses[Key]
        : never;

Cypress.Commands.add('getRouteVariant', (routeVariant) =>
  cy
    .request<{
      preview: {
        body: RouteResponses[typeof routeVariant extends `${infer T}:${string}`
          ? T
          : never];
      };
    }>('GET', `http://localhost:3110/api/mock/variants/${routeVariant}`)
    .then((res) => res.body.preview.body),
);

Can you add support for Scala as a language for matching against.

Instructions

Review this guide. (fwiw There is a tree-sitter grammer available for scala at https://github.com/tree-sitter/tree-sitter-scala)

Acceptance criteria

• Working grammar + language support
• At least 10 test cases, including rewrites and metavariables.

Pattern:

`var($_)`

Code:

#some-id {
  some-property: 5px;
  color: var(--red)
}

We would like to support Ruby as a language for matching against.

Review this guide and join us on Discord to discuss.

Acceptance criteria

1. Working grammar + language support
2. At least 10 test cases, including rewrites and metavariables.

Assignment approach

Since this is a large project, here is how we will tackle it:

• If anyone who has already successfully contributed a grammar wants to take it on, we will assign them first dibs.
• Others are welcome to contribute. We will assign the bounty to the first major PR submitted that is close to complete.

Votes

• 3

I'm looking through the docs and wonder how to handle re-indentation. The use case would be to re-write a python unitttest.TestCase class to using pytest which is plain python functions.

We would like to have a watch mode for grit patterns test --watch.

Watch mode should be similar to other testing/CLI tools that offer a watch mode: when files change, re-run tests.

Acceptance criteria

• Watch mode should watch for file changes to any patterns in the .grit directory and automatically re-run those tests
• If a file changes, only the affected patterns should be re-run. Ex. if I edit this file then only that pattern test should re-run.
• You will also need to account for pattern dependencies. Ex. if https://github.com/getgrit/stdlib/blob/main/.grit/patterns/js/imports.grit is edited then any patterns which use the ensure_import_from pattern should be re-run. Utilities like this can be leveraged for walking the call graph
• At least 2 tests should be included, including ideally at least one integration test of the final binary like we do here.

The error message that comes out when a pattern name contains a - is confusing and not helpful

Reproducer:
grit.yaml

patterns:
  - name: use-winston
    level: error
    body: |
      `console.log($msg)` => `winston.log($msg)` where {
        $msg <: not within or { `it($_, $_)`, `test($_, $_)`, `describe($_, $_)` }
      }

Run:

$ grit apply use-winston
Your working tree currently has untracked changes and Grit will rewrite files in place. Do you want to proceed? yes
ERROR (code: 299) - Pattern syntax error at 1:1. If you hit this error while running grit apply on a pattern from the Grit standard library, try running grit init. If you are running a custom pattern, check out the docs at https://docs.grit.io/ for help with writing patterns.

Currently, we use prettier for auto-formatting JS unit test cases.

We should switch to Biome because:

• It is faster
• It doesn't require any external dependencies, like a node environment
• Biome is partnering with us on the plugin system

For workflows v2, we need a better display.

https://ratatui.rs/ looks like a nice option, used by turborepo.

Context

Since version 27, jest start to switch from using @types/jest where types are globally injected, like jest, expect and others modules, to manually import from @jest/globals.

So, we need to refactor all our tests files and import manually, like this:

 import { jest, expect, describe, it, /* and others */ } from '@jest/globals';`

Question

The challenge is: Find on all *.spec.ts(x) files, what modules were used (jest, expect, etc.) and then add a import {...} from '@jest/globals' for them.

It's possible with gritql to automate this refactoring?