boyter / cs Goto Github PK

Hey!
I wanted to try out this project as it seems very cool. Unfortunately, I cannot seem to manage to get it working. After successful compilation, it seems like it is unable to find files.

Screencast here:
https://asciinema.org/a/7COxeFJv0SMhhcQVh09WmmcuT

Let me know if you need some additional information.

Cheers!

👋 When I run cs in a large file tree such that there is latency in the search, the search will start while I'm still typing. The current search keeps going but then the searches seem to stack, so that while I'm waiting for "full search query" it will spend time on the intermediate things I've typed, like "full s" and "full sear". It would be nice if the current search and any stacked searches were cancelled if there is a new search in the input buffer.

I have realized that only first occurrence within file is displayed. Is this intentional behavior?

Currently, module name does not allow go install github.com/boyter/cs@latest:

go: downloading github.com/boyter/cs v1.0.0
go: github.com/boyter/cs@latest: github.com/boyter/[email protected]: parsing go.mod:
	module declares its path as: cs
	        but was required as: github.com/boyter/cs

If there are no obstacles (e.g. tricky build dependencies) to build this on a user machine, it would be convenient to be able to install the tool with go install one-liner.

Problem definition

Currently, cs looks for a match in all the files that are not ignored by .gitignore. This is great, but sometimes there are hidden directories that are nor gitignored, nor relevant for search.

For example, as a Python developer, I have lots of tryout projects that have a virtualenv in them. It is common practice in the Python world to create virtual environments in a hidden folder called .env. A virtual environment contains Python libraries, hence a lot of searchable code. If the project is a git repo it is likely that .env will be part of .gitignore, but this does not hold for tryouts or local test projects.

Idea

Add a switch to cs to ignore/not-ignore hidden files. This can be a CLI parameter for now, and maybe later added to the TUI interface.

Default behavior

The options are:

• ignore hidden files by default
• ignore hidden files as option

Tools like grep and ripgrep ignore hidden files by default, and that might be the experience that most developer are used to. Still, cs could exibit a different behavior if explicit.

Implementation challenges

I have the feeling that cs aims at being cross platform, hence a different detection mechanism has to be implemented for Windows and Linux. Some pointers can be found here. The combination of this and build tags should lead to a cross platform solution.

Notes

I am available to submit a PR in case this is interesting. I looked into the code and have a first rough implementation that served my Python needs.

Hi,

Thank you for developing this great tool.

Looking at the CLI demonstration video, it looks really useful.

However, README.md mentions binaries for windows, macos and linux in the release, but I can only see macos and linux binary releases. Seems like windows binaries are missing.

Any chance of adding windows binaries?

Thanks.

https://twitter.com/slimsag/status/1666190368189333505?s=20
https://twitter.com/slimsag/status/1666180661710053377?s=20

Need to have a think about this. I suspect some sort of query syntax might be the most ergonomic approach, although since this is likely to only be needed in TUI mode perhaps another input would work better.

Current plan is to use https://github.com/lithammer/fuzzysearch to do the fuzzy matching which works well based on previous work and gets close to a sublime text fuzzy matcher. As much as I would like to write my own version of this I don't think it worth it considering how decent this one is.

Open to suggestions on the implementation for this. Current options are.

1. Add a new command line argument which does the fuzzy filter for console mode, add a new input for TUI mode and HTTP mode to perform this
2. Embed into the search itself using some sort of special syntax, with the cavet that you cannot search for it anymore without exact match through " which is probably fine
3. Both.

I am leaning mostly towards 2. but I suspect there does need to be that command line argument for console mode as it might be annoying to do on the command line otherwise.

Would be nice if you say put a filename after running cs it goes into tui mode but limited to that file

EG

cs batf.txt

Checks if the file exists and if it does drops into TUI mode over just that file

❯ go install github.com/boyter/[email protected]
<empty>
❯ go version
go version go1.20.4 darwin/arm64
❯ which go
/opt/homebrew/bin/go

I've read through the README a couple of times and can run the Release build, but was curious why this wasn't working.

Thanks in advance for your help!

While this overlaps with ripgrep/grep/ag and such it might be useful in TUI mode rather than having to do rg TERM | fzf and it means it would work in the HTTP mode as well.

Adding the following test reproduces the failure.

    8 var leftCases = map[string][]int{
    9         " aaaa":        {4, 0},  // {start position, expected result}
   10         "a aaa":        {4, 1},
   11         "aaaa ":        {4, 4},
   12         " 12345678901": {11, 11},
+  13         // No spaces, even the comma+space is a single codepoint
+  14         "“啊，公爵，热那": {24, 24},  // 24bytes long: Returns (pos, false)
   15 }

As you know, the problem is here:

276         if unicode.IsSpace(rune(res.Content[i])) {

The problem is that a rune is a unicode code-point which can be expressed in a number of ways on disk (utf8, 16 etc). So simply casting to rune is not a stable operation.

We'd have to know what type of encoding the byte slice is and use one of the appropriate decoders (e.g utf8.DecodeRune)

It seems to me that the only real option here is to split on a number of code-points including ASCII space. As shown in this example, the code-point comma+space would have to be included. This strategy would mean iterating rune-by-rune rather than byte-by-byte.

What are your thoughts @boyter ?

Hi,

Thank you for developing this great tool.

Reading through the documentation, I can see that the recommended way to use the tool is to run it in the working directory. But that implies that I need to add cs.exe to each of my projects just to be able to search the code. That feels unacceptable to me. Hence, I was wondering, how can I use this as a tool, meaning, how can I keep cs.exe in some directory A, but still order it to scan the source code in some other directory B.

This closed issue mentions this functionality but it doesn't work for me. When I issue a full path to source code project as a first parameter, cs.exe just exists like nothing happened.

My use case is to be able to create several shortcuts (or bat files, or shell files or whatever) where path to cs.exe would always be the same one, but the path to source code project would be different for each project, sometimes even on different drives.

Thanks.

It would be nice if cs could respect terminal colors in interactive TUI mode.

 
 {"level":"info","unique_code":"e237c070","time":"2020-04-27T04:15:56+02:00","message":"level 1 cache hit"}
 ERROR   2020/04/27 04:15:56 server.go:3059: http: panic serving 127.0.0.1:54022: runtime error: slice bounds out of range [:39] with c
 apacity 32
 goroutine 1746 [running]:
 net/http.(*conn).serve.func1(0xc0006d2280)
         /usr/local/go/src/net/http/server.go:1772 +0x139
 panic(0xaeb3a0, 0xce394fce20)
         /usr/local/go/src/runtime/panic.go:973 +0x396
 github.com/boyter/go-searchcode/vendor/github.com/boyter/cs/string.IndexAllIgnoreCaseUnicode(0xce0fe70e11, 0x6e, 0xc000124090, 0x27, 0
 xffffffffffffffff, 0x0, 0x0, 0x0)
         /home/bboyter/Go/src/github.com/boyter/go-searchcode/vendor/github.com/boyter/cs/string/index.go:188 +0xc34
 github.com/boyter/go-searchcode/service.(*Service).findMatchingLines(0xc000116018, 0x2d586d8, 0xc000124090, 0x27, 0xce0fe70000, 0x322c
 , 0xcd98f1c400, 0x9, 0x10)
         /home/bboyter/Go/src/github.com/boyter/go-searchcode/service/search.go:485 +0x16cb
 github.com/boyter/go-searchcode/service.(*Service).Search(0xc000116018, 0xc000124090, 0x27, 0xc000693700, 0xb, 0xd, 0x14, 0x117d738, 0
 x0, 0x0, ...)
         /home/bboyter/Go/src/github.com/boyter/go-searchcode/service/search.go:244 +0xd76
 github.com/boyter/go-searchcode/handlers.(*Application).search(0xc000116000, 0xcb8240, 0xcdfd6ba0e0, 0xc00062a200, 0xc000124090, 0x27,
  0x171b96b689c)
         /home/bboyter/Go/src/github.com/boyter/go-searchcode/handlers/handler.go:172 +0x483
 github.com/boyter/go-searchcode/handlers.(*Application).Home(0xc000116000, 0xcb8240, 0xcdfd6ba0e0, 0xc00062a200)
         /home/bboyter/Go/src/github.com/boyter/go-searchcode/handlers/handler.go:104 +0x377
 net/http.HandlerFunc.ServeHTTP(0xc000684fc0, 0xcb8240, 0xcdfd6ba0e0, 0xc00062a200)
         /usr/local/go/src/net/http/server.go:2012 +0x44
 github.com/boyter/go-searchcode/vendor/github.com/gorilla/mux.(*Router).ServeHTTP(0xc0001840c0, 0xcb8240, 0xcdfd6ba0e0, 0xc000255400)
         /home/bboyter/Go/src/github.com/boyter/go-searchcode/vendor/github.com/gorilla/mux/mux.go:210 +0xe2
 net/http.serverHandler.ServeHTTP(0xcdfad0c000, 0xcb8240, 0xcdfd6ba0e0, 0xc000255400)
         /usr/local/go/src/net/http/server.go:2807 +0xa3
 net/http.(*conn).serve(0xc0006d2280, 0xcb9480, 0xc00021e440)
         /usr/local/go/src/net/http/server.go:1895 +0x86c
 created by net/http.(*Server).Serve
         /usr/local/go/src/net/http/server.go:2933 +0x35c
 
 
 ERROR   2020/04/28 01:03:25 server.go:3059: http: panic serving 127.0.0.1:33876: runtime error: slice bounds out of range [:53] with c
 apacity 48
 goroutine 2871 [running]:
 net/http.(*conn).serve.func1(0xc02cdd86e0)
         /usr/local/go/src/net/http/server.go:1772 +0x139
 panic(0xaec140, 0xc032069f80)
         /usr/local/go/src/runtime/panic.go:973 +0x396
 github.com/boyter/go-searchcode/vendor/github.com/boyter/cs/string.IndexAllIgnoreCaseUnicode(0xc032de6902, 0x62, 0xc028ba4d80, 0x35, 0
 xffffffffffffffff, 0x0, 0x0, 0x0)
         /home/bboyter/Go/src/github.com/boyter/go-searchcode/vendor/github.com/boyter/cs/string/index.go:188 +0xc34
 github.com/boyter/go-searchcode/service.(*Service).findMatchingLines(0xc0002cc018, 0x7a9a66b, 0xc028ba4d80, 0x35, 0xc030680000, 0x3151
 4, 0xc032701000, 0xf, 0x10)
         /home/bboyter/Go/src/github.com/boyter/go-searchcode/service/search.go:485 +0x16cb
 github.com/boyter/go-searchcode/service.(*Service).Search(0xc0002cc018, 0xc028ba4d80, 0x35, 0xc028de6420, 0x15, 0x0, 0x14, 0x117e738,
 0x0, 0x0, ...)
         /home/bboyter/Go/src/github.com/boyter/go-searchcode/service/search.go:244 +0xd76
 github.com/boyter/go-searchcode/handlers.(*Application).search(0xc0002cc000, 0xcb95c0, 0xc028df2000, 0xc028df4000, 0xc028ba4d80, 0x35,
  0x171bde18370)
         /home/bboyter/Go/src/github.com/boyter/go-searchcode/handlers/handler.go:172 +0x483
 github.com/boyter/go-searchcode/handlers.(*Application).Home(0xc0002cc000, 0xcb95c0, 0xc028df2000, 0xc028df4000)
         /home/bboyter/Go/src/github.com/boyter/go-searchcode/handlers/handler.go:104 +0x377
 net/http.HandlerFunc.ServeHTTP(...)
         /usr/local/go/src/net/http/server.go:2012
 github.com/boyter/go-searchcode/handlers.IpRestrictorHandler.func1(0xcb95c0, 0xc028df2000, 0xc028df4000)
         /home/bboyter/Go/src/github.com/boyter/go-searchcode/handlers/wrapper.go:59 +0x177
 net/http.HandlerFunc.ServeHTTP(0xc00069a9a0, 0xcb95c0, 0xc028df2000, 0xc028df4000)
         /usr/local/go/src/net/http/server.go:2012 +0x44
 github.com/boyter/go-searchcode/vendor/github.com/gorilla/mux.(*Router).ServeHTTP(0xc0001640c0, 0xcb95c0, 0xc028df2000, 0xc028191e00)
         /home/bboyter/Go/src/github.com/boyter/go-searchcode/vendor/github.com/gorilla/mux/mux.go:210 +0xe2
 net/http.serverHandler.ServeHTTP(0xc0135de000, 0xcb95c0, 0xc028df2000, 0xc028191e00)
         /usr/local/go/src/net/http/server.go:2807 +0xa3
 net/http.(*conn).serve(0xc02cdd86e0, 0xcba800, 0xc02818db00)
         /usr/local/go/src/net/http/server.go:1895 +0x86c
 created by net/http.(*Server).Serve
         /usr/local/go/src/net/http/server.go:2933 +0x35c

No idea what is causing this, but probably need to fuzz the input to see if this can be itentiified.

hi, if there is a python glue version could be better

It would be handy if cs had an option to print the line number, so you could write a script that opens your editor to the right place without having to scroll.

A good format might be file:line e.g. path/to/file.txt:42

Implement -i/--include-ext strings option. For instance:

cs --include-ext yaml,yml 

... would search results only from yaml files.

Currently this is not implemented and returns results from all file extensions.