Comments (16)
Ok maybe I could help a bit 🙂
Windows has essentially two functions to start processes: CreateProcess
and ShellExecuteEx
.
The first one is the low-level API which can "only" run .exe and .bat files, while the second one is the higher-level shell API which is able to map file extensions such as .ps1 to their executables such as powershell.exe. Their behvior can be very different depending on what you're trying to do, but Rust uses CreateProcess
.
The following has been added recently to the Rust process
module docs:
//! On Windows, `Command` uses the Windows API function [`CreateProcessW`] to
//! spawn new processes. An undocumented feature of this function is that,
//! when given a `.bat` file as the application to run, it will automatically
//! convert that into running `cmd.exe /c` with the bat file as the next argument.
//!
//! For historical reasons Rust currently preserves this behaviour when using
//! [`Command::new`], and escapes the arguments according to `cmd.exe` rules.
//! Due to the complexity of `cmd.exe` argument handling, it might not be
//! possible to safely escape some special chars, and using them will result
//! in an error being returned at process spawn. The set of unescapeable
//! special chars might change between releases.
//!
//! Also note that running `.bat` scripts in this way may be removed in the
//! future and so should not be relied upon.
I tested it quickly, and ripgrep is able to run a .bat file with --pre
but not a .ps1 script (as that would require ShellExecuteEx
, which in any case doesn't support output redirecting so it's a no-go).
Here's C:\Temp\test.bat
:
@echo off
echo Hello, world!
Here's C:\Temp\test.ps1
:
Write-Output "Hello, world!"
And there's also an empty file in C:\Temp\empty.txt
, just to show that the searched file content is irrelevent.
Here are the results:
rg --no-config --pre C:\Temp\test.bat Hello C:\Temp\empty.txt
1:Hello, world!
rg --no-config --pre C:\Temp\test.ps1 Hello C:\Temp\empty.txt
rg: C:\Temp\empty.txt: preprocessor command could not start: '"C:\\Temp\\test.ps1" "C:\\Temp\\empty.txt"': %1 is not a valid Win32 application. (os error 193)
So you should be able to get the output from a .bat file just fine. In case it's relevant, I compiled my ripgrep with Rust 1.78.0, which includes changes to Command
under Windows, so that may play a role in the results.
from ripgrep.
@ltrzesniewski Thank you!!!
from ripgrep.
This doesn't seem like a ripgrep problem, e.g., https://stackoverflow.com/questions/185042/how-do-i-resolve-1-is-not-a-valid-win32-application
I'm not a Windows users so I cannot be of much more help. But all ripgrep is doing is executing the argument given to --pre
as a process. That's it. I notice, for example, that you don't seem to have tested your preprocess
script directly. You should make sure it works on its own without ripgrep at all.
from ripgrep.
I notice, for example, that you don't seem to have tested your preprocess script directly. You should make sure it works on its own without ripgrep at all.
I tested, see below.
From #989, you said that
The preprocessor flag accepts a command program and executes this program for every input file that is searched. Instead of searching the file directly, ripgrep will instead search the stdout contents of the program.
and
rg -h
--pre=COMMAND Search output of COMMAND for each PATH.
I assume "the command program", i.e. preprocess
receives a filename as an argument and prints the data to standard output. Correct me, if I am wrong.
For example,
if I run preprocess.cmd
in command line, the result is
C:\work2\rg-test-dir>set PATH=C:/env/ripgrep/ripgrep-14.1.0-x86_64-pc-windows-msvc;%PATH%;
C:\work2\rg-test-dir>set PATH=C:/env/poppler/poppler-24.02.0/Library/bin;%PATH%;
C:\work2\rg-test-dir>preprocess.cmd 1995-watson.pdf
Taxonomies and Toolkits of
Regular Language Algorithms
Bruce William Watson
Eindhoven University of Technology
Department of Mathematics and Computing Science
....
if I run preprocess.ps1
in powershell, the result is
PS C:\work2\rg-test-dir> Set-ExecutionPolicy -Scope CurrentUser -ExecutionPolicy Unrestricted
PS C:\work2\rg-test-dir> $env:PATH = "C:/env/ripgrep/ripgrep-14.1.0-x86_64-pc-windows-msvc;$env:PATH"
PS C:\work2\rg-test-dir> $env:PATH = "C:/env/poppler/poppler-24.02.0/Library/bin;$env:PATH"
PS C:\work2\rg-test-dir> ./preprocess 1995-watson.pdf
1995-watson.pdf
Taxonomies and Toolkits of
Regular Language Algorithms
Bruce William Watson
Eindhoven University of Technology
Department of Mathematics and Computing Science
...
So it seems no problem with these preprocess
scripts.
But when it runs with ripgrep --pre option, something goes wrong.
Using Powershell as an example:
PS C:\work2\rg-test-dir> rg --pre ./preprocess.ps1 'The Commentz-Walter algorithm' 1995-watson.pdf
rg: ./preprocess.ps1: could not find executable in PATH
This error message is a little weird as I said before, because preprocess.ps1 is exactly in the current working directory... Anyway, if I force the working directory to be added to PATH, then
PS C:\work2\rg-test-dir> $env:PATH = "C:/work2/rg-test-dir;$env:PATH"
PS C:\work2\rg-test-dir> rg --pre ./preprocess.ps1 'The Commentz-Walter algorithm' 1995-watson.pdf
rg: 1995-watson.pdf: preprocessor command could not start: '"C:/work2/rg-test-dir\\./preprocess.ps1" "1995-watson.pdf"': %1 is not a valid Win32 application. (os error 193)
PS C:\work2\rg-test-dir>
from ripgrep.
I don't know. Sorry. I'd suggest asking on a general help forum for Windows users. Maybe there is something wrong with C:/work2/rg-test-dir\\./preprocess.ps1
? Why not try and debug this a bit more and use an absolute path? e.g., rg --pre C:/work2/rg-test-dir/preprocess.ps1
.
Otherwise, I'd suggest focusing on the error message you're getting. The "%1 is not a valid Win32 application" message suggests something is wrong with your setup somewhere.
Once again, I need to clarify here that I am not a Windows users. I have very little practical experience with it. It is too expensive for me to be doing end user support and diagnosing issues like this for operating systems I'm not familiar with.
from ripgrep.
For '"C:/work2/rg-test-dir\./preprocess.ps1", it seems to be related to path separator, but not"
Use absolute path:
PS C:\work2\rg-test-dir> rg --pre C:\works2\rg-test-dir\preprocess.ps1 'The Commentz-Walter algorithm' 1995-watson.pdf
rg: 1995-watson.pdf: preprocessor command could not start: '"C:\\works2\\rg-test-dir\\preprocess.ps1" "1995-watson.pdf"': The system cannot find the path specified. (os error 3)
PS C:\work2\rg-test-dir> rg --pre C:/works2/rg-test-dir/preprocess.ps1 'The Commentz-Walter algorithm' 1995-watson.pdf
rg: 1995-watson.pdf: preprocessor command could not start: '"C:/works2/rg-test-dir/preprocess.ps1" "1995-watson.pdf"': The system cannot find the path specified. (os error 3)
The error message is not "%1 is not a valid Win32 application" though.
from ripgrep.
Another possibility is that the preprocess.ps1
is not a win32 executable, so ripgrep can not run it directly.
from ripgrep.
Yes. It needs to be executable.
from ripgrep.
From https://doc.rust-lang.org/std/process/struct.Command.html, process::Command::new
seems to require different handling based on OS.
Comparing with
Line 303 in bb8601b
I'm not familiar with Rust though.
from ripgrep.
No, it doesn't. I don't know how else to explain it, but the argument to --pre
needs to itself be executable. The sh -c
and cmd /C
in your doc link is just an example of executing a shell script. The point there is that cmd
and sh
are themselves executable.
Please please please, you'll need to find a more general help forum specific to Windows users.
from ripgrep.
I just converted the preprocess.ps1
to preprocess.exe
via ps2exe, then works fine 😃.
Might be necessary to document it?
the argument to --pre needs to itself be executable
But in your preprocessor example, preprocess
is a bash script. I haven't tried on Linux, but on windows (include MSYS2 and Cygwin), script file (e.g. .cmd, .bat, .ps1) doesn't seem to work. It has to be an .exe file!
I don't know how to make Windows automatically treat a script as an executable, except convert it to .exe.
from ripgrep.
If a Windows expert wants to chime in that this is the only way to make --pre
work on Windows, then I'd be open to adding something like this to the docs. But I suspect there is something else you're missing. Either way, I'm glad you got it working.
from ripgrep.
The point there is that cmd and sh are themselves executable.
Yeah. I mean it would be nice if --pre
supports script. For example, execute cmd.exe
with a script. e.g. cmd.exe /c preprocess.cmd 1995-watson.pdf
.
from ripgrep.
It will never do that. Because then you have to worry about quoting and probably other things. The whole point of --pre
is that it presents an extremely simple interface: you provide an executable and ripgrep runs it. That's the standard cross platform interface for running programs.
from ripgrep.
I tested it again today and found that the current ripgrep can run .bat or .cmd file in --pre
actually.
preprocess.cmd
@echo off
pdftotext %1 -
but Command Prompt must use double quotes "The Commentz-Walter algorithm"
instead of single quotes 'The Commentz-Walter algorithm'
(PowerShell supports single quotes though).
C:\work2\rg-test-dir>rg --pre preprocess.cmd "The Commentz-Walter algorithm" 1995-watson.pdf
231:The Commentz-Walter algorithms : : : : : : : : : : : : : : : : : : : : : : : 82
4636:4.4 The Commentz-Walter algorithms
7509:in input string S , we obtain the Boyer-Moore algorithm. The Commentz-Walter algorithm
.ps1 doesn't work as @ltrzesniewski said.
from ripgrep.
Just a memo.
The --pre preprocess.sh
doesn't work in MSYS2/Cygwin (except using MSYS2/Cygwin build of ripgrep, but that is another story), because Rust Command is not aware of underlying MSYS2 bash. So if some one utilizes MSYS2 bash as a shell on Windows, use
$ rg --pre /c/work2/rg-test-dir/preprocess.bat 'The Commentz-Walter algorithm' 1995-watson.pdf
instead of
$ rg --pre /c/work2/rg-test-dir/preprocess.sh 'The Commentz-Walter algorithm' 1995-watson.pdf
from ripgrep.
Related Issues (20)
- [feature request] Context up to a matching pattern instead of by fixed number of lines HOT 1
- ignore: WalkBuilder::build_parallel() example does not compile HOT 1
- FR: sort results by count HOT 1
- ripgrep does not properly apply ignore rules from .gitignore in parent directory
- global ignore / ripgreprc shell parameter expansion HOT 2
- unicode uppercase HOT 3
- Negation pattern/whitelisting is ignored from `--ignore-file` HOT 8
- .gitignore rule is matched incorrectly while in a subdir
- Adjacent replaced multiline matches result in wrong line numbers HOT 4
- Feature Request (globset): Expose `MatchStrategy` HOT 6
- Support decompression flags / smart use of zstd --long HOT 1
- Add `Select-String`/`sls` to the benchsuite HOT 6
- JSON indexing off HOT 4
- Feature Request: Early Bail Out for Time-Consuming Regex Matches in pcre2 HOT 4
- [perf] rg does not use globs to prune recursion when it can HOT 4
- cursor disappearing HOT 1
- failing test: f2361_sort_nested_files
- The same `LineTerminator` is being used for searching and printing
- ARMv8 release binary for RPI 5 HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ripgrep.