xparq / space_test Goto Github PK
View Code? Open in Web Editor NEWAhh, OK, so it's "space" for a different reason... ;)
Ahh, OK, so it's "space" for a different reason... ;)
While EXEC is a perfect counterpart of SH, and it also conveys exactly what it is, but only for devs...
I mean it's all for devs anyway, but for the overall style... It just reads shit. Like the examples in the README:
EXEC something
EXPECT some result
or
EXEC some command
EXEC other command
EXEC command --with-params
EXPECT "some result
....
Perhaps it just looks way too similar to EXPECT. Or is it just me?!
Maybe DO? Or just keep it RUN? Both are subpar for accuracy of meaning, but just taste better. Their genericity is also a plus, from the overall readability aspect (for high-level test authoring/maintenance), not just a minus (from the low-level technical test coding aspect).
I think I'll just keep it RUN for now, and revisit this if really becomes untenable.
Renaming run_cases is uncontroversial though. The mild crosstalk between RUN and run is kinda still annoying, but nothing that can't be lived with.
(Originally opened for Args (-> xparq/Args#30), but the test scripts now live here, so taking over...)
Now it thinks everything's a test case.
Oh, and of course run_cases run_cases
is a nice infinite loop then! :)
(Originally: xparq/Args#32)
# If listing more than one dependency, separate them with ","
<-- or should it be ";"?!
Either way, eating up whitespace around the separator would also be a problem for the makefiles...
E.g.:
$ run_cases "subset*"
ERROR: Test case "subset*" not found!
$ run_cases "subset*/*"
realpath: C:/sz/prj/Space_Test/test/subset subdir/subtree case 1.case subset subdir/subtree case with build.case: No such file or directory
ERROR: Test case "subset*/*" not found!
Related ("dependent") issues (decisions that have simplified things a lot):
TEST_DIR/run (and run.cmd) has been used as the sole reliable anchor for locating the TEST_DIR.. Where that script is, there's the test stuff. And from there it just also knows that _engine is also there, right underneath.
The current logic is too brittle: it relies solely on $0
being in the _engine
dir, which is not the case when run
is being sourced from a script elsewhere, or if it was run via a symlink (at least on Windows; can't recall if sh resolves that symlink, hopefully not)!
Now at least TEST_DIR
can be set before calling run
(which would then accept it, no questions asked).
Support symlinks to _engine/run[.cmd]
as anchors of the test dir!
-> E.g.: https://github.com/cavo789/dos_batch_tips#getsymlinktargetpath
Well, following through from #11, let the config tell it.
Because the shebangs still had to remain #!/bin/sh
-- but then it's a manifest inconsistency: sh doesn't support this syntax!
The implementations I encountered on Windows all still did tho, because many are just bashes in disguise.
So, all was well, until actually running in a real Linux shell, which took the shebangs seriously, and so the scripts broke...
That should also spare the extra subshell of SH echo!
I'm no shell programmer, you see. I've basically googled and manned and SXed all the .sh syntax & semantics together, so chances are I'm still missing a few glaring issues (even with such a simple language as ash
).
So, it's only needed for bridging over to subprocesses, but not when sourcing scripts:
I may have extra or missing export
s basically anywhere...
(Originally: xparq/Args#33)
The .gitattributes
settings supposedly already ensure that it's that way in the repo.
However, I still had core.autocrlf
enabled locally, and either because of that or it was just never changed, (some of?) the scripts still had CRLF (in the work tree at least).
I recently dos2unix
ed them all, and also did a git add --renormalize
(which didn't change anything then, but suppressed Git's really inexplicable "...will change to CRLF" warning); let's see what happens next time I check something out and feed it e.g. to WSL's bash...
Oh yeah, obviously? And then the CASE
files, too, obviously, right? And then what about single-file test case scripts with natural text as their names? Should they also be "Startup with defaults.CASE"
? I mean not that "Startup with defaults.case"
would look any better, but capitals in the extension is a HIGHLY unusual habit, and may be offputting to some.
OK then... Make it configurable! :) See also #12 then!
E.g. these are some of the vars exported for a build, showing what I mean:
export TEST_NAME
export TEST_DIR
export CASE
export TEST_CASE_DIR
...
And... While we are at it:
...so e.g. a missing test exe problem would be more obvious to recognize.
Not to mention the current feature of also accepting Git's sh!... :-o
The issue is that busybox
versions obviously differ vastly, so relying on "found" versions in a host env. is a hit-and-miss.
OTOH, bundling a version for every possible supported env. seems to be both futile and restrictive. (Or is it? First, it's only really necessary on Windows, and second, there seems to be a (32-bit?) "canonical" version at Ron's page anyway!...)
NOTE: now (-> #22) it's actually also being downloaded on-the-fly, too, so that's also an uncontrolled version dependency issue!
So, the tightening, and/or various levels of shell (mostly BB) version control part is still relevant!
(Originally opened as xparq/Args#35)
This is to help debugging individual cases without entering the test case dir manually just for executing a local exe there, or even just changing the last command too much...
In this mode the EXPECT phase will likely be off, so a warning should be printed.
Note: the params would be appended to every RUN/SH statement of the case.
That should also make it easier to "integrate" (copy...) into other projects, esp. upgrading existing ones (better separation from test data (artifacts) and the "engine")!
See e.g. the README case with 3 echo
lines, each with NL: expected even if there's no NL at the end of the EXPECT text!
Ehheh, OK, fine, fine... I meant "spaces in explicitly named test case names", it seems... :)
Anyway, it was an easy fix: just replacing $*
with $@
in the main runner iterator loop, as per https://linux.die.net/man/1/ash:
@ Expands to the positional parameters, starting from one. When the expansion occurs within
double-quotes, each positional parameter expands as a separate argument. If there are
no positional parameters, the expansion of @ generates zero arguments, even when @ is
double-quoted.
What this basically means, for example, is if $1 is ''abc'' and $2 is ''def ghi'', then "$@" expands
to the two arguments:
"abc" "def ghi"
Woahh... OK, so it had nothing to do with EXPECT being empty, actually! :)
It was a problem of trying to RUN echo
, which must be run via SHELL
.
-> #25
You see, that's exactly what I was talking about in #9...
Well, there are two Git sh
s... Oh, wailt, three even, actually!... :-o And the "wrong" one is on the PATH here.
Test:
set "GITROOT=C:\Program Files\Git"
where sh.exe
rem -> C:\Program Files\Git\usr\bin\sh.exe
where find
rem -> C:\Windows\System32\find.exe
rem -> C:\Program Files\Git\usr\bin\find.exe
@echo.
@echo This will use Windows's find.exe:
sh.exe -c "which find; find . -name '*.*'"
@echo.
@echo This will use Git's find:
"%GITROOT%\bin\sh.exe" -c "which find; find . -name '*.*'"
@echo.
@echo WTF is this trying to do?
"%GITROOT%\git-bash.exe" -c "which find; find . -name '*.*'"
Also, critically: don't forget -c
(the error message is horribly misleading)!
>"...\sh.exe" find -- version
find: find: cannot execute binary file
So:
sh -c "find --version"
run_cases "TC*"
- ERROR: Test case "TC dir with no suffix TC dir with no suffix and no script TC dir with suffix, but no script.case" not found!
Unquoted run_cases TC*
is fine.
But still allow overriding it with any local Makefile* just like in the other test dirs.
(Originally: xparq/Args#20)
Regex support in the EXPECT
clauses could also do, but that won't address e.g. standalone EXPECT
files.
E.g., it was also a "wish item" in the Args README case, which is, indeed, has a split run-script vs. EXPECT
case setup:
# CLEAN s/[^ ]+.+\.exe/\.\.\.\.exe/
-> #20
$ run_cases *bui*
Riding "...\prj\Space_Test\test\_engine/busybox" sh...
"...\prj\Space_Test\test\_engine\busybox" sh -c ". build.sh \"test-demo\""
realpath: .../prj/Space_Test/test/*bui*: Invalid argument
ERROR: Test case "*bui*" not found!
-------- THERE HAVE BEEN FAILED CASES!
$ run_cases *exe*
Riding "...\prj\Space_Test\test\_engine/busybox" sh...
gnumake: 'test-demo.exe' is up to date.
ERROR: Test case "general.exe" not found!
ERROR: Test case "test-demo.exe" not found!
-------- THERE HAVE BEEN FAILED CASES!
I mean better... It's not obvious to first-time visitors that there's anything interesting here at all!
This should fix the missing BB error in the Args GHA test run...
Esp. this confusing one:
SH "echo Dir list:; ls -r"
SH echo command 3
EXPECT "Dir list:
Hi from the test case dir!
CASE
command 3
"
(And some other things below that.)
Yeah, well, sounds scarier than it really is (it's a relic I know of, and it's harmless), but still kinda facepalm...
(The thing, BTW, that makes the difference (but shouldn't), is having or not having the $TEST_CASE_FILE_EXT
in the TC dir's name.)
-> #42
WSL:
.../prj/Space_Test/test$ ./run_cases
./run_cases: 213: [: gcc: unexpected operator
./run_cases: 280: [: gnumake: unexpected operator
./run_cases: 326: /mnt/c/sz/prj/Space_Test/test/_engine/init_once.sh: gnumake: not found
gnumake
dependency in functions.sh
is gross... It's not even called "gnumake" by default. ;)
make
would help even less here: a) it's only available there, on Windows, b) it's very limited, and c) there's likely an NMAKE also available there anyway...Currently:
>bash -c ./run_cases general
./run_cases: 12: [: unexpected operator
./run_cases: 48: [: 0: unexpected operator
-------- THERE HAVE BEEN FAILED CASES!
Is even [ "x" == "y" ]
a bashism, with that ==
?!
-> shellcheck -s bash ...
Notes copied from a dup. (#48):
WSL:
.../prj/Space_Test/test$ ./run_cases
./run_cases: 213: [: gcc: unexpected operator
./run_cases: 280: [: gnumake: unexpected operator
./run_cases: 326: /mnt/c/sz/prj/Space_Test/test/_engine/init_once.sh: gnumake: not found
gnumake
dependency in functions.sh
is gross... It's not even called "gnumake" by default. ;)
make
would help even less here: a) it's only available there, on Windows, b) it's very limited, and c) there's likely an NMAKE also available there anyway...Well, the current 1-depth initial MVP is gonna be too tight a fit soon, obviously...
Too much incompatibilities...
E.g. the makefiles can't just glue the build script path uniformly to $_SH_
, because under Git sh it's /c/shit/..., while BB.exe has no problem with the normal C:/... paths... :-/
OTOH, if it's too explicit, or supports BB alone, then it may not just run fine in a native sh env as it should!...
But we could still stick to BB on Windows at least... (E.g. in cases where there's no $SHELL
yet? Nope: $SHELL
doesn't help with the Git sh vs BB_win32 path discrepancy!)
As an option, as that's entirely (intrusively) C++-specific!
-> e.g. https://github.com/xparq/regenv/blob/master/src/FindListPart.cpp
Updated it there to 0.08.
Fails at the GHA right now, for missing BB... -> #22
The test runner has been passing $MAKE
for a while now, so the actual build command could be selected accordingly.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.