rofl0r / agsutils Goto Github PK
View Code? Open in Web Editor NEWcontains utils for AGS: game extractor, repacker, disassembler and assembler
contains utils for AGS: game extractor, repacker, disassembler and assembler
agsutils by rofl0r ================== tools for (un)packing, disassembling, modifying and recompiling ags games. agsex: runs agstract and agscriptxtract, then creates a Makefile with rules to quickly and automatically repack a pack with changed asm files. this basically does everything for you automatically. can also use `make optimize` to run agsoptimize on all asm files. agstract: extracts the files a game "pack" (.exe) consists of. creates a file agspack.info which contains metadata about the files included in the pack. example: agstract 252test.exe FILES :::AGStract 0.9.1 by rofl0r::: 252test.exe: version 10, containing 6 files. ac2game.dta -> FILES/ac2game.dta acsprset.spr -> FILES/acsprset.spr agsfnt0.wfn -> FILES/agsfnt0.wfn agsfnt1.wfn -> FILES/agsfnt1.wfn agsfnt2.wfn -> FILES/agsfnt2.wfn room1.crm -> FILES/room1.crm agstract can also extract speech.vox and audio.vox files. make sure to use a different output directory, otherwise your agspack.info will be overwritten. in some games, speech.vox is insanely big because the speech files are saved in studio quality. i achieved good results by converting them to 16Khz via ffmpeg: for i in SPEECH/*.ogg ; do ffmpeg -i "$i" -c:a libvorbis -ar 16384 tmp.oga && mv tmp.oga "$i" done then repacking it with agspack, which results in about 6x space improvement. for audio.vox, you might want to use -b:a 80k instead of -ar 16384. this shrinks input files by about 400% with almost CD-like quality. agspack: takes the files created by agstract and makes a new bundle out of it. example: agspack FILES/ 252mytest.ags agspack -e FILES/ 252mytest.exe without arguments, agspack produces a game pack missing the original windows exe stub, but is compatible with the opensource AGS engine, as well as with scummvm ags port. that way, a game is about 500 KB smaller after extract and repack than it originally was. using the -e option, the original exestub is prepended to the game pack, and the offset to the start of the pack data encoded into the end signature, which reproduces the fully working windows executable with the new content. note that agspack always produces a version 20/30 packfile, compatible only with ags 3.0 or newer, so if your exe stub is from an earlier game, it will fail to recognize the attached game data. in such a case you can use the exestub from a newer version, e.g. https://github.com/ags-archives/engines/blob/master/ags302sp1/acwin.exe save it as agspack.exestub in your extract directory. agscriptxtract: detects and unpacks all binary scripts embedded in room and game files. the compiled scripts are stored with a .o extension, the disassembled files with .s. example: mkdir OBJ ; cd OBJ agscriptxtract ../FILES disassembling globalscript.o -> globalscript.s disassembling room1.o -> room1.s agsdisas: disassembles a single .o file to .s. useful to compare a re-assembled file with the original. example: agsdisas room1.o room1.s disassembling room1.o -> room1.s agssemble: creates a compiled object file from a .s assembly file. example: agssemble room1.s creates a new room1.o (you will notice filesize/date has changed) the compiled object file will miss unused strings and thus be smaller than the original. also imports and exports cover only really used ones. agsinject: once you have recompiled an .o file with agssemble, you can inject it into the original container file (either a .crm room file or a game file like "ac2game.dta") example: agsinject 0 OBJ/room1.o FILES/room1.crm injects room1.o at the first found script (0) in room1.crm. rooms have only 1 script so you must always use 0. for ac2game.dta kinda gamefiles, the index is i.e. 0 for the globalscript, 1 for the dialogscript (if one exists), otherwise 1 is the first gamescript, etc. after you injected your .o file, the next thing you want to do is agspack it all up again. then you can test your changes in the ags engine. agsprite: a tool to extract sprites from acsprset.spr files, and to create a new one from a directory full of extracted sprites. has several options to create smaller spritecache files than original, for example by converting all true-color images to high-color, which is almost impossible to differentiate visually. unfortunately ags saves a "uses alpha channel" flag for every sprite in a different file. if it was set, a picture converted from 32 to 16bit will become invisible, unless it is fixed with `agsalphahack` tool. after repacking a .spr file, a new sprindex.dat must be created and the old one replaced with it (if one existed). agsprite has an option for that too. optionally, the old sprindex.dat can simply be removed by commenting out the line with sprindex.dat in agspack.info and reducing the filecount by 1. at present, agsprite only creates and accepts TGA files. if you want to edit a sprite and your tool can't handle TGA format (unlikely), you can still convert the file into a format of your choice with e.g. imagemagick's `convert` tool, and then convert it back to TGA before creating a new sprite pack. run agsprite --help for usage information, and/or check the git log of agsprite.c to read a lot of detailed commit messages. agsalphahack: a tool to remove alphachannel flags from gamefiles. can delete either all or a single specified sprite number's alpha channel flag. this tool is a supplement to agsprite. agsalphainfo: a tool to print alphachannel information for each sprite. agssim: a simple simulator for ags assembly instructions. run agssim --help for more info. agsoptimize: a python script which is run on some .s asm files, detecting common inefficient patterns emitted by the AGS compiler, and replacing them with more efficient versions. using all preset optimization patterns, this improves speed of the CPU-heavy (because of a lots of scripts) game "Operation Forklift" by ~15-20%, which is almost identical to the number of asm statements it removes. another advantage is that the script files become smaller. ascc: the original AGS script compiler with some tweaks to support more C features. it's pretty handy to generate assembly code to inject into scripts. since ascc is based on the original AGS sources and is written in C++, it's available in a different repository: https://github.com/rofl0r/ascc . compilation: ------------ after acquiration of a C compiler toolchain (optimally GCC) and GNU make (e.g. on ubuntu: `apt install build-essential`), simply run `make`. if you need any special CFLAGS, LDFLAGS etc put them into config.mak or append them to the make command, i.e. `make CFLAGS="-O2 -g"` compilation on windos: ---------------------- mingw is not supported. it' s a half-baked PoS, which lacks a dozen or more of the POSIX APIs we require. worse than that, it even has *wrong* prototypes for some of them, for example it defines mkdir(char*), but the proper prototype is mkdir(char*, int), so we'd have to litter the code with #ifdefs and remove parts of the functionality to support it. that's not gonna happen. use cygwin. with cygwin, every works just fine without *any* changes. you can even run `make` and it produces files with .exe extensions even though we don't have a rule for that anywhere. you only need to ship cygwin-1.dll together with the binaries. NEW!!! agsutils can now also be compiled using [PellesC](http://www.smorgasbordet.com/pellesc/) (use [pellescc wrapper](https://github.com/rofl0r/pellescc) to build from Makefile). alternatively you can use WSL/WSL2 and just use native linux compiler and the binaries inside the WSL environment. third-party library requirements: --------------------------------- none. \o/ supported AGS versions: ----------------------- like the opensource AGS engine, version 2.3 and 2.4 aren't properly supported due to lack of available format information. support for those might be completed at some point in the future. all versions >= 2.5.0 are supported, and if one of the tools bails out on them it's considered a bug. License: -------- there is a tiny part of code left from the original AGS engine, about 150 LOC in CLib32.c. it is covered under the artistic license, see file header for details. all other code is (C) 2012-2019 rofl0r and licensed under the LGPL 2.1+ with the "NO C++" exception. the "NO C++" exception means that you are not allowed to convert the code into C++, but you can link it to C++ code.
Hello
I use AGSUtils to solve some problems in the translation of AGS games but I found some problems in the use of the tools agscriptxtract.exe and agsinject.exe
agscriptxtract:
Sometimes the tool works perfectly to extract the scripts like for the game Resonance and other times no extraction is done like for Old Skies Demo.
agsinject:
I have never been able to get this tool to work despite the appearance of a message like this:
injecting OBJ/room1.o into FILES/room1.crm as 0'th script ...OK
The verification of the target file allows me to see that despite this message it has not been modified.
The problems with these 2 tools force me to extract and reinsert the scripts manually with a hexadecimal editor using the SCOM and FE CA EF BE pointers to delimit the script.
For the reinsertion, I must also modify the size of the script located just before SCOM.
Script extraction is skipped when the file extraction produces file names with upper case characters:
agsutils/agstract game/Game.exe game_ex
cd game_ex
mkdir OBJ
(cd OBJ && ../../agsutils/agscriptxtract ..) | wc -l
# => 71
## Ensure all crm files have lower case names
find . -iname "*.crm" -exec /bin/sh -c 'mv -v {} $(echo {} | tr [A-Z] [a-z])' \; | wc -l
# => 80
(cd OBJ && ../../agsutils/agscriptxtract ..) | wc -l
# => 229
Running agsex on the game "Nightmare Frames" gives me the following error:
:::AGStract 0.9.9 by rofl0r:::
mmap: Invalid argument
seek error!
: Invalid argument
CRT: unhandled exception (main) -- terminating
(I tried this under Windows 11)
actually i was wrong about being able to just cat the stub and the data into a combined file; as AGS tries to locate the start of the data via an "end signature" containing the offset, rather than just searching for the start signature, so doing it manually as i suggested would involve using a hex editor to change the offset 16 bytes before the end as a 4 byte little endian value to contain the size of the .exestub file.
since this is impractical for casual users, i now added the -e option to agspack which does everything automatically.
Originally posted by @rofl0r in #16 (comment)
Thanks ! it works!Now it can pack correctly,but i encounter another problems,
When opening the exe file, it shows that
Error: Unable to load the room file 'room41.crm'
Format version not supported
Required format version: 23117, supported 17 -33
it got a collision of versions
subj
agsprite is giving "unsupported sprite file version 12" with 3.6 builds of AGS. Any possible fix?
bash: ./agstract: cannot execute binary file: Exec format error
Sorry for asking here, but how do I open the game in AGS when its main file is game28.dta?
after agsex:
test@test:~/ags$ make
agspack files game.ags
make: agspack: No such file or directory
make: *** [Makefile:18: all] Error 127
Hello.
I'm trying to assemble a room.S into room.O, with agssemble.out, but I get an error. The error message reads:
./Assembler.c:138: get_variable_offset: Assertion '0' failed.
Please, could you tell me what wrong might be happening or give me a hint about how to fix it?
I haven't modified the files. They are the same ones that I extracted with agscriptxtract.out, a few minutes ago.
Thanks for your attention.
Hello.
In the past I was considering a theoretical use of agsutils by a game developer to setup a patching system for their own game, and the remaining problem is a sprite file (acsprset.spr). In big games it may reach gigabytes in size. Currently agsutils allow to extract it from the game, but (afaik) not extract/repack its contents.
Hence, I think an utility to unpack/repack acsprset.spr could be a useful addition.
The related reading/writing code may be found here: https://github.com/adventuregamestudio/ags/blob/ags3/Common/ac/spritecache.cpp#L578
Note, the file comes in 2 variants: with compressed and uncompressed bitmaps. Only supported compression is RLE (there were user requests to add other options, which may occur in future).
subj
Hello,
I am trying to update a language patch for a video game.
And it seems that I have a problem with the spritecache version.
The game seems working with version 11 of spritecache but agsprite converts the spritecache to version 6.
I have the message : warning: converting spritecache version 11 to version 6.
The game I am trying to modify seems working with version 11. It crashes if we give it a spr file with spritecache version 6.
What is the difference between version11 and 6. And how to get a workaround for this issue ?
Thank you in advance.
I have a problem when I extract sprites. Prompt shows this message "Unsupported compression method 3"
The AGS is version 3.6.1.8 and the utils is the last commit.
Most people using agsutils (at least, imho) will just want to do simple edits, like sprite replacement, improvement of graphics and changing text (fixing typos, doing a translation). agsutils is already able to do all of the three, however editing text requires actual disassembly of scripts and handling complex files.
So here's my idea: Could you add support for extraction / injection of .TRS / .TRA files from / into game files? From my understanding, both are simple search-replace lists, with .TRS being in text format and .TRA being in a binary format. Extraction would be a very useful feature, as it would provide a start point for generating new translations (the extracted file would of course just contain the original strings). Injection would provide us with a simple way to edit all text in the AGS game file.
BTW: For editing existing translation files, there's already the excellent AGS Translation Editor.
In the start of chapter 2, it's possible to, after talking to the guy that wants his internet back and opening the envelope to get a 'boot disk' without the bottom 'Read Instructions on floppy disk' context button, which is a dead end because it has the phone number of the IT guy Clyde.
You can't even press it (555-8181) manually without the number in the notebook (with it, you can on).
This is apparently very random because it happened to me now, then i restarted the game and it didn't again. It's very strange and i have no clue how to make it happen.
Which versions are currently supported? I'm trying to extract an older AGS game (5 days a stranger), but agstract says
error: ags clib header signature not found
However the hex dump of the exe shows it has some CLIB markers.
As the title says. If I simply extract the game with agsex and pack it again with agspack, I can't run it anymore (neither via the EXE, nor via ScummVM). In ScummVM I get an error about sprite format, although I changed nothing in that regard.
I can provide more details if needed. The game is easily available via a simple Google search.
After packing ,i try to run the game and an Microsoft error message shows that it cannt run because of Version mismatch and contact the author
Readme mentions that
the new "exe" will miss the original windows exe stub, but is
compatible with the opensource AGS engine, as well as with scummvm ags port.
that way, a game is about 500 KB smaller after extract and repack as it originally was.
It is the reason why it cannt run correctly after packing?
How can i make it correctly?
Hello.
Clib32.c don't change archive file by LibUid.
Example game Zniw Adventure:
ctgame.exe - main Clib archive...
ctgame.001
ctgame.002
...
ctgame.007
File acsprset.spr in ctgame.001 archive... But agstract try extract from ctgame.exe.
Extracting sprites from Primordia.
Gimp and Windows Photo throw an error opening the files produced by AGSprite. This is caused by two errant bytes, one in field 4 and one in field 5 (as referenced on Wikipedia).
Byte at 0x8 is Colormap BPP, this should be at 0x7. Overall BPP is at 0x13 should be at 0x11.
Removing one zero byte from 0x3-0x5 and the zero byte at 0x8 (after removing the first byte) allows the TGA to load successfully. Looking at the byte order of the TGA standard (lo-hi) this appears to be the correct solution.
This may be a compiler issue as well: http://www.paulbourke.net/dataformats/tga/
Unfortunately, repairing this is way above my skill level, I am not a C coder and don't understand how the code generates this bitstream.
Using Windows binaries from agsutils-0.9.5-win32.zip.
Are there any versions of game data that aren't supported? Trying it out, it seems to crash when trying to extract the built-in script header:
(gdb) run
Starting program: /home/mwillcock/build/agsutils/agscriptxtract ..
disassembling [game28.dta] ./globalscript.o -> ./globalscript.s
regenerating script header ./builtinscriptheader.ash
Program received signal SIGSEGV, Segmentation fault.
0x000055555555702d in dump_header (a=0x7fffffffc2c0,
fn=0x7fffffffbfc0 "./builtinscriptheader.ash") at agscriptxtract.c:168
168 for(i=0; i<ADF_get_viewcount(a); ++i) if(ADF_get_viewname(a, i)[0])
subj
Hi,
tried to depack the following abandond ags game & repack it for usage with the opensource AGS 3.5 editor.
Freeware files are from here:
http://web.archive.org/web/20160328095603if_/http://monochrom.sil.at/suzoeg-s1-v1_0MrX-win-install.exe
http://web.archive.org/web/20110706094544if_/http://monochrom.sil.at/monochrom-SUZ2--version1dot2.exe
environment win10 + ubuntu + your fixed binaries from december 27th
after extracting the installer-exe, applied following for the second game, SUZ2 (first one behaves the same):
./agsex suz2prog.exe files obj
./agspack files suz2.agf
importing then into the AGS 3.5 editor fails with an import exception, instantly.
Not sure if a bug or misusage?
(also playing with an older editor, AGS 2.72, which imported at least the dta from the files directory, was ultimately not successful, script are lost that way)
thanks for feedback on this issue
Hello,
I have found that agsprite when extracting files from 'acsprset.spr' changes the palette of the sprites or even creates a new palette.
My purpose was to mix the original sprites of a game with sprites from a language pack.
It worked pretty well but in the result the new sprites have palette issues in the game. The game was working but
the colors were wrong.
To solved this issue I modified you program in order not to modify/create the palette of the sprites.
And now, the game colors are ok.
My suggestion is the following : is it possible to create an option when launching agsprite to preserve le original state of the palette of the sprites ? This is the case for bpp==1 and others.
The repacking does not change.
It is a suggestion for future use of packing/re-packing... personaly my problem is solved.
Thanks for maintaining these tools.
subj
Long story short, i tried to invoke agsex from a parent dir containing the scripts to a game dir and extraction dir children.
It broke a lot.
First the script agsex, where I changed BINDIR=$(dirname $(readlink -f "$0"))
-> BINDIR="$(dirname "$(readlink -f "$0")")"
and
$BINDIR/agstract "$GAME" "$FILES"
$BINDIR/agscriptxtract "$FILES" "$OBJS"
->
"$BINDIR"/agstract "$GAME" "$FILES"
"$BINDIR"/agscriptxtract "$FILES" "$OBJS"
Then it broken in agsextract and i gave up and tried to execute in the same directory as the executable (of the game). Behold, it worked. Maybe it's best to detect this case and warn it's not supported before exiting.
Hello, thanks for the tools! I've been trying to extract resources from a game compiled with AGS 3.5.0 without much success. The code seems to look for a small footer at the end of the file which contains the offset of the clib data followed by the string CLIB\x1\x2\x3\x4SIGE
; on the AGS 3.5.0 game, the end string is there, but the offset are all 0s (Last'n'Furious used as the AGS 3.4.1 reference):
AGS 3.4.1 game
C L I B S I G E
D1 0E 9A 00 43 4C 49 42 01 02 03 04 53 49 47 45
^^^^^^^^^^^
AGS 3.5.0 game
C L I B S I G E
00 00 00 00 43 4C 49 42 01 02 03 04 53 49 47 45
^^^^^^^^^^^
should be offset, it's 0s instead
If I hardcode the offset for the begginning of the clib block (which I found by searching for CLIB\x1A
inside of the executable, in my case it was the 2nd occurence), extraction appears to work (though the extracted acsprset.spr
does not extract with agsprite
, and some of the music files do not play; that's probably a different issue though); I haven't yet looked at how the engine copes with those changes.
maybe in far perspective)
gcc 8.1,9.2.0
Z:\ags>make
cc -Wall -Wextra -Wno-unknown-pragmas -Wno-sign-compare -Wno-switch -Wno-unused -Wno-pointer-sign -o ByteArray.o -c ByteArray.c
process_begin: CreateProcess(NULL, cc -Wall -Wextra -Wno-unknown-pragmas -Wno-sign-compare -Wno-switch -Wno-unused -Wno-pointer-sign -o ByteArray.o -c ByteArray.c, ...) failed.
make (e=2): ═х єфрхЄё эрщЄш єърчрээ√щ Їрщы.
make: *** [Makefile:57: ByteArray.o] Error 2
subj
Hi!
I have tried to extract the files with your extract utility, but I always get an error message. Can you please tell me how I can extract the files from the exe file with your tool?
bye
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.