rofl0r / agsutils Goto Github PK

contains utils for AGS: game extractor, repacker, disassembler and assembler

C 88.87% Makefile 0.77% Shell 0.51% Python 4.74% Assembly 0.19% Lex 0.82% Yacc 4.09%

ags unpacker assembler disassembler

agsutils's Introduction

agsutils by rofl0r
==================

tools for (un)packing, disassembling, modifying and recompiling ags games.

agsex:
runs agstract and agscriptxtract, then creates a Makefile with rules
to quickly and automatically repack a pack with changed asm files.
this basically does everything for you automatically.
can also use `make optimize` to run agsoptimize on all asm files.

agstract:
extracts the files a game "pack" (.exe) consists of.
creates a file agspack.info which contains metadata about the
files included in the pack.
example:

agstract 252test.exe FILES
:::AGStract 0.9.1 by rofl0r:::
252test.exe: version 10, containing 6 files.
ac2game.dta -> FILES/ac2game.dta
acsprset.spr -> FILES/acsprset.spr
agsfnt0.wfn -> FILES/agsfnt0.wfn
agsfnt1.wfn -> FILES/agsfnt1.wfn
agsfnt2.wfn -> FILES/agsfnt2.wfn
room1.crm -> FILES/room1.crm

agstract can also extract speech.vox and audio.vox files.
make sure to use a different output directory, otherwise your
agspack.info will be overwritten.

in some games, speech.vox is insanely big because the speech files are
saved in studio quality.

i achieved good results by converting them to 16Khz via ffmpeg:

for i in SPEECH/*.ogg ; do
ffmpeg -i "$i" -c:a libvorbis -ar 16384 tmp.oga && mv tmp.oga "$i"
done

then repacking it with agspack, which results in about 6x space improvement.

for audio.vox, you might want to use -b:a 80k instead of -ar 16384.
this shrinks input files by about 400% with almost CD-like quality.

agspack:
takes the files created by agstract and makes a new bundle out of it.
example:

agspack FILES/ 252mytest.ags
agspack -e FILES/ 252mytest.exe

without arguments, agspack produces a game pack missing the original
windows exe stub, but is compatible with the opensource AGS engine,
as well as with scummvm ags port.
that way, a game is about 500 KB smaller after extract and repack than it
originally was.

using the -e option, the original exestub is prepended to the game pack,
and the offset to the start of the pack data encoded into the end signature,
which reproduces the fully working windows executable with the new content.

note that agspack always produces a version 20/30 packfile, compatible only
with ags 3.0 or newer, so if your exe stub is from an earlier game, it
will fail to recognize the attached game data. in such a case you can
use the exestub from a newer version, e.g.
https://github.com/ags-archives/engines/blob/master/ags302sp1/acwin.exe
save it as agspack.exestub in your extract directory.

agscriptxtract:
detects and unpacks all binary scripts embedded in room and game files.
the compiled scripts are stored with a .o extension, the disassembled files with .s.
example:

mkdir OBJ ; cd OBJ
agscriptxtract ../FILES
disassembling globalscript.o -> globalscript.s
disassembling room1.o -> room1.s

agsdisas:
disassembles a single .o file to .s. useful to compare a re-assembled file with the original.
example:

agsdisas room1.o room1.s
disassembling room1.o -> room1.s

agssemble:
creates a compiled object file from a .s assembly file.
example:

agssemble room1.s
creates a new room1.o (you will notice filesize/date has changed)

the compiled object file will miss unused strings and thus be smaller than the original.
also imports and exports cover only really used ones.

agsinject:
once you have recompiled an .o file with agssemble, you can inject it into the original
container file (either a .crm room file or a game file like "ac2game.dta")
example:

agsinject 0 OBJ/room1.o FILES/room1.crm

injects room1.o at the first found script (0) in room1.crm.
rooms have only 1 script so you must always use 0.
for ac2game.dta kinda gamefiles, the index is i.e. 0 for the globalscript,
1 for the dialogscript (if one exists), otherwise 1 is the first gamescript, etc.

after you injected your .o file, the next thing you want to do is agspack it all up again.
then you can test your changes in the ags engine.

agsprite:
a tool to extract sprites from acsprset.spr files, and to create a new one
from a directory full of extracted sprites. has several options to create
smaller spritecache files than original, for example by converting all
true-color images to high-color, which is almost impossible to differentiate
visually.
unfortunately ags saves a "uses alpha channel" flag for every sprite in a
different file.
if it was set, a picture converted from 32 to 16bit will become
invisible, unless it is fixed with `agsalphahack` tool.
after repacking a .spr file, a new sprindex.dat must be created and the old
one replaced with it (if one existed). agsprite has an option for that too.
optionally, the old sprindex.dat can simply be removed by commenting out the
line with sprindex.dat in agspack.info and reducing the filecount by 1.
at present, agsprite only creates and accepts TGA files.
if you want to edit a sprite and your tool can't handle TGA format (unlikely),
you can still convert the file into a format of your choice with e.g.
imagemagick's `convert` tool, and then convert it back to TGA before creating
a new sprite pack.
run agsprite --help for usage information, and/or check the git log of
agsprite.c to read a lot of detailed commit messages.

agsalphahack:
a tool to remove alphachannel flags from gamefiles. can delete either all
or a single specified sprite number's alpha channel flag. this tool is a
supplement to agsprite.

agsalphainfo:
a tool to print alphachannel information for each sprite.

agssim:
a simple simulator for ags assembly instructions.
run agssim --help for more info.

agsoptimize:
a python script which is run on some .s asm files, detecting common inefficient
patterns emitted by the AGS compiler, and replacing them with more efficient
versions. using all preset optimization patterns, this improves speed of the
CPU-heavy (because of a lots of scripts) game "Operation Forklift" by ~15-20%,
which is almost identical to the number of asm statements it removes.
another advantage is that the script files become smaller.

ascc:
the original AGS script compiler with some tweaks to support more C features.
it's pretty handy to generate assembly code to inject into scripts.
since ascc is based on the original AGS sources and is written in C++, it's
available in a different repository: https://github.com/rofl0r/ascc .

compilation:
------------

after acquiration of a C compiler toolchain (optimally GCC) and GNU make
(e.g. on ubuntu: `apt install build-essential`),

simply run `make`.

if you need any special CFLAGS, LDFLAGS etc put them into config.mak
or append them to the make command, i.e. `make CFLAGS="-O2 -g"`

compilation on windos:
----------------------

mingw is not supported. it' s a half-baked PoS, which lacks a dozen or more of
the POSIX APIs we require. worse than that, it even has *wrong* prototypes
for some of them, for example it defines mkdir(char*), but the proper prototype
is mkdir(char*, int), so we'd have to litter the code with #ifdefs and remove
parts of the functionality to support it. that's not gonna happen.

use cygwin. with cygwin, every works just fine without *any* changes. you can
even run `make` and it produces files with .exe extensions even though we don't
have a rule for that anywhere. you only need to ship cygwin-1.dll together with
the binaries.

NEW!!! agsutils can now also be compiled using [PellesC](http://www.smorgasbordet.com/pellesc/)
(use [pellescc wrapper](https://github.com/rofl0r/pellescc) to build from Makefile).

alternatively you can use WSL/WSL2 and just use native linux compiler and the
binaries inside the WSL environment.

third-party library requirements:
---------------------------------
none. \o/

supported AGS versions:
-----------------------
like the opensource AGS engine, version 2.3 and 2.4 aren't properly supported
due to lack of available format information. support for those might be
completed at some point in the future.
all versions >= 2.5.0 are supported, and if one of the tools bails out on them
it's considered a bug.

License:
--------
there is a tiny part of code left from the original AGS engine, about 150 LOC in CLib32.c.
it is covered under the artistic license, see file header for details.
all other code is (C) 2012-2019 rofl0r and licensed under the LGPL 2.1+ with the "NO C++"
exception.
the "NO C++" exception means that you are not allowed to convert the code into C++, but
you can link it to C++ code.

agsutils's People

Contributors

Stargazers

Watchers

Forkers

raulpuro iliocatallo crose1988 thecocce ectoo anemochore 5l1v3r1 tevo45 ner00 i30817 angeldevindie happy-ferret-entertainment xloem skynetde

agsutils's Issues

Scripts extraction and injection

Hello

I use AGSUtils to solve some problems in the translation of AGS games but I found some problems in the use of the tools agscriptxtract.exe and agsinject.exe

agscriptxtract:
Sometimes the tool works perfectly to extract the scripts like for the game Resonance and other times no extraction is done like for Old Skies Demo.

agsinject:
I have never been able to get this tool to work despite the appearance of a message like this:
injecting OBJ/room1.o into FILES/room1.crm as 0'th script ...OK
The verification of the target file allows me to see that despite this message it has not been modified.

The problems with these 2 tools force me to extract and reinsert the scripts manually with a hexadecimal editor using the SCOM and FE CA EF BE pointers to delimit the script.
For the reinsertion, I must also modify the size of the script located just before SCOM.

agscriptxtract: extraction is case sensitive

Script extraction is skipped when the file extraction produces file names with upper case characters:

agsutils/agstract game/Game.exe game_ex
cd game_ex
mkdir OBJ

(cd OBJ && ../../agsutils/agscriptxtract ..) | wc -l
# => 71

## Ensure all crm files have lower case names
find . -iname "*.crm" -exec /bin/sh -c 'mv -v {} $(echo {} | tr [A-Z] [a-z])' \; | wc -l
# => 80

(cd OBJ && ../../agsutils/agscriptxtract ..) | wc -l
# => 229

what compiler should i use to build this?

Game "Nightmare Frames", error "mmap: Invalid argument seek error!"

Running agsex on the game "Nightmare Frames" gives me the following error:

:::AGStract 0.9.9 by rofl0r:::
mmap: Invalid argument
seek error!
: Invalid argument
CRT: unhandled exception (main) -- terminating

(I tried this under Windows 11)

version collision after packing in windows

actually i was wrong about being able to just cat the stub and the data into a combined file; as AGS tries to locate the start of the data via an "end signature" containing the offset, rather than just searching for the start signature, so doing it manually as i suggested would involve using a hex editor to change the offset 16 bytes before the end as a 4 byte little endian value to contain the size of the .exestub file.
since this is impractical for casual users, i now added the -e option to agspack which does everything automatically.

Originally posted by @rofl0r in #16 (comment)

Thanks ! it works！Now it can pack correctly，but i encounter another problems，
When opening the exe file, it shows that
Error: Unable to load the room file 'room41.crm'
Format version not supported
Required format version: 23117, supported 17 -33
it got a collision of versions

Feat.req. Adapt agsoptimize to run on Windows

subj

agsprite Sprite file version 12 support

agsprite is giving "unsupported sprite file version 12" with 3.6 builds of AGS. Any possible fix?

running on Win10x64 WSL (Ubuntu 18.08 image), what I do wrong?

bash: ./agstract: cannot execute binary file: Exec format error

How to open game28.dta?

Sorry for asking here, but how do I open the game in AGS when its main file is game28.dta?

Error while running makefile

after agsex:

test@test:~/ags$ make
agspack files game.ags
make: agspack: No such file or directory
make: *** [Makefile:18: all] Error 127

Problems with agssemble

Hello.

I'm trying to assemble a room.S into room.O, with agssemble.out, but I get an error. The error message reads:

./Assembler.c:138: get_variable_offset: Assertion '0' failed.

Please, could you tell me what wrong might be happening or give me a hint about how to fix it?

I haven't modified the files. They are the same ones that I extracted with agscriptxtract.out, a few minutes ago.

Thanks for your attention.

Suggestion: an utility for un/repacking sprite file

Hello.
In the past I was considering a theoretical use of agsutils by a game developer to setup a patching system for their own game, and the remaining problem is a sprite file (acsprset.spr). In big games it may reach gigabytes in size. Currently agsutils allow to extract it from the game, but (afaik) not extract/repack its contents.

Hence, I think an utility to unpack/repack acsprset.spr could be a useful addition.

The related reading/writing code may be found here: https://github.com/adventuregamestudio/ags/blob/ags3/Common/ac/spritecache.cpp#L578
Note, the file comes in 2 variants: with compressed and uncompressed bitmaps. Only supported compression is RLE (there were user requests to add other options, which may occur in future).

Feat.req. Make script to automatically unpack, optimize scripts and pack back (maybe GUI)

subj

Issue with agsprite and spritecache version

Hello,
I am trying to update a language patch for a video game.
And it seems that I have a problem with the spritecache version.
The game seems working with version 11 of spritecache but agsprite converts the spritecache to version 6.
I have the message : warning: converting spritecache version 11 to version 6.
The game I am trying to modify seems working with version 11. It crashes if we give it a spr file with spritecache version 6.
What is the difference between version11 and 6. And how to get a workaround for this issue ?
Thank you in advance.

Problem extract sprites (Unsupported compression method 3)

I have a problem when I extract sprites. Prompt shows this message "Unsupported compression method 3"

The AGS is version 3.6.1.8 and the utils is the last commit.

Attacth spr file

[Feature request] Extraction and injection of .TRA / .TRS translation files

Most people using agsutils (at least, imho) will just want to do simple edits, like sprite replacement, improvement of graphics and changing text (fixing typos, doing a translation). agsutils is already able to do all of the three, however editing text requires actual disassembly of scripts and handling complex files.

So here's my idea: Could you add support for extraction / injection of .TRS / .TRA files from / into game files? From my understanding, both are simple search-replace lists, with .TRS being in text format and .TRA being in a binary format. Extraction would be a very useful feature, as it would provide a start point for generating new translations (the extracted file would of course just contain the original strings). Injection would provide us with a simple way to edit all text in the AGS game file.

BTW: For editing existing translation files, there's already the excellent AGS Translation Editor.

Random bug in kathy rain if you're interested

In the start of chapter 2, it's possible to, after talking to the guy that wants his internet back and opening the envelope to get a 'boot disk' without the bottom 'Read Instructions on floppy disk' context button, which is a dead end because it has the phone number of the IT guy Clyde.

You can't even press it (555-8181) manually without the number in the notebook (with it, you can on).

This is apparently very random because it happened to me now, then i restarted the game and it didn't again. It's very strange and i have no clue how to make it happen.

Support for extracting old AGS versions?

Which versions are currently supported? I'm trying to extract an older AGS game (5 days a stranger), but agstract says

error: ags clib header signature not found

However the hex dump of the exe shows it has some CLIB markers.

Maniac Mansion Deluxe v1.4 - extract/repack breaks the game

As the title says. If I simply extract the game with agsex and pack it again with agspack, I can't run it anymore (neither via the EXE, nor via ScummVM). In ScummVM I get an error about sprite format, although I changed nothing in that regard.

I can provide more details if needed. The game is easily available via a simple Google search.

restoring exe stub after repack

After packing ，i try to run the game and an Microsoft error message shows that it cannt run because of Version mismatch and contact the author

Readme mentions that
the new "exe" will miss the original windows exe stub, but is
compatible with the opensource AGS engine, as well as with scummvm ags port.
that way, a game is about 500 KB smaller after extract and repack as it originally was.

It is the reason why it cannt run correctly after packing?
How can i make it correctly?

Don't support multiarchive games

Hello.
Clib32.c don't change archive file by LibUid.
Example game Zniw Adventure:
ctgame.exe - main Clib archive...
ctgame.001
ctgame.002
...
ctgame.007
File acsprset.spr in ctgame.001 archive... But agstract try extract from ctgame.exe.

Broken TGA header

Extracting sprites from Primordia.

Gimp and Windows Photo throw an error opening the files produced by AGSprite. This is caused by two errant bytes, one in field 4 and one in field 5 (as referenced on Wikipedia).

Byte at 0x8 is Colormap BPP, this should be at 0x7. Overall BPP is at 0x13 should be at 0x11.

Removing one zero byte from 0x3-0x5 and the zero byte at 0x8 (after removing the first byte) allows the TGA to load successfully. Looking at the byte order of the TGA standard (lo-hi) this appears to be the correct solution.

This may be a compiler issue as well: http://www.paulbourke.net/dataformats/tga/

Unfortunately, repairing this is way above my skill level, I am not a C coder and don't understand how the code generates this bitstream.

Using Windows binaries from agsutils-0.9.5-win32.zip.

agscriptxtract: Segmentation fault when extracting

Are there any versions of game data that aren't supported? Trying it out, it seems to crash when trying to extract the built-in script header:

(gdb) run
Starting program: /home/mwillcock/build/agsutils/agscriptxtract ..
disassembling [game28.dta] ./globalscript.o -> ./globalscript.s
regenerating script header ./builtinscriptheader.ash

Program received signal SIGSEGV, Segmentation fault.
0x000055555555702d in dump_header (a=0x7fffffffc2c0, 
    fn=0x7fffffffbfc0 "./builtinscriptheader.ash") at agscriptxtract.c:168
168		for(i=0; i<ADF_get_viewcount(a); ++i) if(ADF_get_viewname(a, i)[0])

Win build pls

subj

soviet´unterzogersdorf extract / repack

Hi,

tried to depack the following abandond ags game & repack it for usage with the opensource AGS 3.5 editor.

Freeware files are from here:
http://web.archive.org/web/20160328095603if_/http://monochrom.sil.at/suzoeg-s1-v1_0MrX-win-install.exe
http://web.archive.org/web/20110706094544if_/http://monochrom.sil.at/monochrom-SUZ2--version1dot2.exe

environment win10 + ubuntu + your fixed binaries from december 27th
after extracting the installer-exe, applied following for the second game, SUZ2 (first one behaves the same):

./agsex suz2prog.exe files obj
./agspack files suz2.agf

importing then into the AGS 3.5 editor fails with an import exception, instantly.
Not sure if a bug or misusage?

(also playing with an older editor, AGS 2.72, which imported at least the dta from the files directory, was ultimately not successful, script are lost that way)

thanks for feedback on this issue

[Feature request]Palette issue with agsprite

Hello,
I have found that agsprite when extracting files from 'acsprset.spr' changes the palette of the sprites or even creates a new palette.
My purpose was to mix the original sprites of a game with sprites from a language pack.
It worked pretty well but in the result the new sprites have palette issues in the game. The game was working but
the colors were wrong.
To solved this issue I modified you program in order not to modify/create the palette of the sprites.
And now, the game colors are ok.
My suggestion is the following : is it possible to create an option when launching agsprite to preserve le original state of the palette of the sprites ? This is the case for bpp==1 and others.
The repacking does not change.
It is a suggestion for future use of packing/re-packing... personaly my problem is solved.
Thanks for maintaining these tools.

latest Cygwin build pls

subj

applications only work in current directory of the game

Long story short, i tried to invoke agsex from a parent dir containing the scripts to a game dir and extraction dir children.

It broke a lot.

First the script agsex, where I changed BINDIR=$(dirname $(readlink -f "$0")) -> BINDIR="$(dirname "$(readlink -f "$0")")" and

$BINDIR/agstract "$GAME" "$FILES"
$BINDIR/agscriptxtract "$FILES" "$OBJS"

"$BINDIR"/agstract "$GAME" "$FILES"
"$BINDIR"/agscriptxtract "$FILES" "$OBJS"

Then it broken in agsextract and i gave up and tried to execute in the same directory as the executable (of the game). Behold, it worked. Maybe it's best to detect this case and warn it's not supported before exiting.

Executable footer changes in AGS 3.5.0

Hello, thanks for the tools! I've been trying to extract resources from a game compiled with AGS 3.5.0 without much success. The code seems to look for a small footer at the end of the file which contains the offset of the clib data followed by the string CLIB\x1\x2\x3\x4SIGE; on the AGS 3.5.0 game, the end string is there, but the offset are all 0s (Last'n'Furious used as the AGS 3.4.1 reference):
AGS 3.4.1 game

            C  L  I  B              S  I  G  E
D1 0E 9A 00 43 4C 49 42 01 02 03 04 53 49 47 45
^^^^^^^^^^^

AGS 3.5.0 game

            C  L  I  B              S  I  G  E
00 00 00 00 43 4C 49 42 01 02 03 04 53 49 47 45
^^^^^^^^^^^
should be offset, it's 0s instead

If I hardcode the offset for the begginning of the clib block (which I found by searching for CLIB\x1A inside of the executable, in my case it was the 2nd occurence), extraction appears to work (though the extracted acsprset.spr does not extract with agsprite, and some of the music files do not play; that's probably a different issue though); I haven't yet looked at how the engine copes with those changes.

Feat.req. GUI

maybe in far perspective)

Can`t compile under Winblows What I do wrong

gcc 8.1,9.2.0
Z:\ags>make
cc -Wall -Wextra -Wno-unknown-pragmas -Wno-sign-compare -Wno-switch -Wno-unused -Wno-pointer-sign -o ByteArray.o -c ByteArray.c
process_begin: CreateProcess(NULL, cc -Wall -Wextra -Wno-unknown-pragmas -Wno-sign-compare -Wno-switch -Wno-unused -Wno-pointer-sign -o ByteArray.o -c ByteArray.c, ...) failed.
make (e=2): ═х єфрхЄё эрщЄш єърчрээ√щ Їрщы.
make: *** [Makefile:57: ByteArray.o] Error 2

Feat.req. nightly build

subj

Extracting doesn't work

Hi!

I have tried to extract the files with your extract utility, but I always get an error message. Can you please tell me how I can extract the files from the exe file with your tool?

bye

rofl0r / agsutils Goto Github PK

agsutils's Introduction

agsutils's People

Contributors

Stargazers

Watchers

Forkers

agsutils's Issues

Recommend Projects

Recommend Topics

Recommend Org