Hey i think i did everything right but i get:

intercept.log

[ 15:08:57 ] Staring up (fake)\dxgi.dll
[ 15:08:57 ] Initialized: C:\WINDOWS\system32\dxgi.dll
[ 15:08:57 ] Hooking d3d11 functions
[ 15:08:58 ] Loading the API
[ 15:08:58 ] Loading API
[ 15:08:58 ] Loading and patching plugin capture.hk
[ 15:08:58 ] Capture turned on
[ 15:08:58 ] Loading and patching plugin gta5.hk
[ 15:08:58 ] LoadLibrary failed 7e
[ 15:08:58 ] Failed to load plugin 'gta5.hk'!
[ 15:08:58 ] Loading and patching plugin python.hk
[ 15:08:58 ] Python plugin turned on
[ 15:08:58 ] Loading and patching plugin server.hk
[ 15:08:58 ] Server turned on
[ 15:09:00 ] Hooking the device and device context
[ 15:09:00 ] Hooking the swap chain
[ 15:09:00 ] Hooking the io

And it just crashes.
Help! :)

Hello, I'm currently attempting to utilize the gamehook_gtav tool for extracting depth maps from the GTA game. However, I have concerns regarding its compatibility with the latest Steam versions of GTA, and possibly other versions as well.
While I've successfully compiled both gamehook and gamehook_gtav, I'm facing an issue when I place the generated files (including dxgi.dll) into the game's folder: nothing happens upon game startup. Despite my attempts to rename the generated DLL to either "d3d10.dll" or "d3d11.dll," I do not see any changes or results.

I've inspected GTAV.exe using Dependency Walker, and observed that the DLLs the game loads do not include "dxgi.dll." The closest match in the dependencies is "d3d9.dll," which is unexpected given that the game employs DirectX 10 or 11. Interestingly, renaming "dxgi.dll" to "d3d9.dll" triggers an error: "The procedure entry point D3DPERF_BeginEvent could not be located in the dynamic link library GTA.exe."

Does this situation imply that the dxgi project within the solution needs a substantial rewrite to function effectively with the current versions of GTA?

Below I attach the result of the dependency scan on my GTAV version.
Thanks for your attention.

GTA_dependencies.txt

All of the computation in startDraw for rage_bonemtx slows down the render loop quite a bit. In particular

https://github.com/philkr/gamehook_gtav/blob/master/gta5.cpp#L301

I haven't found any code in the repo that uses this data - is it necessary?

in speed.cpp it looks like you investigated hooking into the application timer to increase the perceived time delta between frames. I am looking to do something similar, but not having much success. Did you ever get this working with GTA5?

It doesn't seem to get past the first frame with speed anything other than 1.0 (although it does load if speed == 1.0) -- if I try debugging and changing the value mid-game, it doesn't look to have much of an effect although it's clearly calling the nQueryPerformanceCounter function.

Thanks for sharing the code! I'm able to run the latest version (the forked repo from bradyz) and trying to generate optical flow data for training.

However, I find that flow on human limbs are usually incorrect. If I warp the prev. frame according to the flow, some parts of the human are missing while pixels of the cars and background are aligned well. As shown in the figure below (left: current frame, right: warped previous frame), the leg region circled in red in missing.

To further examine this issue, I plot the 3D points of the (t-1) frame in red and the 3D points in the prev_pos variable of the (t) frame in blue under the same coordinate. They are supposed to perfectly overlap. However, the points on the limbs do not overlap.

I'm trying to debug this, but find it difficult. Could you give me some hints of what could go wrong?

Hi philkr, thanks for this amazing work.
I'm trying to reproduce the optical flow capturing inside GTA as you presented in the paper. After compiling the two projects(gamehook and gamehook_gtav) I put dxgi.dll and gta5.hk in the GTA directory as you suggested. The hooking process seems fine but ScriptHookV reported a fatal error:

As shown in the screenshot, my game version(bottom right) is supported by this version of ScriptHookV. I tried to start the game with ScriptHookV.dll only and with both ScriptHookV.dll and dxgi.dll, the game started as usual and according to the intercept log the hooking process is also working well.

Am I doing anything wrong? I tried digging up in the code for a while and didn't see what would affect version detection of ScriptHook.

Thanks for your time.

@bradyz or I'll have to look into it. It looks like the static flow is messed up too (prev_?? matrix might be wrong).

I managed to get everything set up, but I'm getting mostly 'Not hooking DrawInstancedIndirect' printed in the intercept log. Consequently, when I go to http://localhost:8766/bmp?t=flow, all I see is a black screen.

I also get

[ 14:59:14 ] Target 'final' exists, but was not written. Check your plugins to make sure all targets are written properly!
[ 14:59:14 ] Target 'albedo' exists, but was not written. Check your plugins to make sure all targets are written properly!

Am I missing something?

In Dec. 2018, I successfully dumped object id, disparity, albedo, final image and flow.

But I just updated GTA to version 1.0.1604.1 and can only dump albedo and final images to bmp files. Others are all .raw files. I tried to use RenderDoc to find the shader hash and modify the hash in gta5.cpp but cannot help.

How can I fix this problem?

I somehow let the gta v loaded gta5.hk. But, after F12 pressed in game, I only see a lot of *_state.json file in cap directory. How can i capture images?

Hello!

I want to extract depth map using this plugin from GTA-5. Can it be done? If so, can you please provide some guidelines how it can be done.

Thanks in advance
Gopi

Although the bone matricies are one source of slowdown during capture, the other source is the readTarget function at the end of the frame which I assume is forcing a GPU sync. In other work, I usually have a pool of staging resources that I copy to during the Present call, and then Map after a few frames to avoid forcing the GPU to sync. I want to apply that here - but the way you have organized the D3D11 hook makes that difficult (it's hard to access the ID3D11Texture2D's of the targets, and it's not easy to get a ID3D11Device pointer or a ID3D11Context pointer. It feels like you have thought about these problems since you have staging textures in rendertarget.cpp... but it's pretty opaque and I'm having a hard time determining if every target we are saving is being handled properly.

I tried creating a pool for each target (so "final" and "final0" and "final1" etc) that I wanted to copyTarget(from=flow, to=flow0) but I get an error that I "Cannot copy output to custom render targets".

Any ideas on how to achieve this?

Hi! I currently have an Script that spawns vehicles into the GTAV world, and start the capture plugin from GameHook. Is there a way to know relate the car spawned with the object_id you get from the object_id target?

Ik that i can use ((id >> 28) & 0xf) and id & ((1<<28)-1) to get InstanceID and ObjectID but there is no way to relate the vehicle spawned with any of the Id's output

I'm trying to add avg_world, avg_world_view, and avg_world_view_proj to the JSON output. I've created a new function:

std::string toJSON(float4x4 & f) {
	std::stringstream s;
	s << "[ [" << f.d[0][0] << ", " << f.d[0][1] << ", " << f.d[0][2] << ", " << f.d[0][3] << "], ["
		<< f.d[1][0] << ", " << f.d[1][1] << ", " << f.d[1][2] << ", " << f.d[1][3] << "], ["
		<< f.d[2][0] << ", " << f.d[2][1] << ", " << f.d[2][2] << ", " << f.d[2][3] << "], ["
		<< f.d[3][0] << ", " << f.d[3][1] << ", " << f.d[3][2] << ", " << f.d[3][3] << "] ]";
	return s.str();
}

And added members to the GameInfo:

struct GameInfo {
	int time_since_player_hit_vehicle;
	int time_since_player_hit_ped;
	int time_since_player_drove_on_pavement;
	int time_since_player_drove_against_traffic;
	int dead;
	Vec3f position, forward_vector;
	float heading;
	int on_foot, in_vehicle, on_bike, money;
	float4x4 avg_world;
	float4x4 avg_world_view;
	float4x4 avg_world_view_projection;
};

And added the members to the TOJSON macro:

TOJSON(GameInfo, time_since_player_hit_vehicle, time_since_player_hit_ped, time_since_player_drove_on_pavement, time_since_player_drove_against_traffic, dead, position, forward_vector, heading, on_foot, in_vehicle, on_bike, money, avg_world, avg_world_view, avg_world_view_projection)

But, the output returns just a "1":
{"time_since_player_hit_vehicle":-1,"time_since_player_hit_ped":-1,"time_since_player_drove_on_pavement":-1,"time_since_player_drove_against_traffic":-1,"dead":0,"position":[-14.380300,-1438.514282,31.101547],"forward_vector":[0.222740,-0.974878,0.000000],"heading":192.870010,"on_foot":1,"in_vehicle":0,"on_bike":0,"money":11,"avg_world":1,"avg_world_view":1,"avg_world_view_projection":1,"timestamp": 1534989424.864456}

Any ideas?

So I have managed to thumble through some version mismatches accumulated over the years to compile gamehook.dll so I thought I would summarise these here:

For those playing in 2022 - Visual Studio needs both an old SDK and 2017 compatibility installed. 2017 Compatibility can be installed via VS installer but I found that the specific SDK version requested by this project had to be downloaded and installed from here (https://developer.microsoft.com/en-us/windows/downloads/sdk-archive/).
You also need to do what is in this old closed post (philkr/gamehook#11) and clone the simple-web-server/simple-websocket-server repositories to build the server plugin. None of the tagged versions for this work out the box however, in the end I pulled master then made two edits

1. In asio_compatibility.hpp change #if(ASIO_STANDALONE && ASIO_VERSION >= 101300) || BOOST_ASIO_VERSION >= 101300 to #if(0 && ASIO_VERSION >= 101300) || BOOST_ASIO_VERSION >= 101300
2. In server.cpp comment out 'connection->remote_endpoint = std::move(*request -> remote_endpoint);' (Not sure what this will break later??)
   Then you either need to add a PythonDir system environment variable that points to the python version you want to use in building the project or manually configure the project using Project -> Properties then modifying both VC++ Directories -> Include Directories and Linker -> General -> Additional Library Directories to point to the python include and libs folders respectively.
   Finally make sure the build configuration is set to x64 and release.
   Now you should get a successful build and a gamehook.dll file to play with.

To my issue:
I get the command prompt and things appear to be starting but I get More than one GameHookBuffer created. Things will end badly from here on! and then the process gets killed after printing Hooking the io

When loading and running the gtav.hk file with dxgi.dll and Scripthook V in my directory, I get the following error:

I'm currently running GTAV version 1.0.1493.1, but tried reverting to the previous version as well and no dice. Further inspection indicates that this issue is happening when hooking into GTAV.exe (the blacklisting code added to the repo seems to be running fine).

This error is only happening when ScripthookV.dll and gtav.hk are within the GTAV folder together - other *.hk files are loading & running just fine and I'm not noticing any issues if I use dxgi.dll and/or dinput8.dll with the Scripthook file alone.

Any help would be much appreciated! Thanks so much for all your work on this.