Doesn't work with Mangohud about obs-vkcapture HOT 10 CLOSED

Not really, and of course there's no source code to look into.
It does use dlsym and seems to be calling the original functions resolved with dlsym(RTLD_NEXT, ...), but it also is linked to libGL. Maybe that's enough to make it work, hmmm.

from obs-vkcapture.

Nice! I guess the perfect world scenario, where all OpenGL "overlay" applications work together no matter the order they are applied in, is impossible? smile

Yes, there is no standard way how to do it with OpenGL, as it is with Vulkan.

I've pushed another changes and it should now work fine with mangohud even for EGL. But I wonder what else will break.

from obs-vkcapture.

I don't think so, both use dlsym. Similarly I think it can't work with Steam overlay either.

from obs-vkcapture.

I don't think so, both use dlsym. Similarly I think it can't work with Steam overlay either.

Actually Steam overlay does work while I'm recording. It even gets included in the recording.

Screenshot from OBS:

from obs-vkcapture.

Hmm, then maybe Steam is hooking the functions without dlsym.
I think it's not possible to chain hooks with dlsym, but I am not absolutely sure. Does mangohud + libstrangle work together?

Also one thing to note, mangohud by default isn't using dlsym - you have to use mangohud --dlsym ....

from obs-vkcapture.

You can try this

diff --git a/src/glinject.c b/src/glinject.c
index e4b22a2..c9fb3c8 100644
--- a/src/glinject.c
+++ b/src/glinject.c
@@ -16,6 +16,8 @@ You should have received a copy of the GNU General Public License along
 with this program. If not, see <https://www.gnu.org/licenses/>
 */
 
+#define _GNU_SOURCE
+
 #include "glinject.h"
 #include "capture.h"
 #include "utils.h"
@@ -59,7 +61,10 @@ struct gl_data {
 static struct gl_data data;
 
 #define GETADDR(s, p, func) \
-    p.func = (typeof(p.func))real_dlsym(handle, #s #func); \
+    p.func = (typeof(p.func))real_dlsym(RTLD_NEXT, #s #func); \
+    if (!p.func) { \
+        p.func = (typeof(p.func))real_dlsym(handle, #s #func); \
+    } \
     if (!p.func) { \
         hlog("Failed to resolve " #s #func); \
         return false; \

Problem with this is that it makes mangohud + alacritty EGL crash. It will crash when trying to call GL functions obtained from mangohud's eglGetProcAddress.
And with libstrangle + glxgears glXGetProcAddress will return functions from obs-glcapture which crashes it too.

Honestly it's a mess.

from obs-vkcapture.

Any idea how Steam overlay is working fine with all these other hooks? 🤔

from obs-vkcapture.

By the way, I tried mangohud --dlsym just to be sure, and it didn't make a difference.

from obs-vkcapture.

Didn't make a difference when I tried myself either.

With current master it should work now with mangohud, as long as obs-glcapture is before mangohud.

from obs-vkcapture.

Nice! I guess the perfect world scenario, where all OpenGL "overlay" applications work together no matter the order they are applied in, is impossible? 😄

Perhaps at some point we'll have a good translation layer to convert OpenGL calls to Vulkan on the fly, like DXVK does for DirectX 😄 (I wonder if Zink is in a working state for games yet?)

from obs-vkcapture.