Cage does not work with Firefox about cage HOT 11 CLOSED

I found out that Firefox Nightly (version 66) works just fine if it isn't the first application that is started in Cage. That is, if you run ./cage epiphany from one terminal and from another run GDK_BACKEND=wayland WAYLAND_DISPLAY=wayland-1 firefox-nightly, Firefox Nightly should start and work just fine. (Or, maybe easier, is to run ./cage termite and from termite run GDK_BACKEND=wayland firefox-nightly.)

If Cage is compiled without XWayland, Firefox Nightly should also start with ./cage firefox-nightly. If you compile Cage with XWayland and try to run Firefox Nightly like that, you'll get the following:

(EE) failed to read Wayland events: Broken pipe
ExceptionHandler::GenerateDump cloned child 6050
ExceptionHandler::SendContinueSignalToChild sent continue signal to child
ExceptionHandler::WaitForContinueSignal waiting for continue signal...

To summarize:

1. Cage compiled with XWayland, Firefox Nightly works only if another application is started first (i.e., if Cage is properly done starting up).
2. If Cage is compiled without XWayland, Firefox Nightly works as start application as well as if another application is started first.

I'm not sure why this is the case. That is, I'm not sure why Firefox Nightly doesn't work as startup application when XWayland support is enabled. This is what we'll need to figure out to get Firefox Nightly to work properly in all conditions. This works just fine in rootston, so if we can figure out what is different between rootston and Cage we should be good to go.

(Note: the above doesn't work the same for Firefox (stable, i.e. version 64 and not Nightly), but I haven't dug into this more yet since Firefox Nightly is really the version you'll want to use on Wayland.)

from cage.

Confirmed this has to do with how we spawn applications: if I disable all application spawning code in Cage and execute ./build/cage & GDK_BACKEND=wayland WAYLAND_DISPLAY=wayland-1 firefox-nightly with XWayland support enabled, Firefox Nightly spawns as it should.

from cage.

I found the issue. It's completely unrelated to how Cage spawns processes or handles XWayland.

Apparently, when XWayland support is enabled, Firefox first creates an XWayland surface, closes this and then opens an XDG toplevel surface. Cage tries to manage the first XWayland surface, but when it closes, Cage has no surfaces left and hence closes as well. This is Cage specific behavior: it closes when there are no views left. Hence, Cage terminates before it picks up on Firefox's XDG toplevel, and Firefox (rightly) prints it cannot read the Wayland pipe.

So, now we need to figure out why Firefox spawns an XWayland surface with GDK_BACKEND=wayland, or change the way we handle windows. According to @emersion on IRC, GTK is known to open an X11 connection when XWayland is available. Let's see if we can find out why...

from cage.

Is the Xwayland window mapped?

from cage.

Is the Xwayland window mapped?

No, it doesn't get mapped.

By the way, I looked at other GTK applications. They do not spawn XWayland surfaces (with XWayland available), so it seems to be specific to Firefox. Do you happen to know a developer working on Firefox on Wayland?

from cage.

No, it doesn't get mapped.

Maybe you could wait for a window to get mapped before exiting.

They do not spawn XWayland surfaces (with XWayland available), so it seems to be specific to Firefox.

Ah, so they connect and don't spawn anything. Interesting.

Do you happen to know a developer working on Firefox on Wayland?

I think stransky does a lot to port Firefox to Wayland.

from cage.

@stransky, do you know why Firefox Nightly spawns an XWayland surface when XWayland is available, only to (almost?) immediately close it and use xdg-shell after? I have no problem with adding a workaround in Cage, but it seems like this is something you might want to fix in Firefox so I'm pursuing that option first.

from cage.

I have an initial workaround going in the branch firefox-workaround. I'm not yet sure if this is the best way to go about it and if I should generalize this approach to make for example xdg-shell handling more robust as well. It hasn't yet seemed to be needed for xdg-shell...

from cage.

I went ahead and merged the (cleaned up) workaround, since it is also required for Google Chrome/Chromium.

from cage.

@stransky, do you know why Firefox Nightly spawns an XWayland surface when XWayland is available, only to (almost?) immediately close it and use xdg-shell after? I have no problem with adding a workaround in Cage, but it seems like this is something you might want to fix in Firefox so I'm pursuing that option first.

Can you please file a bug at http://bugzilla.mozilla.org/ and also provide a backtrace when the XWayland surface is created? I'm not aware about it. Maybe that comes from Gtk+ itself or from some library.

from cage.

@stransky I submitted a bug: https://bugzilla.mozilla.org/show_bug.cgi?id=1523889.

I wasn't sure what you meant with a backtrace, so I didn't include one. Note that nothing crashes, not in Firefox nor in Cage. Firefox just exits because there is no Wayland socket left.

from cage.