Gnome 40 Introducing Headless Native Back End, Virtual Monitors(phoronix.com)
phoronix.com
Gnome 40 Introducing Headless Native Back End, Virtual Monitors
https://www.phoronix.com/scan.php?page=news_item&px=GNOME-40-Headless-Virtual
30 コメント
The virtual monitors feature is niche but very useful when you need it. It is useful for screensharing when you don't have a spare monitor to use but want to arrange a couple of windows. It is so useful that people by dedicated hardware to convince their system to give them another monitor. (HDMI dummy plugs)
Its useful for steamvr too which shows you your monitors so a dummy monitor gives you an extra monitor in vr
Is is about showing only a certain section of the screen where you have arranged those windows, instead of all of it?
This is one of the sort-of-frustrating as-a-user things about Wayland to me, that every compositor (nee window manager) has to implement capabilities like this for themselves, where-as in X, there was common tooling. And they'll all have different userland tools to use these capabilities.
here[1] for example is wlroots/sway implementing headless.
That said, the implementations are great, & really show how much Linux has progressed. For a while it wasn't possible (for some reason I wasn't having good luck with either mdi or dp dummy plugs on my type-c only laptop), but I'm really looking forward to using one laptop as a virtual desktop for another desktop, & this work is core to that effort. I like a lot that Gnome was super smart & made outputting a virtual monitor as a PipeWire stream built in!! Smart easy win to make screen capturing even easier.
Edit: at my posting rate-limit, so tacking this on, especially to @spivak,
I like the diversity a lot, yes. But I still would like to see a situation other than gnome having it's own tooling to configure virtual monitors, wlroots having it's own, kde having it's own.
Is this just an insufficienct protocolization problem right now? Are there wayland protocols to do things like change a monitor's resolution? Or to change the mouse speed? I think of Wayland protocols as more between a running app and the compositor, which has all been nicely modularized & protocolized. But I don't understand what if any effort has been made to find a common ground to expose user tooling. It feels like they all have their own suite of command line utilities. Swaymsg in sway, for example. Whatever this virtual monitor tool gnome has. How do we begin to modularize these control interfaces?
[1] https://github.com/swaywm/wlroots/commits/master/backend/hea...
here[1] for example is wlroots/sway implementing headless.
That said, the implementations are great, & really show how much Linux has progressed. For a while it wasn't possible (for some reason I wasn't having good luck with either mdi or dp dummy plugs on my type-c only laptop), but I'm really looking forward to using one laptop as a virtual desktop for another desktop, & this work is core to that effort. I like a lot that Gnome was super smart & made outputting a virtual monitor as a PipeWire stream built in!! Smart easy win to make screen capturing even easier.
Edit: at my posting rate-limit, so tacking this on, especially to @spivak,
I like the diversity a lot, yes. But I still would like to see a situation other than gnome having it's own tooling to configure virtual monitors, wlroots having it's own, kde having it's own.
Is this just an insufficienct protocolization problem right now? Are there wayland protocols to do things like change a monitor's resolution? Or to change the mouse speed? I think of Wayland protocols as more between a running app and the compositor, which has all been nicely modularized & protocolized. But I don't understand what if any effort has been made to find a common ground to expose user tooling. It feels like they all have their own suite of command line utilities. Swaymsg in sway, for example. Whatever this virtual monitor tool gnome has. How do we begin to modularize these control interfaces?
[1] https://github.com/swaywm/wlroots/commits/master/backend/hea...
This is more a culture thing than anything technical. Wayland is meant as a replacement for X which is "just" the wire protocol that windows use to talk to the display server. But compositors like Sway, Mutter, and Kwin are angling to replace Xorg, the single monolithic display server that happened to speak X. Xorg could have just added Wayland support (in theory not in practice) and so the death of Xorg left a huge hole that everyone is trying to fill and no dominant project as emerged yet and Wayland has made it easier than ever to just DIY.
To me this is a good thing because having multiple competing implementations of a standard makes the standard authoritative and avoids the "do what Xorg does" problem.
To me this is a good thing because having multiple competing implementations of a standard makes the standard authoritative and avoids the "do what Xorg does" problem.
> angling to replace Xorg
The Xrandr extension enabling all this functionality is defacto part of the X11 protocol for a long time and not just "Xorg".
> Xorg could have just added Wayland support
You already can run any Wayland client in Xorg with a simple Weston wrapper script. However since X11 supports DRI3 which is as efficient as Wayland will ever be it is completely pointless.
> To me this is a good thing
Speak for yourself. For me standardization is a good thing. The "Desktop" is essentially a solved problem. There is no reason to invent a new incompatible standard every week. What is happening here is the opposite of standardization.
The Xrandr extension enabling all this functionality is defacto part of the X11 protocol for a long time and not just "Xorg".
> Xorg could have just added Wayland support
You already can run any Wayland client in Xorg with a simple Weston wrapper script. However since X11 supports DRI3 which is as efficient as Wayland will ever be it is completely pointless.
> To me this is a good thing
Speak for yourself. For me standardization is a good thing. The "Desktop" is essentially a solved problem. There is no reason to invent a new incompatible standard every week. What is happening here is the opposite of standardization.
> The "Desktop" is essentially a solved problem.
Serious question, what are the key differences and tradeoffs between the DRI3 and Wayland APIs? I've done some OpenGL programming, but always in a fully abstracted manner (ie using cross platform wrapper libraries).
As a bystander, my impression is that the real issue (outside of refactoring a massive codebase & associated ecosystem on a volunteer basis) is one of security and efficiency. Without reworking the underlying protocol to require compositing, how do you properly sandbox applications from one another? And as long as you're making such a significant overhaul, why not throw out a bunch of cruft and split off any extra functionalities into their own standards? And as long as you're doing all that, why not completely restructure the protocol to handle things like minimized windows updating thumbnails of themselves at an interval determined by the compositor?
Serious question, what are the key differences and tradeoffs between the DRI3 and Wayland APIs? I've done some OpenGL programming, but always in a fully abstracted manner (ie using cross platform wrapper libraries).
As a bystander, my impression is that the real issue (outside of refactoring a massive codebase & associated ecosystem on a volunteer basis) is one of security and efficiency. Without reworking the underlying protocol to require compositing, how do you properly sandbox applications from one another? And as long as you're making such a significant overhaul, why not throw out a bunch of cruft and split off any extra functionalities into their own standards? And as long as you're doing all that, why not completely restructure the protocol to handle things like minimized windows updating thumbnails of themselves at an interval determined by the compositor?
> Serious question, what are the key differences and tradeoffs between the DRI3 and Wayland APIs?
DRI3 gets you a file descriptor which can be used as a DMA-Buffer to directly render and present graphics. The Wayland protocol never clarifies how the compositor is supposed to implement the rendering but in practice the current implementations do exactly the same thing as DRI3 except in the case of GNOME and KDE on proprietary drivers from NVIDIA which use nvm_eglstreams.
> As a bystander, my impression is that the real issue is one of security and efficiency.
That is correct. As of now X11/Xorg has very broken security.
> Without reworking the underlying protocol to require compositing, how do you properly sandbox applications from one another?
There are many X11 security protocols already standardized [1,2,3] using access control hooks which lets you sandbox any aspect of your application but no client and/or toolkit ever adopted it. Also the current implementation of xinput invalidates many of those protocols.
> And as long as you're making such a significant overhaul, why not throw out a bunch of cruft and split off any extra functionalities into their own standards?
Wayland shows in a very painful way that those functionalities are strongly interdependent and untangling them leads only to more protocol overhead.
> And as long as you're doing all that, why not completely restructure the protocol to handle things like minimized windows updating thumbnails of themselves at an interval determined by the compositor?
That is a very specific problem, however A custom xcompmgr implementation would allow you do that today on X11.
1.: https://www.x.org/wiki/Development/Documentation/Security/
2.: https://www.x.org/releases/current/doc/xextproto/security.ht...
3.: https://www.x.org/releases/X11R7.6/doc/xorg-docs/specs/Xserv...
DRI3 gets you a file descriptor which can be used as a DMA-Buffer to directly render and present graphics. The Wayland protocol never clarifies how the compositor is supposed to implement the rendering but in practice the current implementations do exactly the same thing as DRI3 except in the case of GNOME and KDE on proprietary drivers from NVIDIA which use nvm_eglstreams.
> As a bystander, my impression is that the real issue is one of security and efficiency.
That is correct. As of now X11/Xorg has very broken security.
> Without reworking the underlying protocol to require compositing, how do you properly sandbox applications from one another?
There are many X11 security protocols already standardized [1,2,3] using access control hooks which lets you sandbox any aspect of your application but no client and/or toolkit ever adopted it. Also the current implementation of xinput invalidates many of those protocols.
> And as long as you're making such a significant overhaul, why not throw out a bunch of cruft and split off any extra functionalities into their own standards?
Wayland shows in a very painful way that those functionalities are strongly interdependent and untangling them leads only to more protocol overhead.
> And as long as you're doing all that, why not completely restructure the protocol to handle things like minimized windows updating thumbnails of themselves at an interval determined by the compositor?
That is a very specific problem, however A custom xcompmgr implementation would allow you do that today on X11.
1.: https://www.x.org/wiki/Development/Documentation/Security/
2.: https://www.x.org/releases/current/doc/xextproto/security.ht...
3.: https://www.x.org/releases/X11R7.6/doc/xorg-docs/specs/Xserv...
Thanks for the detailed answer, those security protocols in particular are quite interesting.
Regarding rendering callbacks (thumbnails & etc), it's perhaps not as specific a problem as it might seem. In classic OpenGL, the potential to block the render loop (and thus the entire window event system) when placing a swap buffer (ie present) call is incredibly frustrating. Also, particularly on resource constrained devices, an event driven approach can be quite useful (both for performance and battery reasons). It's generally just a much more flexible model overall.
You mentioned a custom xcompmgr implementation. I didn't think X11 used a callback model though? Was I mistaken? (Again, I've only ever done cross platform OpenGL code using wrapper libraries. That's always involved a potentially blocking call to swap the buffer at the end of the render loop in my experience.)
Regarding rendering callbacks (thumbnails & etc), it's perhaps not as specific a problem as it might seem. In classic OpenGL, the potential to block the render loop (and thus the entire window event system) when placing a swap buffer (ie present) call is incredibly frustrating. Also, particularly on resource constrained devices, an event driven approach can be quite useful (both for performance and battery reasons). It's generally just a much more flexible model overall.
You mentioned a custom xcompmgr implementation. I didn't think X11 used a callback model though? Was I mistaken? (Again, I've only ever done cross platform OpenGL code using wrapper libraries. That's always involved a potentially blocking call to swap the buffer at the end of the render loop in my experience.)
> > Xorg could have just added Wayland support
> You already can run any Wayland client in Xorg with a simple Weston wrapper script. However since X11 supports DRI3 which is as efficient as Wayland will ever be it is completely pointless.
Far from completely pointless, it sounds useful to run Wayland applications on X11; and could well be a useful contingency or ease to transition.
Do you have a link or more detail on which script and how to do this?
> You already can run any Wayland client in Xorg with a simple Weston wrapper script. However since X11 supports DRI3 which is as efficient as Wayland will ever be it is completely pointless.
Far from completely pointless, it sounds useful to run Wayland applications on X11; and could well be a useful contingency or ease to transition.
Do you have a link or more detail on which script and how to do this?
[deleted]
It's a little bit more than just culture. The Wayland protocol has made the compositor, window manager and window server concepts a single thing whereas in X those are provided by different applications talking a standardized protocol. It did that for good reason and there are technical ways to then implement those parts with common code still. But since that sharing is not enforced by the protocol itself there's still duplication of things that in the Xorg world were more stabilized with no real competing implementations.
> more than just culture. The Wayland protocol has made the compositor, window manager and window server concepts a single thing
I realize that any implementation of the Wayland protocol must of necessity involve compositing somewhere in the stack simply due to the nature of the protocol itself. And I also realize that all current Wayland "compositors" happen to roll all this functionality into a single monolithic implementation. But I honestly don't understand what's stopping someone from implementing a modular Wayland display server which you plug a conforming compositor into (with the caveat that, unlike X11, such a module would not be optional).
Is there some technical issue I'm unaware of that would prevent a standalone display server implementation speaking the Wayland protocol from delegating all the necessary behavior via indirection?
I realize that any implementation of the Wayland protocol must of necessity involve compositing somewhere in the stack simply due to the nature of the protocol itself. And I also realize that all current Wayland "compositors" happen to roll all this functionality into a single monolithic implementation. But I honestly don't understand what's stopping someone from implementing a modular Wayland display server which you plug a conforming compositor into (with the caveat that, unlike X11, such a module would not be optional).
Is there some technical issue I'm unaware of that would prevent a standalone display server implementation speaking the Wayland protocol from delegating all the necessary behavior via indirection?
There could be another library providing these common utilities, which multiple window managers could use.
It exists for Wayland: it's called wlroots.
Unfortunately, wlroots is only used by everyone except Gnome and KDE, who seem to prefer to do their own thing.
Unfortunately, wlroots is only used by everyone except Gnome and KDE, who seem to prefer to do their own thing.
Not just that GNOME and KDE's Wayland support both predate wlroots, but they also go through the effort to support EGLStreams, necessary for NVIDIA compatibility. The wlroots author is explicitly against this.
So its used by everyone except for 90% of users.
Is there a library providing these common utilities, which multiple window managers could use?
Or it could be part of the Wayland protocol from the get go sparing everyone of major headaches.
At some point you arrive at a core functionality resembling X11 and you went full circle breaking everybody's workflow and tooling in the process for no reason other than severe NIHS.
At some point you arrive at a core functionality resembling X11 and you went full circle breaking everybody's workflow and tooling in the process for no reason other than severe NIHS.
Have you considered that maybe it was not done just to spite you and break your workflow?
Or that maybe it does have something to do with having a maintainable implementation that doesn't work under assumptions that stopped being valid 20 years ago which make modern usecases extremely hard or impossible to support?
And that indeed, the protocol has been growing and new standards have been in development concurrently with X.org maintenance in order to have a smooth transition?
You're proposing that having everything ready from the get go would be much better, but that goes against pretty much every principle we know about designing large, complex systems, where BUFD results in huge messes that don't conform to reality, and can never do so, unlike a minimalist protocol that grows over time with careful approaches.
Or that maybe it does have something to do with having a maintainable implementation that doesn't work under assumptions that stopped being valid 20 years ago which make modern usecases extremely hard or impossible to support?
And that indeed, the protocol has been growing and new standards have been in development concurrently with X.org maintenance in order to have a smooth transition?
You're proposing that having everything ready from the get go would be much better, but that goes against pretty much every principle we know about designing large, complex systems, where BUFD results in huge messes that don't conform to reality, and can never do so, unlike a minimalist protocol that grows over time with careful approaches.
> in order to have a smooth transition
I was with you until that bit. I like the idea of Wayland, but it simply wasn't a viable daily replacement for typical desktop activities without certain key supporting features. Unfortunately, Wayland was rather aggressively pushed as a suitable replacement long before these things had been ironed out. Predictably, breakage and community backlash resulted.
Typical "real life" usage absolutely requires standardized support for screenshots and screencasts. As far as I understand things, that requires Pipewire support at both the compositor and application levels. That's only just happening now.
For gaming and 3D work, did the major implementations ever standardize a protocol for a window to capture the pointer? (I assume this has been done by now.)
Did an approach for handling key remapping and macros ever get standardized across implementations? (I also assume this was finally taken care of.)
I was with you until that bit. I like the idea of Wayland, but it simply wasn't a viable daily replacement for typical desktop activities without certain key supporting features. Unfortunately, Wayland was rather aggressively pushed as a suitable replacement long before these things had been ironed out. Predictably, breakage and community backlash resulted.
Typical "real life" usage absolutely requires standardized support for screenshots and screencasts. As far as I understand things, that requires Pipewire support at both the compositor and application levels. That's only just happening now.
For gaming and 3D work, did the major implementations ever standardize a protocol for a window to capture the pointer? (I assume this has been done by now.)
Did an approach for handling key remapping and macros ever get standardized across implementations? (I also assume this was finally taken care of.)
> but it simply wasn't a viable daily replacement for typical desktop activities without certain key supporting features
Well that's my point, then don't use it. It has been increasingly ready for more and more people, and it certainly needs more users to share what the pain points are, but you don't have to switch until enough progress has been made so that the transition is smooth enough for you.
> was rather aggressively pushed
By whom? a few distros toyed with the idea of making it the default and went back on it quickly, only now is it becoming a serious idea again and it's still a long time before X11 gets actually dropped rather than just not being the default.
What is a problem which you might consider as it being "pushed" is that X as a primary system is now essentially unmaintained upstream. The people that were maintaining X11 don't feel like it's worth the royal pain in the ass, moved to developing Wayland and are passing the baton to anyone willing to handle that particular burden.
> For gaming and 3D work, did the major implementations ever standardize a protocol for a window to capture the pointer? (I assume this has been done by now.)
Would not be able to tell you without searching for it like you could, but I've tested games that required full screen pointer capture on Sway, Wayfire (both wlroots) and GNOME successfully.
> Did an approach for handling key remapping and macros ever get standardized across implementations?
Key remapping, sort of, it's just xkb which is the low-level layer X builds upon and for which you can build keyboard layouts with no decent documentation in sight. As for key bindings, everyone has their own system, which works, but just isn't portable. This is still a big omission for some usecases which can be worked around but is still a rough edge.
Well that's my point, then don't use it. It has been increasingly ready for more and more people, and it certainly needs more users to share what the pain points are, but you don't have to switch until enough progress has been made so that the transition is smooth enough for you.
> was rather aggressively pushed
By whom? a few distros toyed with the idea of making it the default and went back on it quickly, only now is it becoming a serious idea again and it's still a long time before X11 gets actually dropped rather than just not being the default.
What is a problem which you might consider as it being "pushed" is that X as a primary system is now essentially unmaintained upstream. The people that were maintaining X11 don't feel like it's worth the royal pain in the ass, moved to developing Wayland and are passing the baton to anyone willing to handle that particular burden.
> For gaming and 3D work, did the major implementations ever standardize a protocol for a window to capture the pointer? (I assume this has been done by now.)
Would not be able to tell you without searching for it like you could, but I've tested games that required full screen pointer capture on Sway, Wayfire (both wlroots) and GNOME successfully.
> Did an approach for handling key remapping and macros ever get standardized across implementations?
Key remapping, sort of, it's just xkb which is the low-level layer X builds upon and for which you can build keyboard layouts with no decent documentation in sight. As for key bindings, everyone has their own system, which works, but just isn't portable. This is still a big omission for some usecases which can be worked around but is still a rough edge.
> a few distros toyed with the idea of making it the default and went back on it quickly
As a bystander, it felt like a vocal minority was pretty aggressively underselling the shortcomings. Perhaps that's unfair but it's definitely how it came across to me. They appeared to back off a bit as a result of the community backlash, but the damage was done - many people appear to mentally associate Wayland with someone else threatening to break their system. Now we see endless threads like this one, even though Wayland itself actually seems like a really good idea to me from a technical perspective.
Xorg losing maintainers is certainly a concern, but since Pipewire appears to be quickly approaching mainstream hopefully everything will work out. It's just frustrating to me that anyone ever tried to market Wayland as more than a proof of concept prior to working out such essential details (screenshots, screencasts, pointer capture, input manipulation, and a bunch of others I can't remember).
I think an important takeaway is that breaking even remotely common workflows is likely to attract inordinate amounts of negative attention no matter how good your technical or circumstantial reasons are for doing so. It doesn't matter how cool the project is - most people just have no interest in hacking on their OS.
> with no decent documentation in sight
So it isn't just me then. I'm still using xmodmap because I never managed to figure out how to get xkb to do what I wanted.
As a bystander, it felt like a vocal minority was pretty aggressively underselling the shortcomings. Perhaps that's unfair but it's definitely how it came across to me. They appeared to back off a bit as a result of the community backlash, but the damage was done - many people appear to mentally associate Wayland with someone else threatening to break their system. Now we see endless threads like this one, even though Wayland itself actually seems like a really good idea to me from a technical perspective.
Xorg losing maintainers is certainly a concern, but since Pipewire appears to be quickly approaching mainstream hopefully everything will work out. It's just frustrating to me that anyone ever tried to market Wayland as more than a proof of concept prior to working out such essential details (screenshots, screencasts, pointer capture, input manipulation, and a bunch of others I can't remember).
I think an important takeaway is that breaking even remotely common workflows is likely to attract inordinate amounts of negative attention no matter how good your technical or circumstantial reasons are for doing so. It doesn't matter how cool the project is - most people just have no interest in hacking on their OS.
> with no decent documentation in sight
So it isn't just me then. I'm still using xmodmap because I never managed to figure out how to get xkb to do what I wanted.
This is fantastic. I have a box in the corner with no display attached and wasn’t able to RDP/VNC etc into it as there was no real display. I had mixed results with an abandoned virtual display package and ended up purchasing a tiny HDMI thing that plugs in the back and emulates a real display.
Do you have a link for the HDMI dongle thing?
Yeah this is the one I ordered: https://www.amazon.de/dp/B0788NPQFV/
ELI5 please. What is a headless native back end? These articles seem to be written for people who already know that it exists and what it is. How does that virtual monitor work?
Desktop environments like Gnome assume the hardware on the machine they're running on is all the displays they'll ever have to deal with. "Headless" means there is no display (the "head" of the computer) connected to the machine.
This update allows Gnome to use virtual monitors.
This update allows Gnome to use virtual monitors.