Immediate Mode GUI Programming(eliasnaur.com)
eliasnaur.com
Immediate Mode GUI Programming
https://eliasnaur.com/blog/immediate-mode-gui-programming
119 comments
It sounds like limitations of egui / Rust instead of immediate mode GUI. I've made flexbox-like layout systems in immediate mode GUIs and found them far easier than retained mode, since everything is redrawn every frame.
That's fair. I don't have experience with other immediate mode libraries. It's good to hear that it's not an intrinsic limitation
https://github.com/emilk/egui?tab=readme-ov-file#layout Here the author discusses the issue directly. They note that there are solutions to the issue, but that they all come with (in their opinion) significant drawbacks.
For my use case, if I have to do a lot of manual work to achieve what I consider behavior that should be handled by the framework, then I don't find that compelling and am inclined to use a retained mode implementation.
https://github.com/emilk/egui?tab=readme-ov-file#layout Here the author discusses the issue directly. They note that there are solutions to the issue, but that they all come with (in their opinion) significant drawbacks.
For my use case, if I have to do a lot of manual work to achieve what I consider behavior that should be handled by the framework, then I don't find that compelling and am inclined to use a retained mode implementation.
Could you link to your code? I am interested in the API design of meshing a layout system with immediate mode GUIs
I'm using egui to build an app with a mobile ui and I'm really enjoying it so far. The main reason I chose egui is because I need tight integration with wgpu and this is really seamless with egui.
In the process of building my app I have also created a couple of crates for egui that add drag and drop sorting, infinite scroll and other utilities.
In the example showcasing my crates I also try to show that you can make a pretty ui with complex layouts using egui (check the gallery and chat example): https://lucasmerlin.github.io/hello_egui/
I've had to spend a lot of time improving egui and it's ecosystem in the process of building my app but it seems to be worth it.
If you're not building a graphical app it probably makes more sense to use something like tauri or flutter as the gui to build a cross platform app with rust, at least until it's gui ecosystem matures.
In the process of building my app I have also created a couple of crates for egui that add drag and drop sorting, infinite scroll and other utilities.
In the example showcasing my crates I also try to show that you can make a pretty ui with complex layouts using egui (check the gallery and chat example): https://lucasmerlin.github.io/hello_egui/
I've had to spend a lot of time improving egui and it's ecosystem in the process of building my app but it seems to be worth it.
If you're not building a graphical app it probably makes more sense to use something like tauri or flutter as the gui to build a cross platform app with rust, at least until it's gui ecosystem matures.
I've dabbled with egui, and ran into this limitation almost immediately (har har).
I got around it by storing widget size between frames so that I could center it properly on the next frame. Not perfect, but it worked.
I got around it by storing widget size between frames so that I could center it properly on the next frame. Not perfect, but it worked.
None of these limitations have anything to do with an imgui frontend api though.
Can you elaborate? I'm not sure I understand. To me, these limitations feel intrinsic to immediate mode.
In Dear ImGui for instance, you can get the size of view containers and go as far as placing your UI elements at absolute positions (and also use a lower-level ImDrawList for custom rendering - which is also how you can extend Dear ImGui with your own custom-rendered UI elements, and all that absolute positioning and custom drawing is compatible with the automatic layout system).
The common misconception is that immediate mode UIs don't persist state between frames, but they absolutely do (it's just hidden from the API user). The 'immediate mode idea' is only about the public API, not the internal implementation (e.g. internally there could even be a traditional retained mode widget tree that's only modified, but not rebuilt each frame).
The common misconception is that immediate mode UIs don't persist state between frames, but they absolutely do (it's just hidden from the API user). The 'immediate mode idea' is only about the public API, not the internal implementation (e.g. internally there could even be a traditional retained mode widget tree that's only modified, but not rebuilt each frame).
Err.. but I'm not talking about placing entities at absolute positions. egui supports that - that's how we get one widget in the center of the screen :)
I'm talking about having two labels in the center of the screen, making them full width, making the text word wrap, and having each label flow properly when resizing the window. That requires calculating word wrap for each label plus knowing where labels visually higher on the y-axis have decided to place themselves after considering word wrapping. Labels beneath other labels should get pushed downward when word wrapping occurs.
That seems really hard to achieve in immediate mode libraries without effectively recreating retained mode functionality yourself.
I'm talking about having two labels in the center of the screen, making them full width, making the text word wrap, and having each label flow properly when resizing the window. That requires calculating word wrap for each label plus knowing where labels visually higher on the y-axis have decided to place themselves after considering word wrapping. Labels beneath other labels should get pushed downward when word wrapping occurs.
That seems really hard to achieve in immediate mode libraries without effectively recreating retained mode functionality yourself.
A layout engine is required in both cases. It's not necessarily more difficult in immediate mode than retained mode.
TFA demonstrates this exact functionality without "recreating retained mode". Check out the "Layout" section and note that the button is shrunk to the label, all of which is centered. The layout engine becomes an emergent property of the widgets rather than an algorithm evaluating a tree. Here, the code is the tree.
TFA demonstrates this exact functionality without "recreating retained mode". Check out the "Layout" section and note that the button is shrunk to the label, all of which is centered. The layout engine becomes an emergent property of the widgets rather than an algorithm evaluating a tree. Here, the code is the tree.
> That seems really hard to achieve in immediate mode libraries without effectively recreating retained mode functionality yourself.
The purpose of an IMGUI is to handle whatever data retention will simplify user's life. If you want to extend the IMGUI in some case it's perfectly adequate to do some that work yourself. Ideally the IMGUI infrastructure would make it as painless as possible.
The purpose of an IMGUI is to handle whatever data retention will simplify user's life. If you want to extend the IMGUI in some case it's perfectly adequate to do some that work yourself. Ideally the IMGUI infrastructure would make it as painless as possible.
It's best used for developer tools or simple UIs that don't have complex layout constraints.
For what it's worth, I'm building all of my game's UI using a pseudo-imgui framework, but I've had to do manual layout in specific places, and I updated the layout engine to run a second pass in specific cases (dynamically sized elements, primarily when text is being auto-wrapped to fit available space). This sort of stuff is only possible when you control things end-to-end.
In practice even these IMGUI frameworks don't generally do their layout in "one pass", it just appears to be a single pass to you. oui and its derivative layout.h both do a size calculation pass and then an arranging pass, for example. I originally used layout.h's algorithm, but eventually designed a new one which operates in ~3 passes:
* 1. Calculate minimum sizes and, for containers with multiple rows/columns, construct 'runs' of sequential boxes in a given row/column.
* 2a. For containers with wrapping enabled, scan over runs and when we find one that's too big for its container's available space, split controls from one run into a new one.
* 2b. For containers with children, scan through their children and grow any children that are meant to expand into available space. (You can't do this until you've measured everything and wrapped.)
* 2c. For any containers where you wrapped controls or expanded controls, recalculate their size and throw out the result from 1, since wrapping/expanding potentially changes their size.
* 2d. For simplicity you can introduce an alternative version of pass 2 for 'grid layout', where children all have fixed sizes and predictable wrapping. I've only started doing this recently, but it's great for things like listboxes and dataviews. If you do this, you don't need to do the multiple subpasses of 2a/2b/2c, and arranging becomes easier.
* 3. Now that you've measured everything and distributed boxes into rows/columns, you can scan through each 'run' and put every control into its place, applying things like centering and RTL/LTR modes.
I do think "it should be possible to efficiently perform layout for your whole UI from scratch every frame" is a good principle, it pressures you to architect your application in a cleaner, less fragile way. But sometimes you really want to retain stuff, like shaped glyphs for big blocks of unicode text, etc. Right now my game runs over 120FPS on a terrible laptop from 2015 and around 800FPS on my 3-year-old workstation, but a major portion of the CPU time is all spent doing layout. That's not great.
For what it's worth, I'm building all of my game's UI using a pseudo-imgui framework, but I've had to do manual layout in specific places, and I updated the layout engine to run a second pass in specific cases (dynamically sized elements, primarily when text is being auto-wrapped to fit available space). This sort of stuff is only possible when you control things end-to-end.
In practice even these IMGUI frameworks don't generally do their layout in "one pass", it just appears to be a single pass to you. oui and its derivative layout.h both do a size calculation pass and then an arranging pass, for example. I originally used layout.h's algorithm, but eventually designed a new one which operates in ~3 passes:
* 1. Calculate minimum sizes and, for containers with multiple rows/columns, construct 'runs' of sequential boxes in a given row/column.
* 2a. For containers with wrapping enabled, scan over runs and when we find one that's too big for its container's available space, split controls from one run into a new one.
* 2b. For containers with children, scan through their children and grow any children that are meant to expand into available space. (You can't do this until you've measured everything and wrapped.)
* 2c. For any containers where you wrapped controls or expanded controls, recalculate their size and throw out the result from 1, since wrapping/expanding potentially changes their size.
* 2d. For simplicity you can introduce an alternative version of pass 2 for 'grid layout', where children all have fixed sizes and predictable wrapping. I've only started doing this recently, but it's great for things like listboxes and dataviews. If you do this, you don't need to do the multiple subpasses of 2a/2b/2c, and arranging becomes easier.
* 3. Now that you've measured everything and distributed boxes into rows/columns, you can scan through each 'run' and put every control into its place, applying things like centering and RTL/LTR modes.
I do think "it should be possible to efficiently perform layout for your whole UI from scratch every frame" is a good principle, it pressures you to architect your application in a cleaner, less fragile way. But sometimes you really want to retain stuff, like shaped glyphs for big blocks of unicode text, etc. Right now my game runs over 120FPS on a terrible laptop from 2015 and around 800FPS on my 3-year-old workstation, but a major portion of the CPU time is all spent doing layout. That's not great.
Did you profile the layout code? How many UI elements do you display normally? And what is the O complexity of the layout algorithm above? My intuition is that even if it looks like a lot of code, it should be incredibly fast for at least hundreds of elements.
I profile aggressively using Superluminal. All of the passes are O(N), it's mostly an issue of the amount of time it takes to go through and lay out a few thousand boxes with constraints and configuration flags set. There aren't many 'bottlenecks' and it's more just a bunch of CPU time spread across the whole algorithm.
Have you looked into slint? I'm curious to hear other Rust devs' experience with it.
I haven't. I was just searching for a GUI library that was Bevy-compatible and slint isn't at the moment: https://github.com/slint-ui/slint/discussions/940
Sorry!
Sorry!
Looks like slint aims for a native look and feel, which probably wouldn't match Bevy's goals.
[deleted]
There was a post long time ago here on HN about a PhD that discovered immediate mode gui would be more efficient than the existing paradigm, but that ship has sailed
I don't think primitive layout is necessarily a limitation of the immediate mode GUI paradigm. It just requires layout to be deferred until the end of the frame. And of course to do that performantly, you'd likely need some caching between frames.
But here's the kicker: egui already does cache things between frames—for accessibility support, it already builds a full retained widget tree! From there it's not a huge jump to cache layout too. I really wish someone would experiment with this idea. Maybe Gio will be the ones to do it.
But here's the kicker: egui already does cache things between frames—for accessibility support, it already builds a full retained widget tree! From there it's not a huge jump to cache layout too. I really wish someone would experiment with this idea. Maybe Gio will be the ones to do it.
The baseline difference seems to be the difference between organizing GUI elements as stored method+field entities, vs. dynamically combined function+argument entities.
It is a lot like the difference between greedy and lazy execution.
Which code style is best depends on the structure of the items it is applied to.
--
With 100 different views, respectively over 100 different items to be viewed, storing 100 view+item pairs as objects is both general and efficient.
With 10 different views, repeatedly over 10 different items to be viewed, there are still 100 interface elements. But repeatedly joining view and item information at draw time provides a 10x space/object savings. And allows for other code simplifications.
--
General takeaway: Pre-compiled general libraries, that cannot be treated as inlined compile-time optimizable templates (in the code context where the templates are applied), will be inefficient and overbuilt for cases with more structure and simplicity than they were designed for.
It is a lot like the difference between greedy and lazy execution.
Which code style is best depends on the structure of the items it is applied to.
--
With 100 different views, respectively over 100 different items to be viewed, storing 100 view+item pairs as objects is both general and efficient.
With 10 different views, repeatedly over 10 different items to be viewed, there are still 100 interface elements. But repeatedly joining view and item information at draw time provides a 10x space/object savings. And allows for other code simplifications.
--
General takeaway: Pre-compiled general libraries, that cannot be treated as inlined compile-time optimizable templates (in the code context where the templates are applied), will be inefficient and overbuilt for cases with more structure and simplicity than they were designed for.
I see it as the crossover between approaches amenable to static order and iteration, vs those that demand constraint optimization.
This is a recurring problem in programming, because if you code towards the optimal result, you end up making a static, linearized computation that doesn't need further configuration. But if your goal is to provide interfaces and automation, you are tasked with a constraint problem, where you have multiple potential solutions and you either have to filter them down to a single unambiguous result, or find a heuristic that defines the "best one".
The problem occurs with type systems, graphical layout, multi-body physics simulation, dependency management and a host of other things. I consider it the most unaddressed subject in CS because it's so relevant to applications and yet industry continually reinvents it in terms of a bespoken algorithm for that one application.
And depending on what you're doing, you end up biasing to one or the other method first: a small, well-defined problem only needs the computer to do "brute" things, while a problem dependent on the computer managing the complexity of the problem needs it to find and correct errors.
This is a recurring problem in programming, because if you code towards the optimal result, you end up making a static, linearized computation that doesn't need further configuration. But if your goal is to provide interfaces and automation, you are tasked with a constraint problem, where you have multiple potential solutions and you either have to filter them down to a single unambiguous result, or find a heuristic that defines the "best one".
The problem occurs with type systems, graphical layout, multi-body physics simulation, dependency management and a host of other things. I consider it the most unaddressed subject in CS because it's so relevant to applications and yet industry continually reinvents it in terms of a bespoken algorithm for that one application.
And depending on what you're doing, you end up biasing to one or the other method first: a small, well-defined problem only needs the computer to do "brute" things, while a problem dependent on the computer managing the complexity of the problem needs it to find and correct errors.
Immediate mode GUI is fine for quick and dirty things, but once you start dealing with multiple application views that conditionally show/hide, with branching flows of any kind, you'll find yourself on a hardline track to reinvent retained mode GUI from scratch just so you can handle the event loop in a sane way.
Object oriented, event driven widgets composed into more colored views are really, really good for developing UI. There's a reason every major OS GUI toolkit is this design. If you componentize like you are supposed to--instead of just smashing everything into one form--they work and can be reasoned over with few surprises.
The trouble comes from not componentizing things that you should when your current platform doesn't provide the full menagerie of widgets that you'll need. Basically, it you have any kind of input that results in a value that is not just a raw string, you should be building a component for that input: URLs, paths to files, numbers, dates, selectors for picking a set list of structured objects, etc.
Object oriented, event driven widgets composed into more colored views are really, really good for developing UI. There's a reason every major OS GUI toolkit is this design. If you componentize like you are supposed to--instead of just smashing everything into one form--they work and can be reasoned over with few surprises.
The trouble comes from not componentizing things that you should when your current platform doesn't provide the full menagerie of widgets that you'll need. Basically, it you have any kind of input that results in a value that is not just a raw string, you should be building a component for that input: URLs, paths to files, numbers, dates, selectors for picking a set list of structured objects, etc.
I'm sorry, but I've got to call this out as woefully inaccurate.
> [...] so you can handle the event loop in a sane way.
The whole point of an immediate-mode framework is that the call-stack acts as the event loop. You run through your UI code, and if a thing was interacted with on the previous frame, you just handle it. What about that setup is not sane?
> [...] instead of just smashing everything into one form [...]
There is nothing about immediate mode that makes it more or less able to have everything 'smashed into one form'. You can do that just as well with your retained-mode framework of choice.
> The trouble comes from not componentizing things
This also has nothing to do with the immediate vs. retained mode discussion. You could just as easily make a giant mess in retained-mode by not making reusable components.
> once you start dealing with multiple application views that conditionally show/hide, with branching flows of any kind [..]
In my spare time, I work on a game engine whose editor/debug UI is completely immediate mode. There are discrete views into nearly everything in the engine. Entity data, a color picker, memory and CPU performance views, asset viewers/pickers, a terrain editor with like 10 modes.. you name it. There are plenty of conditionally shown views. There is plenty of 'componentizing' of things going on. The editor UI is in the neighborhood of 10k lines, most of which is generated. If that's below your bar for quick-n-dirty, maybe we have different opinions on what qualifies.
> [...] so you can handle the event loop in a sane way.
The whole point of an immediate-mode framework is that the call-stack acts as the event loop. You run through your UI code, and if a thing was interacted with on the previous frame, you just handle it. What about that setup is not sane?
> [...] instead of just smashing everything into one form [...]
There is nothing about immediate mode that makes it more or less able to have everything 'smashed into one form'. You can do that just as well with your retained-mode framework of choice.
> The trouble comes from not componentizing things
This also has nothing to do with the immediate vs. retained mode discussion. You could just as easily make a giant mess in retained-mode by not making reusable components.
> once you start dealing with multiple application views that conditionally show/hide, with branching flows of any kind [..]
In my spare time, I work on a game engine whose editor/debug UI is completely immediate mode. There are discrete views into nearly everything in the engine. Entity data, a color picker, memory and CPU performance views, asset viewers/pickers, a terrain editor with like 10 modes.. you name it. There are plenty of conditionally shown views. There is plenty of 'componentizing' of things going on. The editor UI is in the neighborhood of 10k lines, most of which is generated. If that's below your bar for quick-n-dirty, maybe we have different opinions on what qualifies.
> You could just as easily make a giant mess in retained-mode by not making reusable components.
Classic example: the Swing library for Java has layout managers, which help panels decide where their children go, and the most "powerful" one is the GridBagLayout, roughly equivalent to HTML tables with extra features and a favorite of GUI builders.
It can be used to make an unmaintainable monster layout of most of a complex window in a single panel, to structure a trivial reusable component that would be easier to write with a simpler layout manager, or to try and design intermediate level components that are self-contained and benefit from general layout configuration, with no guidance at all from the library or from tools.
Classic example: the Swing library for Java has layout managers, which help panels decide where their children go, and the most "powerful" one is the GridBagLayout, roughly equivalent to HTML tables with extra features and a favorite of GUI builders.
It can be used to make an unmaintainable monster layout of most of a complex window in a single panel, to structure a trivial reusable component that would be easier to write with a simpler layout manager, or to try and design intermediate level components that are self-contained and benefit from general layout configuration, with no guidance at all from the library or from tools.
>> The editor UI is in the neighborhood of 10k lines, most of which is generated.
I mean, this sounds like you've created a retained mode DSL that gets compiled into immediate mode.
And yes, 10 KLoC is definitely a small project, especially when "most of which is generated".
I mean, this sounds like you've created a retained mode DSL that gets compiled into immediate mode.
And yes, 10 KLoC is definitely a small project, especially when "most of which is generated".
> this sounds like you've created a retained mode DSL that gets compiled into immediate mode
Nope, I wrote a template-metaprogramming language, which generates the UI. There's nothing retained-mode about it.
The source code:
https://github.com/scallyw4g/bonsai/blob/672a3520d7537ef4e75...
The result: https://github.com/scallyw4g/bonsai/blob/master/generated/do...
> And yes, 10 KLoC is definitely a small project
I said it cleared the bar for quick-n-dirty. I agree it's not a large amount of code.
Nope, I wrote a template-metaprogramming language, which generates the UI. There's nothing retained-mode about it.
The source code:
https://github.com/scallyw4g/bonsai/blob/672a3520d7537ef4e75...
The result: https://github.com/scallyw4g/bonsai/blob/master/generated/do...
> And yes, 10 KLoC is definitely a small project
I said it cleared the bar for quick-n-dirty. I agree it's not a large amount of code.
Isn't the whole point of immediate mode UIs to get rid of the "event loop" though?
> multiple application views that conditionally show/hide
The ImGui way of doing this is to conditionally run or not run the code which describes the conditionally shown UI elements. E.g. a simple
> The trouble comes from not componentizing things...
In ImGui, reusable UI components are just code which describes the UI component by calling lower-level ImGui functions and which itself is a new ImGui-style function. It works surprisingly well.
> multiple application views that conditionally show/hide
The ImGui way of doing this is to conditionally run or not run the code which describes the conditionally shown UI elements. E.g. a simple
if (viewShown) {
...code which describes what the view looks like
}
> The trouble comes from not componentizing things...
In ImGui, reusable UI components are just code which describes the UI component by calling lower-level ImGui functions and which itself is a new ImGui-style function. It works surprisingly well.
If your entire system ran with immediate mode GUIs your performance would either be in the toilet or battery life would be destroyed (or both).
They’re great for games and GPU where you’re in a for a pound anyway. There are some music apps that use it and it’s horrible if you’re on a laptop. No I don’t want to consume a couple watts when I should be idling because the entire screen is being repainted doing nothing. Makes electron seem nice.
They’re great for games and GPU where you’re in a for a pound anyway. There are some music apps that use it and it’s horrible if you’re on a laptop. No I don’t want to consume a couple watts when I should be idling because the entire screen is being repainted doing nothing. Makes electron seem nice.
I'm not sure the perf/battery life tradeoff is a necessary aspect of immediate mode UI.
IMGUI is just a different API design IMO.
while (running) {
event := get_next_event()
process_event(event)
repaint()
}
IMGUI is just a different API design IMO.
> You could even have your UI report "interactive" rectangles to the event layer to prevent it from producing e.g. mouse move events that are irrelevant.
And then what?
With this one innocuous seeming sentence you are hand waving a way a ton of complexity. If you try to implement it you will at least have some respect for toolkit authors.
> IMGUI is just a different API design IMO.
Yes one that is lower level and therefore is fundamentally harder to deal with to get all the things people take for granted in a full blown mature state based GUI toolkit.
You can implement a retained GUI toolkit with IMGUI. Or you can use something already done.
IMGUI eschews this complexity as part of its purpose but it comes with trade offs that limit its use cases.
And then what?
With this one innocuous seeming sentence you are hand waving a way a ton of complexity. If you try to implement it you will at least have some respect for toolkit authors.
> IMGUI is just a different API design IMO.
Yes one that is lower level and therefore is fundamentally harder to deal with to get all the things people take for granted in a full blown mature state based GUI toolkit.
You can implement a retained GUI toolkit with IMGUI. Or you can use something already done.
IMGUI eschews this complexity as part of its purpose but it comes with trade offs that limit its use cases.
I decided to implement it (IMGUI that only repaints when relevant interactions occur) so that my point comes across more clearly:
https://github.com/goodpaul6/imgui-power-saving-example
Of course this is a greatly simplified example, but I can see this extending to any GUI widget that can be represented with a (hierarchy) of rectangles.
https://github.com/goodpaul6/imgui-power-saving-example
Of course this is a greatly simplified example, but I can see this extending to any GUI widget that can be represented with a (hierarchy) of rectangles.
All major GUI toolkits are ultimately implemented on some immediate mode drawing system. So I don’t see the point. (You’re literally just unfactoring what a GUI library does behind the scenes into your loop.)
When you unsimplify your example then you are reinventing a wheel. A big wheel. It can be done, but it’s a lot of work. And then you might go factoring things back so it starts to look very similar to prior art. It’s actually easier than ever to implement accessibility for instance (assuming you’re ok pulling in a large dependency), but show me an actual example of this done on top of imgui.
Things I never said: imgui doesn’t have uses, imgui isn’t great.
But its scope is limited.
When you unsimplify your example then you are reinventing a wheel. A big wheel. It can be done, but it’s a lot of work. And then you might go factoring things back so it starts to look very similar to prior art. It’s actually easier than ever to implement accessibility for instance (assuming you’re ok pulling in a large dependency), but show me an actual example of this done on top of imgui.
Things I never said: imgui doesn’t have uses, imgui isn’t great.
But its scope is limited.
Here's an IMGUI library that's also accessible (via AccessKit): https://github.com/emilk/egui
https://github.com/emilk/egui#cpu-usage
Fantastic - “only” 1 to 2 ms (and that’s for simple stuff). Depending on what you’re doing that’s a big chunk. Also time you could be sleeping. Also something you don’t usually have to worry about.
Fantastic - “only” 1 to 2 ms (and that’s for simple stuff). Depending on what you’re doing that’s a big chunk. Also time you could be sleeping. Also something you don’t usually have to worry about.
It looks like they only repaint when there is interaction as well (so it does sleep while nothing is happening).
However, my point with linking this library was just to demonstrate that accessibility and IMGUI are not inherently incompatible.
My point with the example I created above was that you don't have to trade away battery life in order to take advantage of the IMGUI paradigm. My secondary point was also to implement the "interactive rectangles" optimization I mentioned above (which only took a few lines of code).
While I agree with you that there are definitely tradeoffs, I don't think the aforementioned ones are necessary.
However, my point with linking this library was just to demonstrate that accessibility and IMGUI are not inherently incompatible.
My point with the example I created above was that you don't have to trade away battery life in order to take advantage of the IMGUI paradigm. My secondary point was also to implement the "interactive rectangles" optimization I mentioned above (which only took a few lines of code).
While I agree with you that there are definitely tradeoffs, I don't think the aforementioned ones are necessary.
> my point with linking this library was just to demonstrate that accessibility and IMGUI are not inherently incompatible.
Once again, all major GUI toolkits and browser are implemented on top of immediate mode graphics APIs, so you continue to beat this straw man argument. Obviously everything is ultimately immediate mode. No one was refuting those points and sorry they’re not particularly informative.
The point is you’re just rearranging where that state lives and who manages it. You’re retaining it somewhere. But your app is not the best place to manage a lot of this state (as even egui itself admits), it belongs in a well tested library.
Once you start using dirty rectangles, someone has to keep track of them. What benefit is it to me to drive when to call the apis for those details?
IMGUI has a certain elegance where your answer to that is to just say fuck it, and that works well for like a game editor. (The point about accessibility was not that it can’t be done but IMGUI rarely gets actually used where this is ever done). Because if your answer isn’t fuck it to all those things than just go get a library to it for you, which will be managing that state.
I think a lot of people pining for IMGUI really just want sane data binding. Which is understandable, but IMGUI is often throwing the baby out with the bath water.
Like honestly read that egui advantages/disadvantages. That’s a lot of cons (that first one woo) for that one pro (which is not even inherent to all retained APIs).
Once again, all major GUI toolkits and browser are implemented on top of immediate mode graphics APIs, so you continue to beat this straw man argument. Obviously everything is ultimately immediate mode. No one was refuting those points and sorry they’re not particularly informative.
The point is you’re just rearranging where that state lives and who manages it. You’re retaining it somewhere. But your app is not the best place to manage a lot of this state (as even egui itself admits), it belongs in a well tested library.
Once you start using dirty rectangles, someone has to keep track of them. What benefit is it to me to drive when to call the apis for those details?
IMGUI has a certain elegance where your answer to that is to just say fuck it, and that works well for like a game editor. (The point about accessibility was not that it can’t be done but IMGUI rarely gets actually used where this is ever done). Because if your answer isn’t fuck it to all those things than just go get a library to it for you, which will be managing that state.
I think a lot of people pining for IMGUI really just want sane data binding. Which is understandable, but IMGUI is often throwing the baby out with the bath water.
Like honestly read that egui advantages/disadvantages. That’s a lot of cons (that first one woo) for that one pro (which is not even inherent to all retained APIs).
> IMGUI is just a different API design IMO.
There are so many issues about so called "power saving mode" and several open PRs like https://github.com/ocornut/imgui/pull/5116.
There are so many issues about so called "power saving mode" and several open PRs like https://github.com/ocornut/imgui/pull/5116.
There's also the paradigm where only new information is painted.
If you do that, you have an event loop again (like in the OP), which your comment's grandparent was trying to get rid of (and your parent explained to them why they can't).
In practice there is always a loop. I believe the point flohofwoe was making is that with an immediate UI you don't have to even touch events in your immediate UI code (they are abstracted away, if needed, in a pretty clean manner, IMO), so there's no need to do anything to get a "sane event loop". I might be wrong about what's his point though.
There's a loop yes, but not an event loop - the other argument is that you can have a loop that runs every frame, and you just rerender the whole thing
Not really. To expand on my response: In practice there is always the operating system's event loop, both in retained and immediate mode. This is true for all mainstream operating systems.
I mean, that's just being overly pedantic. Surely you see a difference between these two pieces of code?
while (true) {
redraw();
}
while (true) {
waitForAndProcessNextEvent();
redraw();
}Please don't call me pedantic.
What I am explaining is that in practice you have to use the second code, because of how operating systems work. This can be handled by a windowing library (SDL, GLFW, Sokol), and the adapters for libraries like Dear Imgui, Nuklear, etc. also expect the second code.
There are ways to skip processing events in some OSs, but this is not really advised by OS writers, and in some cases you will get subpar experience, apps that don't close correctly, etc.
What I am explaining is that in practice you have to use the second code, because of how operating systems work. This can be handled by a windowing library (SDL, GLFW, Sokol), and the adapters for libraries like Dear Imgui, Nuklear, etc. also expect the second code.
There are ways to skip processing events in some OSs, but this is not really advised by OS writers, and in some cases you will get subpar experience, apps that don't close correctly, etc.
> If your entire system ran with immediate mode GUIs your performance would either be in the toilet or battery life would be destroyed (or both).
You might be surprised! Modern computers are unbelievable fast.
You might be surprised! Modern computers are unbelievable fast.
Try to mine bitcoins on battery and let me know how much it lasts.
Same thing.
Same thing.
… huh? The entire purpose of Bitcoin mining is to use every compute cycle. My entire point about about ImGui is it doesn’t take as many cycles as people seem to think.
It doesn't need to be inefficient, it depends on the implementation. Immediate mode is describing the API and usage, you can implement it such that it's retained mode behind the scenes, or only redrawing when necessary, etc.
I've written an immediate mode GUI for my projects and it's fine for performance. I've written a music player for instance which uses a negligible amount of CPU, it uses slightly more resources than Clementine, my previous music player.
I've written an immediate mode GUI for my projects and it's fine for performance. I've written a music player for instance which uses a negligible amount of CPU, it uses slightly more resources than Clementine, my previous music player.
> that it's retained mode behind the scenes, or only redrawing when necessary
And why reinvent that wheel? Then localization and accessibility while you’re at it?
> it uses slightly more resources than Clementine
That’s an odd flex, Clementine is extremely horrible for CPU usage (compared to nicer looking examples), probably not due to the GUI, but still. foobar2000 with a ton of plugins works better in wine.
And why reinvent that wheel? Then localization and accessibility while you’re at it?
> it uses slightly more resources than Clementine
That’s an odd flex, Clementine is extremely horrible for CPU usage (compared to nicer looking examples), probably not due to the GUI, but still. foobar2000 with a ton of plugins works better in wine.
[deleted]
>> Isn't the whole point of immediate UIs to get rid of the "event loop" though?
No, not at all. It's to take explicit control of the event loop. It's right there, you have a loop and the buttons still fire events, you just have to check for them on every iteration of the loop.
No, not at all. It's to take explicit control of the event loop. It's right there, you have a loop and the buttons still fire events, you just have to check for them on every iteration of the loop.
You don't have to 'install' event handlers, there are no event objects, and there's no event loop like:
How all this is implemented inside the immediate mode UI framework isn't relevant. The 'immediate mode idea' is only about the user-facing code which describes the UI, not the internal implementation (those can vastly differ, and could even manage a 'retained mode' widget tree under the hood).
while (uiEvent = uiGetNextEvent()) {
uiHandleEvent(uiEvent);
}
if (Button("Click Me!")) {
printf("Button Clicked!");
}
How all this is implemented inside the immediate mode UI framework isn't relevant. The 'immediate mode idea' is only about the user-facing code which describes the UI, not the internal implementation (those can vastly differ, and could even manage a 'retained mode' widget tree under the hood).
[deleted]
The way I think about it, GUIs come down to a few basic principles:
1. Program data is primarily hierarchical. This means that you can generally compose a view for a data item out of smaller views down to a basic set of elements (e.g., text display, combo box, etc.), and also means that you can generally ignore all data not included that view. There are some cases where this breaks down (tables really stretch it, for example), but it definitely holds for most GUIs.
2. UI also has state independent of program data. Immediate GUIs to a degree go ha-ha-there's-no-such-thing and make it somebody else's problem, but this state has to be retained somewhere. However, it's not exactly a binary choice between UI and program state--something like whether or not a checkbox is checked can go either way (I think of it as UI state because I tend to deal in batch-mode programs, where what is going to happen is the UI state will be distilled into the input to code I execute whose results will be displayed).
3. Conversion of program state to UI needs to be lazy--you don't want to forcibly map every element in a list immediately to UI widgets. The best example of this, of course, is the scrollbar.
4. Whereas program state is hierarchical, UI state is far less hierarchical in nature. That is to say, the state of a UI element may influence the display of a UI element in a completely different tree.
The difference between immediate and retained mode is less important than it might seem at first glance. If your UI is simple and largely static, hierarchical application of canned elements, then both an immediate and a retained mode interface will end up looking roughly the same in terms of code. If your UI is complex and deals with heavy and very impure state, it again doesn't matter all that much, because you're going to have to maintain that state all yourself anyways.
As a programmer who hates writing GUI code, though, all I really want is a thoroughly complete set of widgets (the ontology of UI is pretty standard, after all). I don't really care about immediate mode or retained mode, I just want a path that lets me easily render my program state with the minimum hassle. And quite frankly, it seems like the only GUI toolkits that make it as far as figuring out how to include a lazy table widget are the retained mode GUIs, which maybe suggests something about immediate mode GUIs.
1. Program data is primarily hierarchical. This means that you can generally compose a view for a data item out of smaller views down to a basic set of elements (e.g., text display, combo box, etc.), and also means that you can generally ignore all data not included that view. There are some cases where this breaks down (tables really stretch it, for example), but it definitely holds for most GUIs.
2. UI also has state independent of program data. Immediate GUIs to a degree go ha-ha-there's-no-such-thing and make it somebody else's problem, but this state has to be retained somewhere. However, it's not exactly a binary choice between UI and program state--something like whether or not a checkbox is checked can go either way (I think of it as UI state because I tend to deal in batch-mode programs, where what is going to happen is the UI state will be distilled into the input to code I execute whose results will be displayed).
3. Conversion of program state to UI needs to be lazy--you don't want to forcibly map every element in a list immediately to UI widgets. The best example of this, of course, is the scrollbar.
4. Whereas program state is hierarchical, UI state is far less hierarchical in nature. That is to say, the state of a UI element may influence the display of a UI element in a completely different tree.
The difference between immediate and retained mode is less important than it might seem at first glance. If your UI is simple and largely static, hierarchical application of canned elements, then both an immediate and a retained mode interface will end up looking roughly the same in terms of code. If your UI is complex and deals with heavy and very impure state, it again doesn't matter all that much, because you're going to have to maintain that state all yourself anyways.
As a programmer who hates writing GUI code, though, all I really want is a thoroughly complete set of widgets (the ontology of UI is pretty standard, after all). I don't really care about immediate mode or retained mode, I just want a path that lets me easily render my program state with the minimum hassle. And quite frankly, it seems like the only GUI toolkits that make it as far as figuring out how to include a lazy table widget are the retained mode GUIs, which maybe suggests something about immediate mode GUIs.
> There's a reason every major OS GUI toolkit is this design.
Ehh.. the latest os ui toolkit is uwp which lost to react native (a im gui) and it does not look that winui3 will fix that. Linux has two major gui libraries, both are probably only used by enthusiasts and not by enterprises which prefer to target react native (mostly for different reasons tough)
Drawing components won because the object oriented world of gui design sucks and because it’s hard to target multiple systems. All modern ui libs use a kind of immediate mode. It’s way easier to understand and you do not need to explain why you need to use mvvm.
Ehh.. the latest os ui toolkit is uwp which lost to react native (a im gui) and it does not look that winui3 will fix that. Linux has two major gui libraries, both are probably only used by enthusiasts and not by enterprises which prefer to target react native (mostly for different reasons tough)
Drawing components won because the object oriented world of gui design sucks and because it’s hard to target multiple systems. All modern ui libs use a kind of immediate mode. It’s way easier to understand and you do not need to explain why you need to use mvvm.
For desktop use, Qt is quite widely used pretty much everywhere. I only ever see React Native being used so a company can claim to have a "native" mobile app.
React native is an imgui? Aren't views defined with a declarative language similar but distinct from html?
yes, it is
yes, they usually are, which is irrelevant to the question of whether react is retained-mode or immediate-mode
please see https://news.ycombinator.com/item?id=39010462 for more detail
yes, they usually are, which is irrelevant to the question of whether react is retained-mode or immediate-mode
please see https://news.ycombinator.com/item?id=39010462 for more detail
Each tool is good for particular tasks it was designed for.
That's why my Sciter [1] supports both - as retained mode (DOM/CSS) as immediate mode (element.paintXXX()).
Consider the task of marking some div as resizable - with eight square handles [2]. With immediate mode drawing that task is trivial:
[1] https://sciter.com [2] https://stackoverflow.com/questions/20984181/how-do-i-make-a...
That's why my Sciter [1] supports both - as retained mode (DOM/CSS) as immediate mode (element.paintXXX()).
Consider the task of marking some div as resizable - with eight square handles [2]. With immediate mode drawing that task is trivial:
const div = ...;
// draw sizing handles:
div.paintForeground = function(gfx) {
gfx.fillColor = "#0f0";
for(let r of this.sizingHandlePlaces())
gfx.fillRect(r);
}
[1] https://sciter.com [2] https://stackoverflow.com/questions/20984181/how-do-i-make-a...
TBH from looking at the code examples it looks a bit like it's halfway stuck between a traditional object-oriented UI framework, and a "proper" immediate mode UI (for instance why is there a variable that seems to be a button object).
Yea, it's unfortunate.
I also find their layout "framework"/API to be weird and counter intuitive.
I also find their layout "framework"/API to be weird and counter intuitive.
Immediate mode is very common in game development. Super easy to reason about and debug.
I'm a former game dev and I used ImGui a lot and don't think it's used because those reasons.
It's used for quickly hacked debug tools to interleave UI and regular logic and not do a logic/view separation (as it would result in code bloat and a necessity for a refactor). You want UI code do some logic (like modifying properties of some game entity or renderer) and prefer to inline it. Lots of game code is effectively YOLO without even a single test. It's also typically guarded by IfDefs and compiled out of release versions.
But as soon as it stops being just hacky debuggers and people try to write proper tools in it, it becomes much more of a pain - people try to (poorly) emulate a retained mode in it, hold state, cache - and it becomes unreadable mess.
It's used for quickly hacked debug tools to interleave UI and regular logic and not do a logic/view separation (as it would result in code bloat and a necessity for a refactor). You want UI code do some logic (like modifying properties of some game entity or renderer) and prefer to inline it. Lots of game code is effectively YOLO without even a single test. It's also typically guarded by IfDefs and compiled out of release versions.
But as soon as it stops being just hacky debuggers and people try to write proper tools in it, it becomes much more of a pain - people try to (poorly) emulate a retained mode in it, hold state, cache - and it becomes unreadable mess.
> But as soon as it stops being just hacky debuggers and people try to write proper tools in it, it becomes much more of a pain - people try to (poorly) emulate a retained mode in it, hold state, cache - and it becomes unreadable mess.
Effectively people are hasty and don't spend the time to try doing things nicely, in particular because the first steps and debug use allow you to do quick things.
But I don't think it's a fundamental property of IMGUI or Dear ImGui that "proper tools" become particularly more of a pain. Of course it is more work to make a proper tools than hasty-debug-tools, and I can easily see how underengineering can back-fire (over-engineering it likewise).
Effectively people are hasty and don't spend the time to try doing things nicely, in particular because the first steps and debug use allow you to do quick things.
But I don't think it's a fundamental property of IMGUI or Dear ImGui that "proper tools" become particularly more of a pain. Of course it is more work to make a proper tools than hasty-debug-tools, and I can easily see how underengineering can back-fire (over-engineering it likewise).
This is such a good and clear take.
I'm web dev recently introduced to game dev. I'm curious why the two worlds have such different approaches. Another example is ECS being more prevalent in games than in web apps.
because they're doing different things.
Web page: Wait for data from server, update page to match, this usually happens at most every few seconds. (or if it's server based) Fetch data, format into html, set to browser
Game: For 10s to 1000s of objects, run some code for each one at 60 frames a second. That code is usually one or more finite state machines and/or co-routines per object (or some hacked together code that effective does the same). This code updates a bunch of state for each object, and then other code displays the current state.
They're doing different things so they take different approaches.
PS: I get the above is over simplified.
Web page: Wait for data from server, update page to match, this usually happens at most every few seconds. (or if it's server based) Fetch data, format into html, set to browser
Game: For 10s to 1000s of objects, run some code for each one at 60 frames a second. That code is usually one or more finite state machines and/or co-routines per object (or some hacked together code that effective does the same). This code updates a bunch of state for each object, and then other code displays the current state.
They're doing different things so they take different approaches.
PS: I get the above is over simplified.
react is pretty common in web dev and it's immediate-mode
ecs is just watered-down in-ram relational databases, and relational databases are also pretty popular for web apps
ecs is just watered-down in-ram relational databases, and relational databases are also pretty popular for web apps
I think react is not as immediate-mode as what you see in many immediate UI libs. In particular the canonical "let buttonPressed = drawButton("OK")", where your button state/press is actually reported back. In React events on your components have to flow back through some strategy.
This is either a nit or a completely fundamental difference that entirely changes the ask.
Immediate mode isn't _just_ about repeating the UI every frame
This is either a nit or a completely fundamental difference that entirely changes the ask.
Immediate mode isn't _just_ about repeating the UI every frame
this is a good point. more generally, no imgui is pure imgui
That's an interesting take I agree with. It's kinda interesting how lots of technologies are converging from totally different directions.
IMO there is also a few parallels between centralised event stores like Flux/Redux and ECS. Sure the data is organised completely differently (perhaps Flux can learn from ECS here) and updated differently (perhaps ECS can learn from Flux here) but the concept of centralising state is similar IMO.
IMO there is also a few parallels between centralised event stores like Flux/Redux and ECS. Sure the data is organised completely differently (perhaps Flux can learn from ECS here) and updated differently (perhaps ECS can learn from Flux here) but the concept of centralising state is similar IMO.
hmm, that's interesting!
i am a gamedev. i never encountered this until unity became popular, and even then it was always looked down on as a shitty and hacky approach to a well solved class of problem.
10 years after this, its still about the same... which is probably why an article like this has any controversy about it instead of being run-of-the-mill.
10 years after this, its still about the same... which is probably why an article like this has any controversy about it instead of being run-of-the-mill.
Absolutely everyone (including the AAA studios) uses Dear ImGui for tools these days.
Retained mode is probably more common for user-facing GUIs, though.
Retained mode is probably more common for user-facing GUIs, though.
> Absolutely everyone (including the AAA studios) uses Dear ImGui for tools these days.
I've been working in (mostly AAA) game engines and tools since the mid-2000s, largely in custom engines and i never encountered Dear Imgui, so i disagree with the "absolutely everyone". Pretty much every engine i've worked with (and didn't make myself) uses something like wxWidgets, Qt, MFC or some custom toolkit for the tools and custom stuff for in-game debugging (usually a console for keyboard use and some "page/screen" based reporting/lists/menus that are easy to navigate with a gamepad).
I do know that some game engines use it, but it isn't as universal as you think.
I've been working in (mostly AAA) game engines and tools since the mid-2000s, largely in custom engines and i never encountered Dear Imgui, so i disagree with the "absolutely everyone". Pretty much every engine i've worked with (and didn't make myself) uses something like wxWidgets, Qt, MFC or some custom toolkit for the tools and custom stuff for in-game debugging (usually a console for keyboard use and some "page/screen" based reporting/lists/menus that are easy to navigate with a gamepad).
I do know that some game engines use it, but it isn't as universal as you think.
If you look at 'development screenshots' of games in the last 5 years where some sort of inhouse debugging UI is visible, it's almost guaranteed to be Dear ImGui.
Qt, WPF, wxWidgets etc... was a good option up until 2015 or so, but since then the least painful way for writing integrated debugging UIs and standalone inhouse UI tools is Dear ImGui.
Qt, WPF, wxWidgets etc... was a good option up until 2015 or so, but since then the least painful way for writing integrated debugging UIs and standalone inhouse UI tools is Dear ImGui.
Same experience. IMO the keyword here is "integrated".
I'm showing my age here, but when game companies used to hire people to do tools, or leverage internal knowledge, it was often things like MFC in Windows... or the ones you mention. They would often run outside the game, often displaying lo-fi graphics.
The only "integrated" tool back in the day was a Quake-like console.
But eventually people started demanding tools that were more integrated, and immediate GUI was in the right place at the right time.
I'm showing my age here, but when game companies used to hire people to do tools, or leverage internal knowledge, it was often things like MFC in Windows... or the ones you mention. They would often run outside the game, often displaying lo-fi graphics.
The only "integrated" tool back in the day was a Quake-like console.
But eventually people started demanding tools that were more integrated, and immediate GUI was in the right place at the right time.
Recent and related:
Dear ImGui: Graphical User Interface library for C++ - https://news.ycombinator.com/item?id=38710818 - Dec 2023 (109 comments)
Dear ImGui: Graphical User Interface library for C++ - https://news.ycombinator.com/item?id=38710818 - Dec 2023 (109 comments)
I'm seeing "immediate mode" recently, and I hadn't encountered it before. It is synonymous with a React / Flutter / SwiftUI approach to UI to my untrained eye. Help me fill in the rest?
The Dear ImGui readme is a good starting point:
https://github.com/ocornut/imgui
...now of course Dear ImGui is a specific implementation of the immediate mode UI philosophy, but the general ideas transfer to other imguis as well.
https://github.com/ocornut/imgui
...now of course Dear ImGui is a specific implementation of the immediate mode UI philosophy, but the general ideas transfer to other imguis as well.
There is also microui, which I like[0], it's much smaller but still gets the job done.
Which I forked to work with SDL2[1], no guarantees. It's fun to hack on.
[0]https://github.com/rxi/microui
[1]https://github.com/kennethrapp/microui
Which I forked to work with SDL2[1], no guarantees. It's fun to hack on.
[0]https://github.com/rxi/microui
[1]https://github.com/kennethrapp/microui
It is not synonymous, but you are right that they are very similar concepts.
React's VDOM actually works just like immediate mode, updating it all on every "re-render" (with diffing taking care of updating only the parts of DOM that actually need an update). So it's a bit like "immediate on top of retained mode".
React's VDOM actually works just like immediate mode, updating it all on every "re-render" (with diffing taking care of updating only the parts of DOM that actually need an update). So it's a bit like "immediate on top of retained mode".
popular imgui toolkits like dear imgui (fortunately, nobody yet argues that dear imgui isn't really imgui) also actually retain some per-widget state
i would say that the key question is whether widget deletion and updating is implicit or explicit; it's a question about how the api is designed, not how it's implemented
with immediate-mode graphics like <canvas> or windows gdi, if you update the screen and forget to visit a certain rectangle, that rectangle disappears from the screen. the same thing happens in an immediate-mode gui if you are drawing a window and forget to visit a certain checkbox. both whether it appears or not, and everything about how it's drawn, are guaranteed to be up-to-date with your
with retained-mode graphics like svg or tk canvas or current opengl, if you update the screen and forget to visit a certain rectangle, by contrast, that rectangle stays exactly the same as it was before. the same thing happens in a retained-mode gui if you are drawing a window and forget to visit a certain checkbox: the checkbox is displayed in the same way it was displayed before, and it may be outdated with respect to the program data it's nominally supposed to represent
omar's explainer at https://github.com/ocornut/imgui/wiki/About-the-IMGUI-paradi... pretty much agrees with the above, though he goes into a lot more detail. similarly casey's talking-at-the-camera video in which he popularized the concept https://youtu.be/Z1qyvQsjK5Y?t=6m30s
so it should be apparent that react falls solidly on the imgui side of the line, which is why people use it
i would say that the key question is whether widget deletion and updating is implicit or explicit; it's a question about how the api is designed, not how it's implemented
with immediate-mode graphics like <canvas> or windows gdi, if you update the screen and forget to visit a certain rectangle, that rectangle disappears from the screen. the same thing happens in an immediate-mode gui if you are drawing a window and forget to visit a certain checkbox. both whether it appears or not, and everything about how it's drawn, are guaranteed to be up-to-date with your
with retained-mode graphics like svg or tk canvas or current opengl, if you update the screen and forget to visit a certain rectangle, by contrast, that rectangle stays exactly the same as it was before. the same thing happens in a retained-mode gui if you are drawing a window and forget to visit a certain checkbox: the checkbox is displayed in the same way it was displayed before, and it may be outdated with respect to the program data it's nominally supposed to represent
omar's explainer at https://github.com/ocornut/imgui/wiki/About-the-IMGUI-paradi... pretty much agrees with the above, though he goes into a lot more detail. similarly casey's talking-at-the-camera video in which he popularized the concept https://youtu.be/Z1qyvQsjK5Y?t=6m30s
so it should be apparent that react falls solidly on the imgui side of the line, which is why people use it
React maybe to a point but in general no.
Immediate means you get to decide (or even *forced*) how each and every frame gets drawn. Its opposite is retained GUI which means you have a set of data structures that are automagically drawn by the GUI library. Immediate GUI libraries work with functional-like types while retained GUI uses more data-oriented / object oriented types. Immediate mode is imperative, retained mode is declarative.
Immediate can offer very low level control and easier combination of various drawing interfaces. Wanna put a button on top of your game canvas? It is basically inserting more code / function calls that just draws the thing in an event loop. However you're again responsible for parsing input (helper functions exist of course) and directing things to correct place yourself.
In retained GUI you define you need a canvas such and such place and a button with red borders and a click() callback that gets called automatically. The GUI framework does all the routing and drawing for you. If it doesn't support putting a button over a canvas, you have to do 5x work to customize it and add a new component type.
Immediate mode seems the way to go isn't it? But now you have a huge event loop that you have to split and organize. It usually takes weeks to onboard newbies and you have to write everything yourself (with the help of the libraries). If you mess up the organization, you'll have to search what exact bit of code draws what. Refreshing the UI and looks will require many manual modifications rather than a simple stylesheet change. And immediate GUI has no caching implemented. You are responsible for implementing any such thing. Otherwise you'll learn about the thermal and power limits of the system very quickly.
At its very core all GUI is immediate. Because a GPU is still a processor. A very special one that's optimized for doing mostly branchless and mostly simple arithmetic but still a processor with a machine code and memory. So all retained GUI libraries have a immediate drawing core.
Immediate means you get to decide (or even *forced*) how each and every frame gets drawn. Its opposite is retained GUI which means you have a set of data structures that are automagically drawn by the GUI library. Immediate GUI libraries work with functional-like types while retained GUI uses more data-oriented / object oriented types. Immediate mode is imperative, retained mode is declarative.
Immediate can offer very low level control and easier combination of various drawing interfaces. Wanna put a button on top of your game canvas? It is basically inserting more code / function calls that just draws the thing in an event loop. However you're again responsible for parsing input (helper functions exist of course) and directing things to correct place yourself.
In retained GUI you define you need a canvas such and such place and a button with red borders and a click() callback that gets called automatically. The GUI framework does all the routing and drawing for you. If it doesn't support putting a button over a canvas, you have to do 5x work to customize it and add a new component type.
Immediate mode seems the way to go isn't it? But now you have a huge event loop that you have to split and organize. It usually takes weeks to onboard newbies and you have to write everything yourself (with the help of the libraries). If you mess up the organization, you'll have to search what exact bit of code draws what. Refreshing the UI and looks will require many manual modifications rather than a simple stylesheet change. And immediate GUI has no caching implemented. You are responsible for implementing any such thing. Otherwise you'll learn about the thermal and power limits of the system very quickly.
At its very core all GUI is immediate. Because a GPU is still a processor. A very special one that's optimized for doing mostly branchless and mostly simple arithmetic but still a processor with a machine code and memory. So all retained GUI libraries have a immediate drawing core.
Immediate mode is a fuzzy concept, as witnessed by this writeup: https://github.com/ocornut/imgui/wiki/About-the-IMGUI-paradi...
Immediate mode means instead of changing the state of UI widgets individually, the entire UI is redrawn with the current state. Afaik this has slightly worse performance but can simplify the code because the UI itself doesn't hold any state.
> because the UI itself doesn't hold any state.
...which isn't actually true in most immediate mode UI frameworks. They absolutely do persist state between frames. That state is just on the other side of the API invisible from the API user.
'Immediate mode UI' is only an API design philosophy, it says nothing about what happens under the hood.
...which isn't actually true in most immediate mode UI frameworks. They absolutely do persist state between frames. That state is just on the other side of the API invisible from the API user.
'Immediate mode UI' is only an API design philosophy, it says nothing about what happens under the hood.
> 'Immediate mode UI' is only an API design philosophy, it says nothing about what happens under the hood.
Well, if you have a traditional immediate mode API like:
Well, if you have a traditional immediate mode API like:
if (button("Start"))
start();
Which kinda makes sense, because a non-existing button usually isn't clickable, and also wouldn't be able to emit events in an event-driven API ;)
Btw, the drawing doesn't happen in the button function, it just tells the UI system that there exists a button with the label "Start" in the current frame. The rendering usually happens much later after the UI system has collected the entire UI description for the frame, otherwise rendering would be very inefficient.
Btw, the drawing doesn't happen in the button function, it just tells the UI system that there exists a button with the label "Start" in the current frame. The rendering usually happens much later after the UI system has collected the entire UI description for the frame, otherwise rendering would be very inefficient.
In graphics programming, immediate mode is the legacy, manual way of drawing.
You'd have code that would "draw thing 1", move, then "draw thing 2".
Modern pipelines instead have you upload your vertex data to the GPU and write shader code that tells the system how to draw it. They become managed by the GPU and your code cares less about explicit drawing.
Another way to look at this is that "immediate mode" feels much more imperative than the modern asynchronous graphics pipeline. You tell the system to draw or render something, and it immediately does so.
This post has another good explanation with simple code:
https://stackoverflow.com/questions/6733934/what-does-immedi...
It's much easier to wrap your head around immediate mode though, and several tools with this imperative/immediate philosophy (such as imgui) are popular.
Edit:
Here's a fantastic comparison: https://cognitivewaves.wordpress.com/opengl-vbo-shader-vao/
(See "Immediate" vs the more modern code that follows.)
You'd have code that would "draw thing 1", move, then "draw thing 2".
Modern pipelines instead have you upload your vertex data to the GPU and write shader code that tells the system how to draw it. They become managed by the GPU and your code cares less about explicit drawing.
Another way to look at this is that "immediate mode" feels much more imperative than the modern asynchronous graphics pipeline. You tell the system to draw or render something, and it immediately does so.
This post has another good explanation with simple code:
https://stackoverflow.com/questions/6733934/what-does-immedi...
It's much easier to wrap your head around immediate mode though, and several tools with this imperative/immediate philosophy (such as imgui) are popular.
Edit:
Here's a fantastic comparison: https://cognitivewaves.wordpress.com/opengl-vbo-shader-vao/
(See "Immediate" vs the more modern code that follows.)
Any decent immediate mode framework, including Dear ImGUI, has backends which just send quads to the GPU. The code is imperative but it just winds up filling a vertex buffer.
Despite both having "immediate mode" in the name, immediate mode OpenGL and immediate mode GUIs share very little in common. Pros/cons definitely don't carry over at all.
Apart from most of your comment being irrelevant for immediate mode UIs, the rest is also at best misleading, since in modern 3D-APIs draw commands are also issued from scratch each frame (if only to kick off GPU shader code).
The very early D3D versions actually had an optional 'retained mode API' in the mid-90's, but that quickly went the way of the Dodo.
The very early D3D versions actually had an optional 'retained mode API' in the mid-90's, but that quickly went the way of the Dodo.
Casey Muratori of handmade hero fame is one of the original pioneers (potentially the first?) of immediate mode user-interfaces.
Check out this lecture/demo from nearly 20 years ago: https://www.youtube.com/watch?v=Z1qyvQsjK5Y
Check out this lecture/demo from nearly 20 years ago: https://www.youtube.com/watch?v=Z1qyvQsjK5Y
For what it's worth, early user interface libraries existed on a spectrum from what we'd now call retained-mode to immediate-mode.
For example, the original Macintosh Toolbox (1984) worked _somewhat_ like that: on one hand, you had to handle events – even to the extent that you were just given a mouse click event, and had to dispatch it to the menu, scrollbars, title bars, etc; you then had to manually draw controls (but not while explicitly passing in their state.)
On the other hand, control state was "retained", to the extent that you updated their values, displayed them etc. with handles, and their state existed off in a linked list somewhere.
Here's how a later version of Inside Macintosh described having to handle a scroll bar operation, once a click and its location was identified. (I present it because it's long and aggravating...)
For example, the original Macintosh Toolbox (1984) worked _somewhat_ like that: on one hand, you had to handle events – even to the extent that you were just given a mouse click event, and had to dispatch it to the menu, scrollbars, title bars, etc; you then had to manually draw controls (but not while explicitly passing in their state.)
On the other hand, control state was "retained", to the extent that you updated their values, displayed them etc. with handles, and their state existed off in a linked list somewhere.
Here's how a later version of Inside Macintosh described having to handle a scroll bar operation, once a click and its location was identified. (I present it because it's long and aggravating...)
1. Use the FindControl, GetControlValue, and TrackControl functions to help calculate the scrolling distance.
2. If you are scrolling for any reason other than the user dragging the scroll box, use the SetControlValue procedure to move the scroll box a corresponding amount.
3. Use a routine—such as the QuickDraw procedure ScrollRect or the TextEdit procedure TEPinScroll—to move the bits displayed in the window by the calculated scrolling distance. Then either use a call that generates an update event or else directly call your application’s DoUpdate routine, which should perform the rest of these steps.
4. Use the UpdateControls procedure to update the scroll bars and then call the Window Manager procedure DrawGrowIcon to redraw the size box.
5. Use the QuickDraw procedure SetOrigin to change the window origin by an amount equal to the scroll bar settings so that the upper-left corner of the document lies at (0,0) in the window’s local coordinate system. (You perform this step so that your application’s document-drawing routines can draw in the correct area of the window.)
6. Call your application’s routines for redrawing the document inside the window.
7. Use the SetOrigin procedure to reset the window origin to (0,0) so that future Window Manager and Control Manager routines draw in the correct area of the window.
8. Return to your event loop.
These steps are explained in greater detail in the rest of this section.Games and terminals have existed before that
2003 https://perso.univ-lyon1.fr/thierry.excoffier/ZMW/
2003 https://perso.univ-lyon1.fr/thierry.excoffier/ZMW/
Is Streamlit an immediate mode UI framework?
It has a very unique, easy-to-start, model for creating UIs which avoids callbacks, but I don't know what the technical term for it is.
It has a very unique, easy-to-start, model for creating UIs which avoids callbacks, but I don't know what the technical term for it is.
yeah, i would call streamlit an imgui framework. also react
i tire of IM vs retained discussions. the difference is academic up-to tools and code-gen in my experience. you can make one with the other up-to a point of usability. the classic point of split i find is if a UI designer wants to insert intermediate "for-show" elements, e.g. for animation. in that space imgui is, in practice, just going to disable your ui designers vs a retained mode solution using well established tools for defining and animating ui in standard ways. sure you can undo that problem, but thats code work, code work that blocks your ui people from doing their job until its done, and more points of failure for bugs to creep in...
the arguments i hear about debuggability seem brain-dead, but on reflection that might be the utility here. "make things easier to debug, shittier programmers can do work"... and that is not to be taken lightly or dismissively when it has real world weight.
retained mode most often exists in the backend, e.g. Win32, although some of it may use IM style interfaces, e.g. MessageBox, and so a lot of im gui tools will still use something retained under the hood, and have to go to some lengths to be able to identify "dynamically created" things. yes it can be worked around, but its another headache and bug vector...
ive honestly never seen IM GUI as a great approach, most often i implement it due to demand, and with some frustration at the brain dead arguments levelled for it. it solves a problem of programmer burden, which is not that useful in real world projects in my experience, and since the interfaces are interchangeable, its not really a fundamental decision to be making either until someone else forces it.
EDIT: to be clear, imagine trying to debug your data-defined if-statement. really think it through and implement it.
the arguments i hear about debuggability seem brain-dead, but on reflection that might be the utility here. "make things easier to debug, shittier programmers can do work"... and that is not to be taken lightly or dismissively when it has real world weight.
retained mode most often exists in the backend, e.g. Win32, although some of it may use IM style interfaces, e.g. MessageBox, and so a lot of im gui tools will still use something retained under the hood, and have to go to some lengths to be able to identify "dynamically created" things. yes it can be worked around, but its another headache and bug vector...
ive honestly never seen IM GUI as a great approach, most often i implement it due to demand, and with some frustration at the brain dead arguments levelled for it. it solves a problem of programmer burden, which is not that useful in real world projects in my experience, and since the interfaces are interchangeable, its not really a fundamental decision to be making either until someone else forces it.
EDIT: to be clear, imagine trying to debug your data-defined if-statement. really think it through and implement it.
> the program is in control of the main event loop, user input is handled as part of program flow
Is that supposed to be a good thing? I don't want to write an event loop - I'll probably screw it up and introduce bugs.
> the program state is both minimal [...]
Um... The comparison of normal web programming and your proposed alternative shows an order of magnitude more code for the same result. And it's not even really the same result, I can't select text and right click to copy for example. What about complex things like varying the UX depending on the size of the screen and input method. Or accessibility features for someone who's blind or can't move their hands... those aren't optional.
> and cleanly separated.
Huh? You've got a function called "loop()" that contains all of the code. Where is the separation?
Is that supposed to be a good thing? I don't want to write an event loop - I'll probably screw it up and introduce bugs.
> the program state is both minimal [...]
Um... The comparison of normal web programming and your proposed alternative shows an order of magnitude more code for the same result. And it's not even really the same result, I can't select text and right click to copy for example. What about complex things like varying the UX depending on the size of the screen and input method. Or accessibility features for someone who's blind or can't move their hands... those aren't optional.
> and cleanly separated.
Huh? You've got a function called "loop()" that contains all of the code. Where is the separation?
Kinda an unfortunate name collision with GIO as used in another UI toolkit stack: https://en.wikipedia.org/wiki/GIO_(software)
"Please don't complain about tangential annoyances—e.g. article or website formats, name collisions, or back-button breakage. They're too common to be interesting."
https://news.ycombinator.com/newsguidelines.html
https://news.ycombinator.com/newsguidelines.html
I've found the immediate-mode UI EGUI in Rust to be easy to use and expressive. It doesn't look native, but the code is easy to read and write for reasons alluded to in the article.
Heh, my long weekend project has been to write an egui web application as a crutch for my first steps in Rust. I had no idea that it doesn’t look native.
immediate mode is a tool for certain jobs. drawing arbitrary stuff on screen without layouts or structure would be where I bring it in.
I tried Gio on Android several years ago, and sadly it ran at around 15fps, and the APK was around 50mb I think.
Please don't ship immediate mode GUIs to normal users. They are good for adding UI to a game or throwing together some quick internal tool, but you'll quickly run into tons of limitations, inconsistent behavior for the platform, and difficult styling.
> creating user interfaces for mobile and the desktop, avoiding the platform bound and often complex native toolkits
Especially for mobile, just use the native toolkit, or something that uses the native toolkit for its backend. I'm sick of using apps that don't use the native toolkit, they almost always have worse performance and just behave in weird ways. Even flutter is guilty of this, and that has all the manpower of a google team behind it.
A mobile app using immediate mode, written in _Go_, just sounds like a recipe for an app that will never function as well as its equivalent written specifically for the platform.
Stop sacrificing the experience of your users for your own personal developer experience.
> creating user interfaces for mobile and the desktop, avoiding the platform bound and often complex native toolkits
Especially for mobile, just use the native toolkit, or something that uses the native toolkit for its backend. I'm sick of using apps that don't use the native toolkit, they almost always have worse performance and just behave in weird ways. Even flutter is guilty of this, and that has all the manpower of a google team behind it.
A mobile app using immediate mode, written in _Go_, just sounds like a recipe for an app that will never function as well as its equivalent written specifically for the platform.
Stop sacrificing the experience of your users for your own personal developer experience.
The RemedyBG debugger (https://remedybg.handmade.network/) and the Tracy profiler (https://github.com/wolfpld/tracy) both use Dear ImGui and so far I've only read high praise from people who used those tools compared to the 'established' alternatives.
For tools like this, programmers are also just "normal users", and from the developer side, I'm sure they evaluated various alternatives with all their pros and cons before settling for Dear ImGui.
For tools like this, programmers are also just "normal users", and from the developer side, I'm sure they evaluated various alternatives with all their pros and cons before settling for Dear ImGui.
I didn't use RemedyBG or Tracy, but I did try ImHex (https://github.com/WerWolv/ImHex) and it loaded 12% of the CPU because everything is being repainted 60 times per second. Heck, it even has an option to limit the FPS, which solves the CPU load a bit, but at the same time results in sluggish input because the event handling is tied to the drawing frequency.
So yes, the experience was not good, and I don't see what these tools would lose by using a proper GUI. I don't want every utility to drain my laptop battery like a decent video game.
ImGui is great if you already have a loop where everything is unconditionally redrawn every frame, but otherwise it's a really odd choice for an end-user application.
So yes, the experience was not good, and I don't see what these tools would lose by using a proper GUI. I don't want every utility to drain my laptop battery like a decent video game.
ImGui is great if you already have a loop where everything is unconditionally redrawn every frame, but otherwise it's a really odd choice for an end-user application.
> it even has an option to limit the FPS, which solves the CPU load a bit, but at the same time results in sluggish input because the event handling is tied to the drawing frequency.
It seems like an issue of how it is implemented. I think Tracy does it well. It's party my fault since Dear ImGui lib and backends currently doesn't have an "official" way to tackle this, so everyone does their own sauce and it's easy to make a mistake. But I have zero doubt it is doable.
> I don't see what these tools would lose by using a proper GUI.
What they would lose is that they wouldn't exist in the first place or wouldn't be as full-featured. I'm surprised this is so hard to comprehend? In spite of its shortcomings, software like Dear ImGui is an enabler to make things exists and happen.
It seems like an issue of how it is implemented. I think Tracy does it well. It's party my fault since Dear ImGui lib and backends currently doesn't have an "official" way to tackle this, so everyone does their own sauce and it's easy to make a mistake. But I have zero doubt it is doable.
> I don't see what these tools would lose by using a proper GUI.
What they would lose is that they wouldn't exist in the first place or wouldn't be as full-featured. I'm surprised this is so hard to comprehend? In spite of its shortcomings, software like Dear ImGui is an enabler to make things exists and happen.
> What they would lose is that they wouldn't exist in the first place or wouldn't be as full-featured.
These are some pretty bold statements.
* "They wouldn't exist in the first place" implies that ImGui was the primary reason and foundation for creating these programs. As if using the traditional retained mode GUI is so unbearable that without ImGui the authors would have abandoned the idea of creating these tools in the first place.
* "Or wouldn't be as full-featured" implies that ImGui is either more full-featured or (if you meant time) is faster to develop with compared to a traditional retained mode GUI.
> I'm surprised this is so hard to comprehend?
Well, I'm surprised that some people keep presenting the immediate mode GUI as the silver-bullet alternative to the traditional GUI. Don't get me wrong: I understand that IMGUI is a great tool if you need to quickly add a throway GUI to a game, but otherwise there is a price to pay, both by the developer and the end user.
These are some pretty bold statements.
* "They wouldn't exist in the first place" implies that ImGui was the primary reason and foundation for creating these programs. As if using the traditional retained mode GUI is so unbearable that without ImGui the authors would have abandoned the idea of creating these tools in the first place.
* "Or wouldn't be as full-featured" implies that ImGui is either more full-featured or (if you meant time) is faster to develop with compared to a traditional retained mode GUI.
> I'm surprised this is so hard to comprehend?
Well, I'm surprised that some people keep presenting the immediate mode GUI as the silver-bullet alternative to the traditional GUI. Don't get me wrong: I understand that IMGUI is a great tool if you need to quickly add a throway GUI to a game, but otherwise there is a price to pay, both by the developer and the end user.
I am not saying it is a silver bullet.
I'm saying it is making some development - those that are well aligned with the frameworks qualities - particularly efficient. Efficiency and productivity are everything. Productivity is often a major contributor in bridging the gap between cancelled and released, between painful and pleasant, between under-featured and full-featured, between abandoned and maintained, between unprofitable and profitable.
So while not saying things are simple to describe and compare, they are not, Dear ImGui focus on high-productivity is the reason why it has been adopted by some people.
> without ImGui the authors would have abandoned the idea of creating these tools in the first place.
It is probable those particular authors would have, yes.
I meant, it's not a secret that many engineers are totally afraid or uninterested in UI programmers. A common feedback is of certain people saying "hey, dear imgui made UI programming fun for me". So I'm confidently saying that SOME software wouldn't have existed without dear imgui. It being so brutally different in term of philosophy, coding style, culture, by definition makes it reach a different crowd.
> implies that ImGui is either more full-featured or (if you meant time) is faster to develop with compared to a traditional retained mode GUI.
Dear ImGui is clearly LESS full-featured than e.g Qt, but for some uses is is faster to develop with than most other frameworks.
I'm saying it is making some development - those that are well aligned with the frameworks qualities - particularly efficient. Efficiency and productivity are everything. Productivity is often a major contributor in bridging the gap between cancelled and released, between painful and pleasant, between under-featured and full-featured, between abandoned and maintained, between unprofitable and profitable.
So while not saying things are simple to describe and compare, they are not, Dear ImGui focus on high-productivity is the reason why it has been adopted by some people.
> without ImGui the authors would have abandoned the idea of creating these tools in the first place.
It is probable those particular authors would have, yes.
I meant, it's not a secret that many engineers are totally afraid or uninterested in UI programmers. A common feedback is of certain people saying "hey, dear imgui made UI programming fun for me". So I'm confidently saying that SOME software wouldn't have existed without dear imgui. It being so brutally different in term of philosophy, coding style, culture, by definition makes it reach a different crowd.
> implies that ImGui is either more full-featured or (if you meant time) is faster to develop with compared to a traditional retained mode GUI.
Dear ImGui is clearly LESS full-featured than e.g Qt, but for some uses is is faster to develop with than most other frameworks.
[deleted]
egui does not support putting two widgets in the center of the screen: https://github.com/emilk/egui/issues/3211
It's really easy to get started with immediate mode and bust out some simple UIs, but the second you start trying to involve dynamically sized elements and responsive layouts -- abandon all hope. The fact it has to calculate everything in a single pass makes these things hard/impossible. Coming from a strong CSS/React background I find the limitation maddening.
... that said, I'm still using it to build a prototype UI for https://ant.care/ (https://github.com/MeoMix/symbiants) because it's the best thing I've found so far.
I'm crossing my fingers that Bevy's UI story (or Kayak https://github.com/StarArawn/kayak_ui) becomes significantly more fleshed out sooner rather than later. Bevy 0.13 should have lots more in this area though (https://github.com/bevyengine/bevy/discussions/9538)