A Thought Experiment: Using the ECS Pattern Outside of Game Engines(adventures.michaelfbryan.com)
adventures.michaelfbryan.com
A Thought Experiment: Using the ECS Pattern Outside of Game Engines
http://adventures.michaelfbryan.com/posts/ecs-outside-of-games/
74 comments
Not that different from MVC a couple decades ago, then! :D (just kidding)
I love the history of ECS. Scott Bilas' goals with his work in Dugeon Siege, and before that, in Gabriel Knight 3 [1], was to have something close to what you're doing, in order to speed up development. He was storing game logic in text files, as opposite to having it hardcoded, to enable level designers and artists to iterate faster. I guess he would have considered SQLite if it was as widespread back then.
It was only later that people realised ECS could be a performance optimisation, associated it with OOP and made libraries for it.
[1] A very fun adventure game, and fun to reverse engineer. One of the "Easter Eggs" involves teaching you how to extract data from itself.
I love the history of ECS. Scott Bilas' goals with his work in Dugeon Siege, and before that, in Gabriel Knight 3 [1], was to have something close to what you're doing, in order to speed up development. He was storing game logic in text files, as opposite to having it hardcoded, to enable level designers and artists to iterate faster. I guess he would have considered SQLite if it was as widespread back then.
It was only later that people realised ECS could be a performance optimisation, associated it with OOP and made libraries for it.
[1] A very fun adventure game, and fun to reverse engineer. One of the "Easter Eggs" involves teaching you how to extract data from itself.
Didn't know that origin story, thanks!
FWIW, what I'm doing is storing all the runtime state in in-memory SQLite, instead of writing my own ECS. Components are rows in tables, entity IDs are used as foreign keys, game logic just does regular SQL SELECTs to query the game state, and then UPDATEs, INSERTs or DELETEs stuff as needed.
I came up with this experiment when I was writing yet another implementation of ECS for that game, and in the middle of optimizing it for processing performance I realized that all I'm doing is essentially hand-writing database indexes, so why not use a proper database instead? And it turns out, SQLite is fast enough when used in-memory.
FWIW, what I'm doing is storing all the runtime state in in-memory SQLite, instead of writing my own ECS. Components are rows in tables, entity IDs are used as foreign keys, game logic just does regular SQL SELECTs to query the game state, and then UPDATEs, INSERTs or DELETEs stuff as needed.
I came up with this experiment when I was writing yet another implementation of ECS for that game, and in the middle of optimizing it for processing performance I realized that all I'm doing is essentially hand-writing database indexes, so why not use a proper database instead? And it turns out, SQLite is fast enough when used in-memory.
There's a game that already did that, Sword of the Stars 2.
Widely panned at release due to being forced to release old code after VCS failure, though I'd argue the only problem after patches were left was that movement logic change didn't play out well (it was a brave attempt though).
Had some bad performance issues but I never tracked down whether they were related to SQLite or issues in their engine.
Widely panned at release due to being forced to release old code after VCS failure, though I'd argue the only problem after patches were left was that movement logic change didn't play out well (it was a brave attempt though).
Had some bad performance issues but I never tracked down whether they were related to SQLite or issues in their engine.
Well, of course there is. Here I thought I'm doing something somewhat original...
Fortunately, existence of prior art doesn't invalidate any of the goals I had for this experiment, so I'll just carry on with it.
Fortunately, existence of prior art doesn't invalidate any of the goals I had for this experiment, so I'll just carry on with it.
Of course it does not invalidate your experiment, but I wanted to suggest maybe looking at save files from the game.
That's super cool. I think that approach is quite scalable, too. Only games with very complex logic would require something faster than SQLite for game logic.
Unrelated, but I also guess that using SQLite for offline data could be a timesaver, since you can leverage web tech to make tooling for your artists and level designers.
Unrelated, but I also guess that using SQLite for offline data could be a timesaver, since you can leverage web tech to make tooling for your artists and level designers.
Last time I built an ECS it was effectively a bare bones relational database. A system just queries the information that it needs. The code basically looked like this. There was no need for explicit where or select queries because all I really needed to check was the presence of a component. Here is a contrived example. ecs has an inner join function that accepts all components (stored in an array) that you want to join and everything gets passed to a callback that receives all the components.
void autoheal_system() {
ecs.inner_join(ecs.status, ecs.inventory, ..., [](status, inventory, ...){
if(status.health < (0.2 * status.max_health) {
inventory.find("potion").use()
}
})
}
> If you go for "ECS as a performance optimization" - a frequently touted benefit - you'll end up storing a lot of global arrays (perhaps arranged in structures), each packing every instance of a component's property across all entities.
This is basically what I am doing in the backend I am writing for an app that I am working on. (Also not a game btw.)
No database, no overhead :D
You really can fit a lot of data in memory, and I think a lot of people tend to forget that.
I so so wish that I get to see the day when memristors become cheap and abundant. It would be so pleasant to program for a system like that I think. No persisting to disk or loading from disk, ever – imagine that!
This is basically what I am doing in the backend I am writing for an app that I am working on. (Also not a game btw.)
No database, no overhead :D
You really can fit a lot of data in memory, and I think a lot of people tend to forget that.
I so so wish that I get to see the day when memristors become cheap and abundant. It would be so pleasant to program for a system like that I think. No persisting to disk or loading from disk, ever – imagine that!
New NVMe drives claim 5GB/s on PCIe 4, is memory mapping something like that not good enough for you already?
In these applications you primarily care about latency. Even these new SSD's have latencies 3 orders of magniture higher than DRAM. The 20 usecs is access time is about 100k executed instructions (scalar + single thread).
In what applications? The parent didn't even say what they were working on. Everyone wants ideally wants a perfect memory technology with no latency, large sizes and persistence, so I'm not sure what you are talking about exactly. If something needs less latency, you cache it or load it into memory. You seem to be saying that this current technology, that actually exists, is not good enough for a mystery application because it isn't as good as theoretical memresistors.
[deleted]
You always need a memory hierarchy. Not even DRAM is fast enough for our CPUs so we have L3 caches and those are not fast enough either so we also have L2 and L1 caches and registers and physical wires which temporarily store the state of the processor by taking advantage of propagation delays.
I could use some critical feedback on my system [0]. I went with what felt to be the most intuitive pattern for my own brain, but later found that it was apparently against the grain: making everything plain old classes and keeping all the logic inside components.
When I need a particular behavior, I just create a new class and keep everything related to that behavior in a single file. Entities and systems are little more than arrays, and components can query their entity for other components at runtime.
So there may be a MouseEventComponent or a TouchEventComponent, added by the engine depending on your device, then a PointerEventComponent to provide a layer of abstraction for OS-independent player input, and finally some PointerControlledShootingComponent etc.
Or a KeyboardEventComponent feeding a generic DirectionEventComponent, and you can swap the keyboard for a gamepad at runtime, or have different input sources for different entities.
To my understanding, doing stuff like that did not seem as intuitive in the more "traditional" ECS patterns, as opposed to just editing a list of classes to update every frame, or components holding references to other components in other entities (e.g. to share an input stream, or to implement a self-playing demo mode.)
It's meant to be for 2D games only and I haven't hit a wall with it so far, performance or otherwise, but some things like serializing the object graph for save/load or parallel execution might be a little tricky when I get to those.
My goal is to eventually be able to describe game scenes with a declarative SwiftUI-like syntax, where you'd add components like view modifiers and so on. Come to think of it, SwiftUI might be considered as an ECS for apps, or is that stretching the definition a little too far. :)
[0] https://github.com/InvadingOctopus/octopuskit
When I need a particular behavior, I just create a new class and keep everything related to that behavior in a single file. Entities and systems are little more than arrays, and components can query their entity for other components at runtime.
So there may be a MouseEventComponent or a TouchEventComponent, added by the engine depending on your device, then a PointerEventComponent to provide a layer of abstraction for OS-independent player input, and finally some PointerControlledShootingComponent etc.
Or a KeyboardEventComponent feeding a generic DirectionEventComponent, and you can swap the keyboard for a gamepad at runtime, or have different input sources for different entities.
To my understanding, doing stuff like that did not seem as intuitive in the more "traditional" ECS patterns, as opposed to just editing a list of classes to update every frame, or components holding references to other components in other entities (e.g. to share an input stream, or to implement a self-playing demo mode.)
It's meant to be for 2D games only and I haven't hit a wall with it so far, performance or otherwise, but some things like serializing the object graph for save/load or parallel execution might be a little tricky when I get to those.
My goal is to eventually be able to describe game scenes with a declarative SwiftUI-like syntax, where you'd add components like view modifiers and so on. Come to think of it, SwiftUI might be considered as an ECS for apps, or is that stretching the definition a little too far. :)
[0] https://github.com/InvadingOctopus/octopuskit
That's an interesting take. I read the README and the architecture document at [0]. Based on that, the way I see it, your approach is different but essentially equivalent to the typical non-performance-oriented ECS architecture. Entities are still runtime-configurable, Component updates are done in batches and in appropriate order. You've thinned out Systems, moved their code to their corresponding Components, and provided places to hook component mount/unmount code.
To me, doing stuff like this seems more intuitive than more "traditional" ECS approaches, but perhaps that's because I was never comfortable with the idea of Systems.
Eyeballing the code, the only slight architectural drawback I see is that the code must always belong to a particular Component, even when that code operates on multiple Components as equal partners (this is the ECS eqiuvalent of "your language doens't have multimethods" problem in OOP). If I understood correctly, you solve that by creating a Component composing other Components.
One particular thing I like about your overall architecture is that it explicitly covers state machines - both at the game (scene) level and at per-entity level. Various other ECS examples I saw on the web tend to completely ignore this topic, and it's pretty much the first thing I hit when I start using them: where do I put state machines?
Overall, OctopusKit looks like a solid piece of work. Congratulations, and do a Show HN: if you haven't done it yet!
--
[0] - https://invadingoctopus.io/octopuskit/documentation/architec....
To me, doing stuff like this seems more intuitive than more "traditional" ECS approaches, but perhaps that's because I was never comfortable with the idea of Systems.
Eyeballing the code, the only slight architectural drawback I see is that the code must always belong to a particular Component, even when that code operates on multiple Components as equal partners (this is the ECS eqiuvalent of "your language doens't have multimethods" problem in OOP). If I understood correctly, you solve that by creating a Component composing other Components.
One particular thing I like about your overall architecture is that it explicitly covers state machines - both at the game (scene) level and at per-entity level. Various other ECS examples I saw on the web tend to completely ignore this topic, and it's pretty much the first thing I hit when I start using them: where do I put state machines?
Overall, OctopusKit looks like a solid piece of work. Congratulations, and do a Show HN: if you haven't done it yet!
--
[0] - https://invadingoctopus.io/octopuskit/documentation/architec....
Thank you!
> the ECS equivalent of "your language doesn't have multimethods" problem in OOP ... you solve that by creating a Component composing other Components.
I'm not sure about having solved that, but yes, a component may check its entity for other components to inspect and/or modify, even add/remove components from its entity, or add/remove entire entities from the scene (like a ShootingComponent asking a PhysicsComponent to simulate recoil, and spawning bullet entities.)
If a dependency is missing, a component alters or omits parts of its behavior or silently skips its frame update.
I'd really like to try the SwiftUI idiom in games, where the code is a declarative description of a scene, that an engine can take to construct the actual scene using whatever architecture it wants under the hood.
I also wonder how far you could take SwiftUI itself for making games without using any other framework.
> the ECS equivalent of "your language doesn't have multimethods" problem in OOP ... you solve that by creating a Component composing other Components.
I'm not sure about having solved that, but yes, a component may check its entity for other components to inspect and/or modify, even add/remove components from its entity, or add/remove entire entities from the scene (like a ShootingComponent asking a PhysicsComponent to simulate recoil, and spawning bullet entities.)
If a dependency is missing, a component alters or omits parts of its behavior or silently skips its frame update.
I'd really like to try the SwiftUI idiom in games, where the code is a declarative description of a scene, that an engine can take to construct the actual scene using whatever architecture it wants under the hood.
I also wonder how far you could take SwiftUI itself for making games without using any other framework.
I have gone down the exact same line of thinking. I think there is value in creating a simple relational data structure to have simple select and insert functions but not the overhead of sqlite. Many times when I think about it, the state of most programs could use a handful of the same structure and be done.
Yeah, I think so too. My goal after hopefully making a completely playable game running on SQLite is to eventually rip it out and replace with my own ECS, based on gained experience. I start with explicit SQL queries, and am slowly building an interface on top of it. Or rather, I'm moving towards the interface I abandoned prior to switching to SQL.
Before SQLite, I was working on my (yet another) own implementation, and eventually realized that I don't like the "Systems" part of the ECS, given I planned a lot of logic that required inspecting multiple components of any given entity. Thinking about it, I figured out that a better abstraction for a "System" would be a piece of code executed on a set of entities, defined by some selection logic. That is, a query. I quickly ended up with this interface:
Which, as you can see, looks essentially like an SQL query. :components is select + join, :where is filtering condition, :order-by saves me from doing an explicit sort on the results, and the :into my-array part allows this query to not allocate new storage for results each frame.
Now each time such call showed up in my code, some extra book keeping storage would be allocated, and underlying logic ensured that most of the query doesn't have to be recomputed every frame. For instance, the :where part was handled by tracking addition and deletion of components to/from an entity. Implementing similar optimizations for :order-by part, I quickly realized that most of my "book keeping" tricks are essentially equivalent to compiled queries and indices on tables. At which point I ditched it all and switched to SQLite.
(I meant to write a blog post about this at some point. I guess I just wrote half of it here.)
Before SQLite, I was working on my (yet another) own implementation, and eventually realized that I don't like the "Systems" part of the ECS, given I planned a lot of logic that required inspecting multiple components of any given entity. Thinking about it, I figured out that a better abstraction for a "System" would be a piece of code executed on a set of entities, defined by some selection logic. That is, a query. I quickly ended up with this interface:
(select-entities
:components (has-position has-health (not player))
:where ((entity-id (has-health (hit-points hp)))
(< hit-points 50))
:order-by (((lhs (has-health (lhs-hp hp)) &optional (targetable (lhs-priority priority)))
(rhs (has-health (rhs-hp hp)) &optional (targetable (rhs-priority priority))))
(if (and lhs-priority
rhs-priority
(= lhs-hp rhs-hp))
(< lhs-priority rhs-priority)
(< lhs-hp rhs-hp)))
:into my-array)
Which, as you can see, looks essentially like an SQL query. :components is select + join, :where is filtering condition, :order-by saves me from doing an explicit sort on the results, and the :into my-array part allows this query to not allocate new storage for results each frame.
Now each time such call showed up in my code, some extra book keeping storage would be allocated, and underlying logic ensured that most of the query doesn't have to be recomputed every frame. For instance, the :where part was handled by tracking addition and deletion of components to/from an entity. Implementing similar optimizations for :order-by part, I quickly realized that most of my "book keeping" tricks are essentially equivalent to compiled queries and indices on tables. At which point I ditched it all and switched to SQLite.
(I meant to write a blog post about this at some point. I guess I just wrote half of it here.)
I've never written a game where I needed to query for entities (except spatially). Have I just not created the right type of game? If I needed to find certain types of entities they'd get added/add themselves to some list on creation. Querying things at runtime seems anathema to game dev if you want perf which most game devs obsess over
> If I needed to find certain types of entities they'd get added/add themselves to some list on creation.
From the point of view of the code operating on these entities, this is a query. That it happens to return results in constant time because it's being continuously executed in the background - that's just an implementation detail.
The example I've pasted doesn't execute a full query every frame either. The first time it's called, it ensures that there exists an array within ECS system that stores entities which match the :components part of my query. That array is then continuously updated when appropriate components are added or removed from entities. Multiple different queries using the same :components part (which gets canonicalized, so you can write them in any equivalent order) reuse that array.
The above is equivalent to a typical System in ECS, except I don't create a class or a global instance for it - I just shove a call to select-entities in the place I need it. This makes the architecture a bit more flexible conceptually, resolving some discomfort I had with Systems.
The :where and :order-by parts get executed each time, on the aforementioned array. I could've made them execute when components are added or removed, at the cost of increased memory usage (no more sharing of the filtered entities array); I chose against it in this project. But this isn't really a problem; :where and :order-by subsume the code you'd typically put in your System - processing entities matching a particular condition (e.g. hp < 50), and processing them in particular order, respectively. There should also be a :limit part, allowing you to pick first N entities for processing, but I switched to SQLite before finishing that feature.
Finally, if I pass an array as :into, the result set is stored in that array (which I've statically allocated previously), instead of allocating new memory.
So in the end, this does the same stuff you'd write in your system, but it handles book keeping for you. The interface lends itself for extra optimizations (this is Lisp, I could parse and process the code I put in :where and :order-by clauses, though I don't do that right now). And because it's not a System, not a Thing, I end up using it more often. Say I need to code a spell that picked weak NPCs and healed them automatically. Writing a System that's responsible for handling this effect is something that wouldn't even cross my mind. But a query like in the example above? Sure, why not? So I use it, and it executes almost the same code that I'd written otherwise, except I get some optimizations handled to me automatically. It's a deep interface, and pretty clean conceptually.
(There goes the other half of the blog post I wanted to write. I guess I have no excuse but to do it now.)
From the point of view of the code operating on these entities, this is a query. That it happens to return results in constant time because it's being continuously executed in the background - that's just an implementation detail.
The example I've pasted doesn't execute a full query every frame either. The first time it's called, it ensures that there exists an array within ECS system that stores entities which match the :components part of my query. That array is then continuously updated when appropriate components are added or removed from entities. Multiple different queries using the same :components part (which gets canonicalized, so you can write them in any equivalent order) reuse that array.
The above is equivalent to a typical System in ECS, except I don't create a class or a global instance for it - I just shove a call to select-entities in the place I need it. This makes the architecture a bit more flexible conceptually, resolving some discomfort I had with Systems.
The :where and :order-by parts get executed each time, on the aforementioned array. I could've made them execute when components are added or removed, at the cost of increased memory usage (no more sharing of the filtered entities array); I chose against it in this project. But this isn't really a problem; :where and :order-by subsume the code you'd typically put in your System - processing entities matching a particular condition (e.g. hp < 50), and processing them in particular order, respectively. There should also be a :limit part, allowing you to pick first N entities for processing, but I switched to SQLite before finishing that feature.
Finally, if I pass an array as :into, the result set is stored in that array (which I've statically allocated previously), instead of allocating new memory.
So in the end, this does the same stuff you'd write in your system, but it handles book keeping for you. The interface lends itself for extra optimizations (this is Lisp, I could parse and process the code I put in :where and :order-by clauses, though I don't do that right now). And because it's not a System, not a Thing, I end up using it more often. Say I need to code a spell that picked weak NPCs and healed them automatically. Writing a System that's responsible for handling this effect is something that wouldn't even cross my mind. But a query like in the example above? Sure, why not? So I use it, and it executes almost the same code that I'd written otherwise, except I get some optimizations handled to me automatically. It's a deep interface, and pretty clean conceptually.
(There goes the other half of the blog post I wanted to write. I guess I have no excuse but to do it now.)
> or Clojure's "let's use maps for everything".
Small world. I'm writing a roguelike in Clojure and indeed use maps for everything. Datascript seemed promising, but it is hard to beat Clojure's builtin data selection functions.
Small world. I'm writing a roguelike in Clojure and indeed use maps for everything. Datascript seemed promising, but it is hard to beat Clojure's builtin data selection functions.
Small world indeed. I work a bit in Clojure from time to time, so I have a passing familiarity with the coding style.
FWIW, for the same game I described above, I also ended up cloning half of Stuart Sierra's famous Component library to use it as my half-baked Dependency Injection framework. This was motivated by the fact that - due to various life reasons - I do a good chunk of my development from a sidearm machine via SSH to my desktop, so I need an architecture where I can develop and test the game in headless mode (no graphics, mock inputs) as much as possible. I write a lot of unit tests now.
FWIW, for the same game I described above, I also ended up cloning half of Stuart Sierra's famous Component library to use it as my half-baked Dependency Injection framework. This was motivated by the fact that - due to various life reasons - I do a good chunk of my development from a sidearm machine via SSH to my desktop, so I need an architecture where I can develop and test the game in headless mode (no graphics, mock inputs) as much as possible. I write a lot of unit tests now.
I did this lightweight ECS for Go and TypeScript:
TS: https://github.com/netgusto/ecs-typescript
Go: https://github.com/bytearena/ecs
Very interesting programming concept indeed!
---
Edit: These implementations support tagged queries, and indexed views for faster iterations (ECS uses iterations a lot).
TS: https://github.com/netgusto/ecs-typescript
Go: https://github.com/bytearena/ecs
Very interesting programming concept indeed!
---
Edit: These implementations support tagged queries, and indexed views for faster iterations (ECS uses iterations a lot).
See also: Richard Fabian's "Data-Oriented Design" book. The fourth chapter is about component-based objects, and the preceding chapters about relational databases and existential processing are relevant.
http://www.dataorienteddesign.com/dodbook/
Prior HN discussion: https://news.ycombinator.com/item?id=20380397
http://www.dataorienteddesign.com/dodbook/
Prior HN discussion: https://news.ycombinator.com/item?id=20380397
I've built the Texel ASCII Art Editor (https://crates.io/crates/texel) using Specs (https://crates.io/crates/specs) as my first experience with ECS.
The original goal was an ASCII game but since I needed to create the resources for it it morphed into Texel. I think the use of ECS here wasn't a bad decision but Specs proved to be just too cumbersome and overoptimizing.
As others have mentioned ECS is a bit "loosly defined" and each implementation seems to go over the line to add more specializing one way or the other. I want to switch to something more simple and elegant like DCES (https://crates.io/crates/dces) for my next refactor.
I think for "runtime resource management" ECS is fine if you need a fairly large, distinct pool of entities to handle. One of my main usage problems was the re-use of same components in the same entity.
E.g. imagine having an entity with a global world position, but also an internal position for something like "last cursor position". It's not possible to just "add another Position component" to the same entity, for good internal reasons, but still. It's pretty important people take these kind of limitations into account before planning out their entity/component maps.
The original goal was an ASCII game but since I needed to create the resources for it it morphed into Texel. I think the use of ECS here wasn't a bad decision but Specs proved to be just too cumbersome and overoptimizing.
As others have mentioned ECS is a bit "loosly defined" and each implementation seems to go over the line to add more specializing one way or the other. I want to switch to something more simple and elegant like DCES (https://crates.io/crates/dces) for my next refactor.
I think for "runtime resource management" ECS is fine if you need a fairly large, distinct pool of entities to handle. One of my main usage problems was the re-use of same components in the same entity.
E.g. imagine having an entity with a global world position, but also an internal position for something like "last cursor position". It's not possible to just "add another Position component" to the same entity, for good internal reasons, but still. It's pretty important people take these kind of limitations into account before planning out their entity/component maps.
In the Clojure world, this used to be a popular way to structure backend apps: https://github.com/stuartsierra/component
> Components provide some basic guidance for structuring a Clojure application, with boundaries between different parts of a system. Components offer some encapsulation, in the sense of grouping together related entities.
> Each component receives references only to the things it needs, avoiding unnecessary shared state. Instead of reaching through multiple levels of nested maps, a component can have everything it needs at most one map lookup away.
> Components provide some basic guidance for structuring a Clojure application, with boundaries between different parts of a system. Components offer some encapsulation, in the sense of grouping together related entities.
> Each component receives references only to the things it needs, avoiding unnecessary shared state. Instead of reaching through multiple levels of nested maps, a component can have everything it needs at most one map lookup away.
Stuart Sierra components and ECS components are not really the same thing though. They share the name “components” but not thaaat much else.
Stuart Sierra Components are more akin to Unity’s behaviour components prior to their introduction of an ECS. They’re a way of encapsulating state which may have dependencies and has a lifecycle (needing to be started or stopped).
ECS components are purely blobs of (usually fine grained) state, the existence and combination of which in an entity creates behaviours (that is, the systems which implement the behaviour will dynamically do so for any entity that has the prerequisite components, but the components are just data).
I always preferred naming ECS components as “traits” (they are the traits that an entity has, eg “animated”) and the systems “behaviours” (they are the behaviours exhibited by entities that have certain traits).
Stuart Sierra Components are more akin to Unity’s behaviour components prior to their introduction of an ECS. They’re a way of encapsulating state which may have dependencies and has a lifecycle (needing to be started or stopped).
ECS components are purely blobs of (usually fine grained) state, the existence and combination of which in an entity creates behaviours (that is, the systems which implement the behaviour will dynamically do so for any entity that has the prerequisite components, but the components are just data).
I always preferred naming ECS components as “traits” (they are the traits that an entity has, eg “animated”) and the systems “behaviours” (they are the behaviours exhibited by entities that have certain traits).
Used to be? What replaced it? I used it in my last Clojure project, and I'm about to start expanding it further; should I replace it with something else?
It's fine, hasn't just been getting much press lately.
try integrant
https://github.com/weavejester/integrant
You can move from using component's functions/macros to using a map and cross references using reader literals.
Having used both, I find integrant to work better for me.
https://github.com/weavejester/integrant
You can move from using component's functions/macros to using a map and cross references using reader literals.
Having used both, I find integrant to work better for me.
I find ECS a pretty intuitive concept, but it always ends up being a massive yak shave to me
Agreed. The vast majority of the benefit is simply from learning to use composition over inheritance.
Most applications that aren't games don't need to support things like finding all active objects of a type or maintain many discrete actors executing their own run loops.
But Unity isn't dumping a lot of marketing into talking about the decorator pattern so everyone is jumping right to ECS now.
Most applications that aren't games don't need to support things like finding all active objects of a type or maintain many discrete actors executing their own run loops.
But Unity isn't dumping a lot of marketing into talking about the decorator pattern so everyone is jumping right to ECS now.
When performance matters I often find myself yak shaving existing code to basically turn it into structure-of-arrays format (like is required for ECS), if not converting to ECS entirely. The alternatives scale so poorly...
What were your problems with it? Was it using a particular library or engine, or you did it yourself?
I'm curious because I started using ECS-like techniques exactly because traditional OOP in games felt a lot like yak shaving.
I'm curious because I started using ECS-like techniques exactly because traditional OOP in games felt a lot like yak shaving.
Who are these people who think that inheritance is the solution to every architectural problem, or any?
I didn't think that Java coders were especially well represented in the gaming world. It has been over 20 years since C++ programmers learned that inheritance is just a way to dress up function pointers (not that that does no good, but function pointers are pretty specialized machinery).
Seriously, you don't need to use a named style to write programs. Your language offers lots of different facilities you can put together in any way that is useful to achieve your aims. Functional, OO, ECS, MVC, procedural, data-oriented, reactive, whatever -- it is all just code.
You have only three problems: code needs to be organized well enough that you can understand it, code should run fast enough, and you should be able to change one thing without rewriting everything else. Nothing else matters.
I didn't think that Java coders were especially well represented in the gaming world. It has been over 20 years since C++ programmers learned that inheritance is just a way to dress up function pointers (not that that does no good, but function pointers are pretty specialized machinery).
Seriously, you don't need to use a named style to write programs. Your language offers lots of different facilities you can put together in any way that is useful to achieve your aims. Functional, OO, ECS, MVC, procedural, data-oriented, reactive, whatever -- it is all just code.
You have only three problems: code needs to be organized well enough that you can understand it, code should run fast enough, and you should be able to change one thing without rewriting everything else. Nothing else matters.
I'm accidentally using this approach in my CMS for saving .NET Core classes (with interfaces - and supports explicit interface implementations) into document stores.
EDIT: Actually, now when I think about it, this approach sorta makes every object (or Entity) into an independent, isolated database. Hm ...
EDIT: Actually, now when I think about it, this approach sorta makes every object (or Entity) into an independent, isolated database. Hm ...
Apple provides an ECS in gameplaykit that can be used in iOS/UIKit apps, not sure why anyone would want to though.
https://developer.apple.com/documentation/gameplaykit
https://developer.apple.com/documentation/gameplaykit
> not sure why anyone would want to though.
What do you mean by that?
GameplayKit has its limitations, but I've been building an entire engine around it, and I like it so far, especially for the ability to stick with pure Swift and native APIs:
https://github.com/InvadingOctopus/octopuskit
What do you mean by that?
GameplayKit has its limitations, but I've been building an entire engine around it, and I like it so far, especially for the ability to stick with pure Swift and native APIs:
https://github.com/InvadingOctopus/octopuskit
That's exactly the idea that I have been toying with recently: using ECS (and even spec crate in particular) to implement an authoritative game server. Haven't yet figured out how to make spec's and tokio's scheduling to play along, though.
How is an Entity Component System different from Objects and Controls in something like Visual Basic for example?
Visual Basic controls are more like React components, I think.
ECS is, traditionally, just a way to dynamically add behaviour (components) into "blank slate" entities.
Instead of making a EvilMonster class in a game you just have it as a conceptual entity (normally it's just an integer, like a database id) associated with a bunch of components (normally just plain structs) and their parameters: Renderable, RigidBody, Collider, HasEnergy, HasTransform, AudioEmitter, etc.
You can stitch entities and components together using either code or some external data source. The "S" part of ECS is called the system, and it implements the behaviour defined by components.
The upsides of this IMO is that this is a fantastic way to structure your code. The data-oriented aspect is also great for enabling non-coders to assembly complex entities. And some other clever people (such as the sibling answer) found out this is also a great optimisation technique.
There are multiple implementations, of course, so people have different opinions.
ECS is, traditionally, just a way to dynamically add behaviour (components) into "blank slate" entities.
Instead of making a EvilMonster class in a game you just have it as a conceptual entity (normally it's just an integer, like a database id) associated with a bunch of components (normally just plain structs) and their parameters: Renderable, RigidBody, Collider, HasEnergy, HasTransform, AudioEmitter, etc.
You can stitch entities and components together using either code or some external data source. The "S" part of ECS is called the system, and it implements the behaviour defined by components.
The upsides of this IMO is that this is a fantastic way to structure your code. The data-oriented aspect is also great for enabling non-coders to assembly complex entities. And some other clever people (such as the sibling answer) found out this is also a great optimisation technique.
There are multiple implementations, of course, so people have different opinions.
I wrote a comment on another post where I explain the cache optimization benefits of ECS, https://news.ycombinator.com/item?id=21674818
Ecs is more like a database.
ECS is all these things and none of these things. It's several separate concepts under the same name; different applications/games pick different pieces of it.
It's kind of meta, really; you can imagine ECS itself as an entity, and different meanings of it as different components that could be included in it.
It's kind of meta, really; you can imagine ECS itself as an entity, and different meanings of it as different components that could be included in it.
ECS is meta? Love the recursive definition ;)
I look forward to seeing the ladder logic editor. I run PLCFiddle [0], which is a purely React ladder logic editor.
[0]: https://www.plcfiddle.com
[0]: https://www.plcfiddle.com
So many of these kinds of OOP modeling complaints and solved by mixins.
I'd earnestly like to hear the responses to this from the down-voters. I had the same thought and suspect there is a good reason not to use mixins.
One problem off the top of my head is run-time changing of components. An entity that inherits from multiple mixins can't inherit from new mixins (or lose existing ones) at runtime.
One problem off the top of my head is run-time changing of components. An entity that inherits from multiple mixins can't inherit from new mixins (or lose existing ones) at runtime.
Yes, ECS is usually about runtime changes in behavior. It also uses IDs rather than hard references, like in a database. Deleting an object makes references to it invalid, rather than references keeping objects alive like in a functional or object-oriented object graph.
But the complaints about OOP in the article are all about the difficulty in modeling orthogonal traits in a single-inheritance hierarchy. That is definitely addressed with mixins.
Nothing in the "Inheritance isn't Always the Best Tool for the Job" section talks about runtime changes in behavior.
Nothing in the "Inheritance isn't Always the Best Tool for the Job" section talks about runtime changes in behavior.
It does touch on it towards the end:
> Most object-oriented languages are designed so that an object's underlying type will be the same for its entire lifetime. This makes things interesting when users want to scale a Circle without maintaining aspect ratio.
> Most object-oriented languages are designed so that an object's underlying type will be the same for its entire lifetime. This makes things interesting when users want to scale a Circle without maintaining aspect ratio.
Are you saying that this is different from mixins?
Absolutely. Single inheritance is a degenerate form of mixin inheritance, where a subclass has a single, fixed superclass.
Mixins simply fix single inheritance so a subclass's parent class is parameterized, and so concrete classes can be composed of several non-related mixins (which will often seem like multiple super-classes).
ECS is a much different and bigger beast. It's a runtime composition pattern, and has almost basically to do with classes or objects.
If you're only complaint about OOP is with single-inheritance, ECS is probably overkill.
Mixins simply fix single inheritance so a subclass's parent class is parameterized, and so concrete classes can be composed of several non-related mixins (which will often seem like multiple super-classes).
ECS is a much different and bigger beast. It's a runtime composition pattern, and has almost basically to do with classes or objects.
If you're only complaint about OOP is with single-inheritance, ECS is probably overkill.
I get that they are different "in action" (I also appreciate that ecs is implemented differently in various places), but conceptually, and for all intents and purposes, they should be regarded as quite similar if I'm not missing something?
Is it the fact that (some implementations of?) mixins' members could collide and conflict with eachother?
Your 3rd paragraph, did you miss "nothing" or perhaps another word?
Is it the fact that (some implementations of?) mixins' members could collide and conflict with eachother?
Your 3rd paragraph, did you miss "nothing" or perhaps another word?
Isn't this usually referred to as the decorator pattern when used this way outside of a game loop?
Short answer is no. Decorator is an OOP design pattern, ECS is an architectural pattern. In that sense it’s a more abstract concept. One could say decorator is an OOP form of it, but I think the data oriented design crowd would balk at doing so in a mainstream OOP language like Java or C#.
I suppose you're right. ECS explicitly calls out a system that separate from the components while most of the decorator examples run code in the components. I've written decorators that are closer to ECS but I guess its fair to say ECS implies a specific implementation.
The critical differences are memory layout and explicit declaration of logical dependencies: this makes the code parallel by default.
This is just false unless you think the featured article isn't ECS. Rendering is an ordered operation that at the very least shouldn't be considered parallel by default. Simply putting it in an ECS pattern doesn't guarantee anything like that.
Its probably better to say it might help you write tighter loops because (ideally) you have a small amount of code looping over a large array of data, instead of a sea of actors hopping around the heap. Of course, you can make tight loops in a lot of different patterns...
Its probably better to say it might help you write tighter loops because (ideally) you have a small amount of code looping over a large array of data, instead of a sea of actors hopping around the heap. Of course, you can make tight loops in a lot of different patterns...
Rendering on modern hardware is fundamentally parallel by default, even if the commands you issue appear to be sequential. In practice multiple commands can be issued in parallel by a modern GPU and fragments are rasterized in parallel as well (divide and conquer), see https://youtu.be/Nc6R1hwXhL8?t=465 and note how it's chunking many triangles up into groups and rasterizing them in parallel (there's a predictable spatial order, but it's not rendering one tri at a time or one screen quadrant or a time). This is necessary to exploit the massive number of cores on these GPUs (thousands, in some cases).
Newer graphics APIs also allow you to build many command buffers at once (in parallel) and allow you to fill GPU vertex/index/texture buffers in parallel from multiple threads once you've mapped them into your address space.
GPU compute is also basically async and operates in parallel with rendering on modern GPUs. See https://www.extremetech.com/extreme/213519-asynchronous-shad...
Rendering is, in practice, parallel. You can enforce sequential ordering if you need it, but you often don't. (Z-buffer based rendering effectively makes parts of your scene parallelizable since the rendering is order-independent, and as demonstrated above tris can be rendered in parallel)
I've been doing scene rendering in parallel for something like 8 years on Direct3D 9 (XNA) and classic OpenGL. Most of my current parallelization is explicit ordering of scene elements which allows me to prepare buffers/draw commands in parallel, and filling GPU buffers in parallel. If I ever move to Vulkan or D3D11/12 I'll be able to exploit parallelism more there. Those old APIs allow mapping GPU resources into user address space which in some cases already allow you to prep future rendering while existing operations are in flight.
It's also common for modern D3D and OpenGL drivers to create hidden threads in your processes that perform rendering operations behind the scenes while you issue your sequential commands from your threads. This effectively turns those APIs into secretly-parallel APIs, and the driver threads can exploit any parallelism hidden away like performing multiple buffer uploads at once or building command buffers in parallel.
Newer graphics APIs also allow you to build many command buffers at once (in parallel) and allow you to fill GPU vertex/index/texture buffers in parallel from multiple threads once you've mapped them into your address space.
GPU compute is also basically async and operates in parallel with rendering on modern GPUs. See https://www.extremetech.com/extreme/213519-asynchronous-shad...
Rendering is, in practice, parallel. You can enforce sequential ordering if you need it, but you often don't. (Z-buffer based rendering effectively makes parts of your scene parallelizable since the rendering is order-independent, and as demonstrated above tris can be rendered in parallel)
I've been doing scene rendering in parallel for something like 8 years on Direct3D 9 (XNA) and classic OpenGL. Most of my current parallelization is explicit ordering of scene elements which allows me to prepare buffers/draw commands in parallel, and filling GPU buffers in parallel. If I ever move to Vulkan or D3D11/12 I'll be able to exploit parallelism more there. Those old APIs allow mapping GPU resources into user address space which in some cases already allow you to prep future rendering while existing operations are in flight.
It's also common for modern D3D and OpenGL drivers to create hidden threads in your processes that perform rendering operations behind the scenes while you issue your sequential commands from your threads. This effectively turns those APIs into secretly-parallel APIs, and the driver threads can exploit any parallelism hidden away like performing multiple buffer uploads at once or building command buffers in parallel.
Ehh...I see what you mean but... A lot of tricks have gone into the render pipeline to get pixels drawn in parallel but a lot of rendering is fundamentally ordered. Rasters, blends, grab passes etc. and other non communicative operations are done in sequence even if they can be done in parallel per pixel. And these are just the required cases. Plenty of time you want to order things for performance reasons.
>Most of my current parallelization is explicit ordering of scene elements which allows me to prepare buffers/draw commands in parallel, and filling GPU buffers in parallel. If
That's not naturally parallel. You did all that work to know the order of the object before hand. Its not naturally parallel like, say, functional programming with no side effects is naturally parallel.
And you've hyper focused on the 3D render pipeline when many 2D frameworks have a single Layout/UI thread.
And the major point is none of that has to do with ECS!
>Most of my current parallelization is explicit ordering of scene elements which allows me to prepare buffers/draw commands in parallel, and filling GPU buffers in parallel. If
That's not naturally parallel. You did all that work to know the order of the object before hand. Its not naturally parallel like, say, functional programming with no side effects is naturally parallel.
And you've hyper focused on the 3D render pipeline when many 2D frameworks have a single Layout/UI thread.
And the major point is none of that has to do with ECS!
Is rendering commonly closely coupled to the ECS? Rendering involves a lot of spatial trees, sorting, and indeed dealing with the nontrivial problem of parallelizing the interdependent rendering work items. It would seem to call for more specialized data structures.
It not. That's my point. You can't just throw things in the pattern and make them parallelized.
I think a decorator could be implemented as ECS.
You decorate your entities with components that get updated via a system.
You decorate your entities with components that get updated via a system.
I've been playing around with ECS. It does seem to solve a lot of architectural problems I had.
I have been using this.
Common Lisp Object system perfect and natural for this kind of programming.
Common Lisp Object system perfect and natural for this kind of programming.
This sounds a lot like OOP purists finding out not everything can be solved by inheritance so they rename something that existed already.
Despite "clear" definition from Wikipedia, ECS as a pattern suffers from multiple personality disorder. There are several different goals that all share the same name, and different implementations pick a different goal set, ending up looking not quite the same, and getting different kinds of benefits.
So outside games, if you pick "ECS as composition over inheritance", you get something like here, or Clojure's "let's use maps for everything". If you go for "ECS as a performance optimization" - a frequently touted benefit - you'll end up storing a lot of global arrays (perhaps arranged in structures), each packing every instance of a component's property across all entities.
If you go for "ECS as a way to have data-driven entities modifiable on the fly" or "ECS as a way to make game logic cleaner", then congratulations, you've just reinvented a relational database! Since you're not developing a game, you might as well store your data in SQLite and call it a day.
(In fact, I'm currently working on a roguelike game as a side project, whose distinct feature is that it stores all its game data in an in-memory SQLite database, precisely to experiment with a pure form of "ECS for game logic" pattern.)
In the end, the author explored one way of doing ECS. There are plenty others, worth their own thought experiments :).