Phantom – forward-thinking concept of OS that's not Linux-derived(phantomos.org)
phantomos.org
Phantom – forward-thinking concept of OS that's not Linux-derived
http://phantomos.org/
53 comments
Actually things in Phantom are not SO different from usual world.
- You can install new version of class code. Of course, app code have to do migration by hand, but that's the case in nowadays world - if you change data file format, you have to convert.
- There's no problem to migrate object tree over the net. Even storing it to a file is not a problem.
- Persistence does not _forbid_ to restart things.
It just does not restart 'em on reboot. Eggs in your fridge are (more or less) persistent :), but you still can throw 'em away and put in fresh ones.
We're living in a persistent world and do not suffer too much from it, right? Non-persistence of software world is just innatural.
We're very used to it, and that's all.
- You can install new version of class code. Of course, app code have to do migration by hand, but that's the case in nowadays world - if you change data file format, you have to convert.
- There's no problem to migrate object tree over the net. Even storing it to a file is not a problem.
- Persistence does not _forbid_ to restart things.
It just does not restart 'em on reboot. Eggs in your fridge are (more or less) persistent :), but you still can throw 'em away and put in fresh ones.
We're living in a persistent world and do not suffer too much from it, right? Non-persistence of software world is just innatural.
We're very used to it, and that's all.
Thanks for the answers, I am glad to hear from the author!
- What do you mean "app code have to do migration by hand"? I'd say that most of the existing file formats need no migration -- images, videos, music basically don't change; and even very complex file formats like Microsoft doc only change with major versions, not with the minor ones -- MS word had tens of thousands code updates (service packs, localization, etc..) but only a dozen or so file formats.
- What do you mean by "restart" in "Persistence does not _forbid_ to restart things."?
if by "restart", you mean "kill all the threads related to object", then I don't think it will help in most cases. If you have a bug in "Draw()" function, this bug will still be there after thread restart.
If by "restart", you mean "wipe all the state, and start from the clean slate", then won't this destroy all the documents user made?
---
We live in persistent world, and we do suffer a lot from it, we are just very used to it. I have to wash my dishes every day, buy new clothes, and repair my car. I would love to be able to "save" the make, model, and license plate # of my car, then "restart" it to get a brand-new car with the same properties.
Default persistence of everything is annoying, and moving it to software is just unnatural :)
- What do you mean "app code have to do migration by hand"? I'd say that most of the existing file formats need no migration -- images, videos, music basically don't change; and even very complex file formats like Microsoft doc only change with major versions, not with the minor ones -- MS word had tens of thousands code updates (service packs, localization, etc..) but only a dozen or so file formats.
- What do you mean by "restart" in "Persistence does not _forbid_ to restart things."?
if by "restart", you mean "kill all the threads related to object", then I don't think it will help in most cases. If you have a bug in "Draw()" function, this bug will still be there after thread restart.
If by "restart", you mean "wipe all the state, and start from the clean slate", then won't this destroy all the documents user made?
---
We live in persistent world, and we do suffer a lot from it, we are just very used to it. I have to wash my dishes every day, buy new clothes, and repair my car. I would love to be able to "save" the make, model, and license plate # of my car, then "restart" it to get a brand-new car with the same properties.
Default persistence of everything is annoying, and moving it to software is just unnatural :)
Well, you just can't imagine non-persistent world. :) I'll try to illustrate.
In non-persistent world you have to disassemble and store to special boxes all the objects in a room before getting out of that room. Or else it all will just vanish. :)
On migration: if you change code in Phantom in a way that does not change object representation, you can just use new code with old data. New paintDocumentPart() can be called for same document object as long as interface is the same.
The problem is with objects with running threads in them. You can't change implementation which is currently running.
Hopefully, it would not be a usual case. As far as I see, Phantom applications won't need forever running threads at all.
os ->(repaint)->window->(get state of)->document
or
doc->(update state trigger)->window->(repaint)->canvas
In either case it's just an event. Update and finish.
Actually, Phantom was a huge mind game of mine for years (first ideas started to form some 25 years ago) and one of the reasons to start actual implementation was that I can't imagine all of the potential problems and solutions for such an environment.
But - I don't want Phantom to be just a test bed for new ideas. I hope it will find its place as practical tool.
In non-persistent world you have to disassemble and store to special boxes all the objects in a room before getting out of that room. Or else it all will just vanish. :)
On migration: if you change code in Phantom in a way that does not change object representation, you can just use new code with old data. New paintDocumentPart() can be called for same document object as long as interface is the same.
The problem is with objects with running threads in them. You can't change implementation which is currently running.
Hopefully, it would not be a usual case. As far as I see, Phantom applications won't need forever running threads at all.
os ->(repaint)->window->(get state of)->document
or
doc->(update state trigger)->window->(repaint)->canvas
In either case it's just an event. Update and finish.
Actually, Phantom was a huge mind game of mine for years (first ideas started to form some 25 years ago) and one of the reasons to start actual implementation was that I can't imagine all of the potential problems and solutions for such an environment.
But - I don't want Phantom to be just a test bed for new ideas. I hope it will find its place as practical tool.
I can certainly imagine a non-persistent world -- why do you keep saying that I cannot?
It is called "restaurant". You come in and get a clean, fresh table. You sit on it and eat, and if you have leftovers, you put them in special boxes. When you leave, everything on the table that you did not put in a box disappears. Very nice and handy!
=====
So what happens in case of errors?
os ->(repaint)->window->(get state of)->document->(EXCEPTION: LIST INDEX OUT OF RANGE)
Does this result in losing the document forever?
=====
So what happens in case of errors?
os ->(repaint)->window->(get state of)->document->(EXCEPTION: LIST INDEX OUT OF RANGE)
Does this result in losing the document forever?
Word->Open file->Error reading file: LIST INDEX OUT OF RANGE
Does this?
Your point is obvious, but not specific to persistent environment. Things are getting broken sometimes.
Actually, back in 2009 when first message about Phantom appeared, I've got a message from team in Stanford. THey did an experiment about things we discuss.
Relatively big Java program was stopped in the middle of work, serialized to (I believe) XML and some objects were removed from that XML. After that program state was de-serialized and program continued to run.
Experiments shown that modern code degrades quite gracefully in such cases. Most of time problems were not global and affected just parts of program's abilities.
But again, it is not a persistent world specific problem.
If your code is wrong, no matter where index lies - in persistent memory object or in file. Index will be wrong.
The question is how system must be built, what kinds of services it must provide to limit app ability to shoot itself in a foot. That is one of the goals of this project.
Does this?
Your point is obvious, but not specific to persistent environment. Things are getting broken sometimes.
Actually, back in 2009 when first message about Phantom appeared, I've got a message from team in Stanford. THey did an experiment about things we discuss.
Relatively big Java program was stopped in the middle of work, serialized to (I believe) XML and some objects were removed from that XML. After that program state was de-serialized and program continued to run.
Experiments shown that modern code degrades quite gracefully in such cases. Most of time problems were not global and affected just parts of program's abilities.
But again, it is not a persistent world specific problem.
If your code is wrong, no matter where index lies - in persistent memory object or in file. Index will be wrong.
The question is how system must be built, what kinds of services it must provide to limit app ability to shoot itself in a foot. That is one of the goals of this project.
In Word (especially when save-in-place disabled), old file disappears only when the new file is complete. And since file writing naturally walks over all the structures, any exceptions usually happen during save and display time.
If the file ended up on disk, it is usually OK.
And even if you are unlucky, you can still restore it from your backup (time machine, dropbox folder, restore point, etc..)
If you have explicit "save" action, all of those things happen automatically -- there is no need for programmer to do any sort of action here, other than to make sure saves are atomic.
Only persistent environments have this problem, because once your persistent-word crashes, you cannot do anything to roll back the change -- other than revert entire OS to previous snapshot, but this will roll back all other apps, too.
(Practically, I've had MS Word crash on me hundreds of times. I've only got corrupted save file once or twice. So at least from my anecdotal evidence switching to "persistent world" would make data loss much more frequent.)
And even if you are unlucky, you can still restore it from your backup (time machine, dropbox folder, restore point, etc..)
If you have explicit "save" action, all of those things happen automatically -- there is no need for programmer to do any sort of action here, other than to make sure saves are atomic.
Only persistent environments have this problem, because once your persistent-word crashes, you cannot do anything to roll back the change -- other than revert entire OS to previous snapshot, but this will roll back all other apps, too.
(Practically, I've had MS Word crash on me hundreds of times. I've only got corrupted save file once or twice. So at least from my anecdotal evidence switching to "persistent world" would make data loss much more frequent.)
Returning to restaurant: you just went off a problem. In your case restaurant is persistent. And a road to your home exists. Suppose, home exists too and you do not rebuild it returning from the dinner.
And at the moment when Linux boots, all the world is absent and is built again from files.
(There's not just no spoon - there's no restaurant. :) And no white rabbit. :)
And at the moment when Linux boots, all the world is absent and is built again from files.
(There's not just no spoon - there's no restaurant. :) And no white rabbit. :)
My home is persistent, and this means I have to clean it up and repair it. I would like to have a magic "reboot" button, which restores my home to blueprints and discards everything not stored in cabinets, drawers and containers.
Ants infestation? "reboot". Walls need painting? "reboot" Don't want to wash the dishes? "reboot"
Isn't this cool? And the best part, we have this for PC already -- and we use it a lot. I can see why one would want to have a small part of the system persistent, but the the entire system? This does not seem useful.
Ants infestation? "reboot". Walls need painting? "reboot" Don't want to wash the dishes? "reboot"
Isn't this cool? And the best part, we have this for PC already -- and we use it a lot. I can see why one would want to have a small part of the system persistent, but the the entire system? This does not seem useful.
The source: https://github.com/dzavalishin/phantomuserland
Previous discussion (6 months ago): https://news.ycombinator.com/item?id=19672610
Previous discussion (6 months ago): https://news.ycombinator.com/item?id=19672610
Hi everybody. As the author of original Phantom OS idea I'd like to thank you for the discussion.
I am here to answer any question, please ask.
If you have some collaboration or experiment with Phantom in mind, I'll be happy to discuss. I'm looking for practical but simple use case for Phantom OS.
I am here to answer any question, please ask.
If you have some collaboration or experiment with Phantom in mind, I'll be happy to discuss. I'm looking for practical but simple use case for Phantom OS.
Being on the vacations I decided to write some Phantom OS internals book, and if you're interested, here it is:
https://phantomdox.readthedocs.io/en/latest/
https://phantomdox.readthedocs.io/en/latest/
> Your program will survive OS reboot. All variables are now.. like files?
Sounds like we will have to format the hard drive and re-install everything every week or even more often.
Sounds like we will have to format the hard drive and re-install everything every week or even more often.
It might just be that programs will have to be written slightly differently.
Sort of how drivers that have to deal with sleep/wake have to be tweaked.
Whem machines got sleep/wake it was a definite step forward for usability (and other things like machine efficiency)
Sort of how drivers that have to deal with sleep/wake have to be tweaked.
Whem machines got sleep/wake it was a definite step forward for usability (and other things like machine efficiency)
Some ideas seem similar to Taos:
https://news.ycombinator.com/item?id=9806607
https://news.ycombinator.com/item?id=9806607
A nitpick, in your FAQ's it should be "in the 1960's" not "the 1960th" if I'm reading the year correctly.
A nitpick: don't use the apostrophe in the phrase "in the 1960s". It's not a possessive in that sentence.
See, e.g., https://www.thepunctuationguide.com/apostrophe.html
See, e.g., https://www.thepunctuationguide.com/apostrophe.html
This is not a hard-and-fast rule. There's hardly agreement in style guides on this note. Don't be a jerk and nitpick style on the internet; it does not add to the conversation.
https://english.stackexchange.com/questions/13631/is-an-apos...
This is largely to taste. An apostrophe doesn't always mean it's possessive, as just demonstrated, but it is also used for plural abbreviations or for numbers (like 1960's).
Casual English is fairly fluid. People rarely talk exactly according to the rules of English class.
Casual English is fairly fluid. People rarely talk exactly according to the rules of English class.
A nitpick, in your comment, it should be “FAQs” not “FAQ’s”.
That's peanuts compared to the typos in "tommorow", also in the FAQ.
Probably the thing that concerns me the most here: If it is as described... turning it off and then back on again probably won't fix whatever it's doing wrong. :/
I'm very curious what troubleshooting a system like this would look like.
I'm very curious what troubleshooting a system like this would look like.
It really depends on how much control on the life cycle of state exists between the user, program and operating system. More to the user, easier to press reset and clear the state. More to the OS then it needs to have some clear guidelines about how to deal with bad behaving programs. If its all up to the programs then it will probably be very brittle since applications may not implement what's needed to clear state when necessary. Although if a program crashes you'd expect that the OS would clear its state.. which I assume is the case.
A button that resets an individual application’s state?
I mean this is already an issue today where people “reboot” their system which really just sleeps/wakes it and are surprised that their problems aren’t fixed.
I think it would be a good thing for UX if an OS made this explicit and had a correct “factory reset” behavior for individual apps. It would make troubleshooting a breeze.
It does kinda suck because under this model to ensure correctness apps either can’t share any state between them or the entire forest has to be reset together.
I mean this is already an issue today where people “reboot” their system which really just sleeps/wakes it and are surprised that their problems aren’t fixed.
I think it would be a good thing for UX if an OS made this explicit and had a correct “factory reset” behavior for individual apps. It would make troubleshooting a breeze.
It does kinda suck because under this model to ensure correctness apps either can’t share any state between them or the entire forest has to be reset together.
Yes. But, really, turning off and on fixes not quite everything. I've seen Word files that can't be open and crash Word completely.
Application state can be unrecoverably wrong with any kind of storage, be it files or persistent object tree.
Application state can be unrecoverably wrong with any kind of storage, be it files or persistent object tree.
But only files offer fully automated backups which do not need any sort of support from application at all.
Most likely what happens when Windows can't restore state from sleep/suspend and you have to delete the state file and do a normal boot. Though hopefully with fine grain control over what you can nuke.
> hopefully with fine grain control over what you can nuke.
And therein lies one of the many challenges of a system like this. It's fine as an experiment - but turning it into something usable for any serious purpose is unlikely to happen.
There are much more reliable and usable ways to implement automatically persistent systems. The way databases use transaction logs is an example.
And therein lies one of the many challenges of a system like this. It's fine as an experiment - but turning it into something usable for any serious purpose is unlikely to happen.
There are much more reliable and usable ways to implement automatically persistent systems. The way databases use transaction logs is an example.
It depends.
Think of:
- OO DB, like key/val storage, but with stored procedures. It's Phantom.
- Smart home UI, SCADA front - lots of METAdata about widgets, windows, screens, etc - but getting all the actual data from network. It's Phantom.
- Robotics. How long your self-driving car takes to reboot, fsck and restart all applications when it drives 100 mph?
- Medic equipment. Please do not die for five minutes, we need to reload application.
There are places that in any case need very clever and special software, would gain of fast same-state restart and need complex but more or less static configuration.
Think of partial use of Phantom nature. Be it usual Unix app, but with ability to have access to pre-built complex object structure to use.
One example: if you have 200 fonts on your PC, starting any appl that uses fonts is "wait half an hour till I rescan all these fonts for you".
Let Phantom personality to prepare persistent font data structures and Unix personality to use them.
It is really interesting even to start thinking in persistent manner. There are new worlds behind, forgive me being pathos man a bit. :)
Think of:
- OO DB, like key/val storage, but with stored procedures. It's Phantom.
- Smart home UI, SCADA front - lots of METAdata about widgets, windows, screens, etc - but getting all the actual data from network. It's Phantom.
- Robotics. How long your self-driving car takes to reboot, fsck and restart all applications when it drives 100 mph?
- Medic equipment. Please do not die for five minutes, we need to reload application.
There are places that in any case need very clever and special software, would gain of fast same-state restart and need complex but more or less static configuration.
Think of partial use of Phantom nature. Be it usual Unix app, but with ability to have access to pre-built complex object structure to use.
One example: if you have 200 fonts on your PC, starting any appl that uses fonts is "wait half an hour till I rescan all these fonts for you".
Let Phantom personality to prepare persistent font data structures and Unix personality to use them.
It is really interesting even to start thinking in persistent manner. There are new worlds behind, forgive me being pathos man a bit. :)
A system like Phantom is not the best solution to any of those problems. They've already been solved in much better ways. If Phantom were developed to the point of commercial usability, it would have to reinvent the techniques that those systems use.
It's funny how all of your specific examples are already solved, many years ago.
- "fsck" is not longer required with journalling filesystems.
- For faster boot, there is "ureadahead" which requests the block from the disk in the right order -- so time to start up the system with 1GB of memory used is roughly the same as read that 1GB from the disk drive.
- For "200 fonts", there is "fc-cache" tool which keeps the font cache -- and you only run it when a new font is installed.
- "fsck" is not longer required with journalling filesystems.
- For faster boot, there is "ureadahead" which requests the block from the disk in the right order -- so time to start up the system with 1GB of memory used is roughly the same as read that 1GB from the disk drive.
- For "200 fonts", there is "fc-cache" tool which keeps the font cache -- and you only run it when a new font is installed.
The robust way is to abstract control logic from the UI logic entirely. So a dedicated processor(s) running bare metal code or an RTOS with a link to a fancier HMI. The HMI can have all the ugly crash happy UI code heaped on top to display data or send changes. If the HMI crashes the controller won't notice or care. There should never be a dependency on the HMI.
>that's not Linux derived
I get that it's supposed to be a strength but it will also be the reason it will be absolutely useless. Even BSDs implemented the Linux glue because everything is made for Linux.
Even if this is absolutely useless, there is an intrinsic value to the author to create this, to learn, and an extrinsic value to this: It may inspire some linux or BSD kernel dev to fix a long standing bug or add increased functionality.
Linus did not intend Linux to be ran on anything but his PC setup either, but here we are.
The „Linux glue” in FreeBSD is just for closed source applications that can’t be rebuilt. Pretty much all the software written for Linux can be easily ported to other Unix systems, since most of the APIs are portable, and the ones that are Linux-specific are usually either useful only for system software (eg netlink) or are broken but design and nobody uses them anyway (eg signalfd(3)).
Ooh what is broken about signalfd?
> There is (very simple) POSIX subsystem in Phantom.
ChromeOS, iOS, Android and plenty of IoT MIT licensed OSes prove otherwise.
We just need money and willingness to press forward no matter what challenges come into developers direction, to avoid a Linux monoculture.
And no, the fact that ChromeOS and Android use Linux kernel doesn't matter, as it is hardly exposed to userspace.
We just need money and willingness to press forward no matter what challenges come into developers direction, to avoid a Linux monoculture.
And no, the fact that ChromeOS and Android use Linux kernel doesn't matter, as it is hardly exposed to userspace.
Fuchsia + Zircon kernel is probably the most serious answer to being a mainstream "open-source" non-linux alternative operating system other than GNU/Linux, Windows and macOS.
I'm intrigued by the comments suggesting that rebooting a machine is a good way to fix problems. I certainly know that it can work but really it's a regrettable last resort to solve a problem.
It would be good to have an OS that brings back the idea of persistent state. Plenty of programs have to run without stopping so require thinking of long term behavior (power plants, various spacecraft, medical implants, etc). People without stopping and in fact halting them is greatly discouraged.
It would be good to have an OS that brings back the idea of persistent state. Plenty of programs have to run without stopping so require thinking of long term behavior (power plants, various spacecraft, medical implants, etc). People without stopping and in fact halting them is greatly discouraged.
Especially those highly critical apps absolutely depend on the the ability to recover via reboot at any moment. There could be random hardware glitches, or software crashes -- and the fix is to discard the RAM, very carefully verify the stored persistent data, and re-create the state from it.
Famously, during the moon landing, the computer had an error and crashed, multiple times [1]. The landing was still a success because while the navigation state was explicitly saved, everything other state was reset. Good thing the designers did not use persistent memory!
[1] https://www.wired.com/story/apollo-11-mission-out-of-control...
Famously, during the moon landing, the computer had an error and crashed, multiple times [1]. The landing was still a success because while the navigation state was explicitly saved, everything other state was reset. Good thing the designers did not use persistent memory!
[1] https://www.wired.com/story/apollo-11-mission-out-of-control...
Religions are very different around the world. Though my gods do not forbid anybody to 'very carefully verify the stored persistent data, and re-create the state from it' even if persistent data is stored in... persistent memory.
Well, just kidding, I'm agnostic, there's no such thing as my gods. ;)
Key point is code that 'very carefully verifies'. Forced RAM nullification on reboot does not help per se.
And there's no need to reboot in any case. If you feel you're busted, in persistent env you still can clear all state and recreate it from some other state.
It's just very traditional to think of files, not RAM as storage.
Memory mapped files, btw, are quarter a step to Phantom.
Well, just kidding, I'm agnostic, there's no such thing as my gods. ;)
Key point is code that 'very carefully verifies'. Forced RAM nullification on reboot does not help per se.
And there's no need to reboot in any case. If you feel you're busted, in persistent env you still can clear all state and recreate it from some other state.
It's just very traditional to think of files, not RAM as storage.
Memory mapped files, btw, are quarter a step to Phantom.
Back in the days of core memory I remember restarting a machine after the power had gone out. No need to reboot (unless power had gone out while the monitory ("kernel") was running) -- only the running process, with its semiconductor registers, would die. Everything else was persistent.
The original Multics design did not make a distinction between memory pages on disk and in RAM. Files were just a way to describe a collection of memory pages, wherever they might be.
The original Multics design did not make a distinction between memory pages on disk and in RAM. Files were just a way to describe a collection of memory pages, wherever they might be.
[deleted]
A "forward-thinking" project shouldn't have been written in C. They should've used Rust.
You're quite right, Rust would be better tool. But first Phantom code was written some 15 years ago and now it's too late to switch. :)
- Upgrade software: right now, my "txt" files are independent from "/usr/bin/emacs". I can upgrade emacs to the later version, and use it to re-open the same documents. This does not seem possible if by file is "text window + object pointer + editor code".
- Move/copy files between machines -- I can copy most of the files, but not objects. I can also save older versions, send them by email or messengers, and so on.
- Recover from bad software -- the other day, my computer decided it wants to have a full-screen uncloseable window. A reboot fixed this right up. Good thing it was not persistent.
- Recover from crashes. What do you do if you write a presentation, and a programming error causes program to stop responding if you ever draw a green circle of certain size? In regular OS, you restart your software, load the latest savefile, and avoid green circles in the future. In PhantomOS, looks like you will be screwed.
The only thing that Phantom seems to offer is snapshots, but they are pretty extreme (the whole system is reverted), and even then, it only helps with some classes of bugs, the ones which are obvious enough that you notice them right away.