The Gates of Argonath in VR at 50 FPS on Unreal Engine(old.reddit.com)
old.reddit.com
The Gates of Argonath in VR at 50 FPS on Unreal Engine
https://old.reddit.com/r/unrealengine/comments/zlytkq/hi_guys_i_made_the_argonath_from_the_lord_of_the/
122 comments
Off the shelf yes, but wow…. The requirements for Unreal are unreal.
Recent versions handle real-time global illumination (Lumen) and dynamically render geometry to insane levels of detail (nanite). Both of these are groundbreaking achievements. But having had to install this in a bunch of university computers I can tell you that it comes at a cost.
Interestingly, the most significant issue is the space these installations require. We have optimized our installation as much as we can, but we nonetheless had to introduce systems whereby local storage was wiped clean every two weeks.
Recent versions handle real-time global illumination (Lumen) and dynamically render geometry to insane levels of detail (nanite). Both of these are groundbreaking achievements. But having had to install this in a bunch of university computers I can tell you that it comes at a cost.
Interestingly, the most significant issue is the space these installations require. We have optimized our installation as much as we can, but we nonetheless had to introduce systems whereby local storage was wiped clean every two weeks.
Wait, disk space was the limiting factor, not GPUs?
Yep. The install size of unreal is a known issue: https://m.youtube.com/watch?v=n7vwGnKSbmE&feature=youtu.be
Our lab computors have 500gb of space. Each time a new profile is activated by a student, that eats up around 5 to 10 gig. In addition… Video editing eats up storage space like there is no tomorrow (the Cache requirements of Resolve are another issue). Nanite supports scene with very dance meshes (downloaded directly from Unreals online store). These add up quickly in terms of file size, especially as each comes in several versions, with multiple levels of detail.
Ideally we would have liked to install more than one version of Unreal. Unreal updates regularly and files created with each version cannot be guaranteed to be opened by subsequent versions. However, this was one saving we made.
There are sample files which are actually very instructive but which we did not install.
All of these issues compounded and towards the end of sem we were having a tough time.
Our lab computors have 500gb of space. Each time a new profile is activated by a student, that eats up around 5 to 10 gig. In addition… Video editing eats up storage space like there is no tomorrow (the Cache requirements of Resolve are another issue). Nanite supports scene with very dance meshes (downloaded directly from Unreals online store). These add up quickly in terms of file size, especially as each comes in several versions, with multiple levels of detail.
Ideally we would have liked to install more than one version of Unreal. Unreal updates regularly and files created with each version cannot be guaranteed to be opened by subsequent versions. However, this was one saving we made.
There are sample files which are actually very instructive but which we did not install.
All of these issues compounded and towards the end of sem we were having a tough time.
> Our lab computors have 500gb of space.
There's your problem. Development and content creation machines need far more disk than this. It doesn't have to be the OS drive; a data drive or two is usually fine, but 500GB in total is far too little.
There's your problem. Development and content creation machines need far more disk than this. It doesn't have to be the OS drive; a data drive or two is usually fine, but 500GB in total is far too little.
A 2tb ssd is little over 100. There's no good reason this isn't upgraded
Exactly. I don’t understand why universities don’t factor in human costs. If it requires manual intervention every 2 weeks, investing $100 should be a no brainer.
I couldn't find the exact video, but Digital Foundry did a great breakdown on the performance bottlenecks in Unreal 5. They showed how disk speed directly influenced asset streaming, and it was pretty interesting to watch. Their HDD-speed drive choked constantly when running the UE5 demo, but a SATA SSD had enough bandwidth to handle the assets.
So, it's kinda a weird spot to be. Not every scene was streaming 250mb/s of assets, but when it did the worse drives took a notable hit to performance.
So, it's kinda a weird spot to be. Not every scene was streaming 250mb/s of assets, but when it did the worse drives took a notable hit to performance.
Are these types of assets typically compressed?
Depends on the system, AFAIK. Modern consoles have so much disk bandwidth that compression is completely unnecessary, but PS5 and Xbox both have APIs for streaming and decompressing compressed assets. I haven't seen any titles using them, and it's doubtful that we'll reach a point where it's required with modern systems.
That being said, compression would be an interesting avenue to explore for SD cards and SATA drives. The performance add can be pretty marginal, but sometimes that's just what these drives need.
That being said, compression would be an interesting avenue to explore for SD cards and SATA drives. The performance add can be pretty marginal, but sometimes that's just what these drives need.
Yep, unreal is surprisingly efficient on the graphics side of things, but it also wants 70-something GB just to install and that's before it starts setting up the graphics cache which adds another 40 GB iirc.
1TB SSDs are $90.
HN is rich and detached from reality in other parts of the world. People still have to manage their savings to buy storage in non rich world.
How much is the rest of a VR setup?
And 2TB is like $120
And thanks to inflation, while $90 used to be like six to eight bags of groceries not that long ago, now it's only two or three. So that's not like you can buy much else with that $90.
I think it will be commonplace to virtually walk around in movie scenes turned into 3d scenery by AI by the end of next year. I can see this turning into a real industry. Who wouldn't like to wander, fly and swim around Pandora, the world of Avatar?
It will also lead to questions about the copyright of the resulting models, probably somewhat dependant on how automated this process is.
It will also lead to questions about the copyright of the resulting models, probably somewhat dependant on how automated this process is.
> It will also lead to questions about the copyright of the resulting models
I'm sure there will be no questions there.
The questions about the copyright of models like DALL-E, Stable Diffusion, etc. exist solely because it infringes on the rights of thousands of random artists all around the world. There is no one single entity with enough power to clearly oppose it so people making the models can get away with "questionable copyright", "legal gray area", etc. kind of talk.
Try the same with a model trained on a single movie created by a multinational with infinite money and all this will be resolved and defined real quick and strongly worded cease & desist letters sent out by end of work day.
I'm sure there will be no questions there.
The questions about the copyright of models like DALL-E, Stable Diffusion, etc. exist solely because it infringes on the rights of thousands of random artists all around the world. There is no one single entity with enough power to clearly oppose it so people making the models can get away with "questionable copyright", "legal gray area", etc. kind of talk.
Try the same with a model trained on a single movie created by a multinational with infinite money and all this will be resolved and defined real quick and strongly worded cease & desist letters sent out by end of work day.
> The questions about the copyright of models like DALL-E, Stable Diffusion, etc. exist solely because it infringes on the rights of thousands of random artists all around the world
How does it infringe? Are your memories of a movie copyright infringement?
https://youtu.be/IFe9wiDfb0E
How does it infringe? Are your memories of a movie copyright infringement?
https://youtu.be/IFe9wiDfb0E
I do enjoy that video, but also, despite my deep cynicism about copyright, I can't help but think this is not in any risk of happening. If you trace the economics it makes no sense. Why would a creator make a thing, then sell you the experience of that thing, then be upset that you have memories of the experience of that thing? If they do try to claw it back in an era where memory editation is possible, why would anybody shell out any money to have an experience if they won't be allowed to remember it tomorrow? Why would I use copyright law to prevent word-of-mouth marketing, the most powerful form of marketing there is?
The point of copyright in general is to protect the economic value for the producer. Being so grabby about "IP rights" that they literally claw back the experience of consuming it means that the economic value of the IP drops to zero. It only has value in the context of being consumed, remembered, etc., and where the person purchasing the experience has confidence that such a thing will be allowed.
I suppose hypothetical future simulated humans could be shaped and molded until they give up some real value for an "experience" they won't be allowed to remember tomorrow, but by then they have passed so far beyond what is "human" that I have no further guesses what they may act like. (That's the true meaning of the "singularity" term, not any particular future, but the point at which our predictions are meaningless.)
The point of copyright in general is to protect the economic value for the producer. Being so grabby about "IP rights" that they literally claw back the experience of consuming it means that the economic value of the IP drops to zero. It only has value in the context of being consumed, remembered, etc., and where the person purchasing the experience has confidence that such a thing will be allowed.
I suppose hypothetical future simulated humans could be shaped and molded until they give up some real value for an "experience" they won't be allowed to remember tomorrow, but by then they have passed so far beyond what is "human" that I have no further guesses what they may act like. (That's the true meaning of the "singularity" term, not any particular future, but the point at which our predictions are meaningless.)
Not normally, but if you use those memories to re-create the movie and sell it to someone else then yeah, that's totally copyright infringement.
If I am an artist and I go to art school, I'm going to spend years studying the masters to learn their techniques and style. If I then produce an impressionist portrait on canvass, informed and influenced by the years of study of existing works.... am I violating copyright?
If you're creating new works inspired by copyrighted works, no that's not (or should not be) copyright infringement. If you're creating exact reproductions of copyrighted works (e.g. The Gates of Argonath from Peter Jackson's adaptation of Lord of the Rings), just in a different format (3d instead of 2d) then yeah, that probably is.
Right. Most of the conversations about IP on this site seem to rely on extremely abstract reasoning or idealist thinking about what constitutes "property" or what a "copy" is. My understanding is that judges are not so easily misled. If you are selling a "replica statue of two kings" online that is clearly intended to resemble the Gates of Argonath, you are setting yourself up for a lawsuit. It doesn't matter if you built it from your memory of the LOTR films, used an AI to generate a bunch of similar looking statues and picked the ones most similar to the films, or what have you. In practice any judge will see that you are attempting to make money off of the LOTR IP. (Note: not legal advice.)
Note that this doesn't mean that Stable Diffusion (and friends) are copyright infringing just because they're trained on copyrighted material. My brain doesn't infringe the LOTR copyright because of my memories of the films. If I turn my experience of fantasy epics into a new novel with a different storyline than LOTR, that's not copyright infringement. That's creativity. But those same memories can likewise be used to make works that are infringing. The question of infringement isn't in the creative act, it's in the artifact. I think it's plausible (though not certain of course) that IP questions will be settled for AIs in the same way: it's not infringement to train your AI on copyrighted material. It is infringement to use your AI to generate works that a reasonable person would conclude are intended to replicate or imitate copyrighted material.
Note that this doesn't mean that Stable Diffusion (and friends) are copyright infringing just because they're trained on copyrighted material. My brain doesn't infringe the LOTR copyright because of my memories of the films. If I turn my experience of fantasy epics into a new novel with a different storyline than LOTR, that's not copyright infringement. That's creativity. But those same memories can likewise be used to make works that are infringing. The question of infringement isn't in the creative act, it's in the artifact. I think it's plausible (though not certain of course) that IP questions will be settled for AIs in the same way: it's not infringement to train your AI on copyrighted material. It is infringement to use your AI to generate works that a reasonable person would conclude are intended to replicate or imitate copyrighted material.
"My brain doesn't infringe the LOTR copyright because of my memories of the films. "
I guess you can pirate Lord of the Rings on bitorrent all you want then? Cause computers are just a bunch of brains remembering things?
This is not a winning legal argument. However this is resolved, no court is going to treat an intentionally designed piece of computer software as a "brain". Or the machine storage of information as equivilent to a human being's "memory".
I guess you can pirate Lord of the Rings on bitorrent all you want then? Cause computers are just a bunch of brains remembering things?
This is not a winning legal argument. However this is resolved, no court is going to treat an intentionally designed piece of computer software as a "brain". Or the machine storage of information as equivilent to a human being's "memory".
I didn't say that? The act of piracy is an act of copyright infringement. The act of watching a legally owned copy of LOTR until my brain remembers almost every single detail down to the smallest element is not copyright infringement, obviously. I don't see how piracy is relevant here at all.
Obviously if OpenAI or other AI researchers are committing media piracy as part of their research, that is something they're legally liable for. It doesn't mean that their AI "remembering" it is inherently infringing.
Obviously if OpenAI or other AI researchers are committing media piracy as part of their research, that is something they're legally liable for. It doesn't mean that their AI "remembering" it is inherently infringing.
My point is the brain metaphor does not work. The copyright act does not regulate "brains" but it does regulate software so what your brain remembers or does not remember is irrelevant.
I predict A.I. trained on LOTR will be found to be infringing because it's a derivative work unless the court determines that use of the original work constitutes a fair use.
My general point in my intentionally silly brain metaphor is brain metaphors are inapplicable and unconvincing.
I predict A.I. trained on LOTR will be found to be infringing because it's a derivative work unless the court determines that use of the original work constitutes a fair use.
My general point in my intentionally silly brain metaphor is brain metaphors are inapplicable and unconvincing.
> I predict A.I. trained on LOTR will be found to be infringing because it's a derivative work unless the court determines that use of the original work constitutes a fair use.
Okay, but that's just a prediction, not an argument. I don't agree with the prediction. I think it makes some sense to think about the AI training data on analogy with memories, because they contain impressions of a thing without being able to reproduce it exactly. They can both be the source of both real creativity as well as the source of infringing works.
I don't see an argument for considering the training data produced by AI training a derivative work, at least not in any copyrightable sense. ChatGPT is "software"; the training data that it operates on is not "software" in the strictest sense and exists in an uncomfortable middle space at present with respect to copyright law. That's what the memory analogy is supposed to show: just like human memories aren't subject to copyright, AI training data that is similar in structure and capabilities to human memories is likewise not subject to copyright. More precisely, my point is that that's one entirely reasonable way courts could rule: AI training data represents "fair use" of its sources (provided they are legally obtained of course), but the products of an AI trained on those sources might not be.
Okay, but that's just a prediction, not an argument. I don't agree with the prediction. I think it makes some sense to think about the AI training data on analogy with memories, because they contain impressions of a thing without being able to reproduce it exactly. They can both be the source of both real creativity as well as the source of infringing works.
I don't see an argument for considering the training data produced by AI training a derivative work, at least not in any copyrightable sense. ChatGPT is "software"; the training data that it operates on is not "software" in the strictest sense and exists in an uncomfortable middle space at present with respect to copyright law. That's what the memory analogy is supposed to show: just like human memories aren't subject to copyright, AI training data that is similar in structure and capabilities to human memories is likewise not subject to copyright. More precisely, my point is that that's one entirely reasonable way courts could rule: AI training data represents "fair use" of its sources (provided they are legally obtained of course), but the products of an AI trained on those sources might not be.
A court isn't going to anthropomorphise computers, as you are, doing to settle the issue. It's just sloppy thinking IMHO.
It also has no precedent in it's favor. As far as I know no court has ever used "Is it like a brain?" to settle a copyright issue.
It also has no precedent in it's favor. As far as I know no court has ever used "Is it like a brain?" to settle a copyright issue.
> it's not infringement to train your AI on copyrighted material.
If that's the case, it should probably should be codified in law. Copyright has a lot of exceptions, and if the existence of AI models trained with copyrighted data are in public interest then it could be an exception too, but this should not be up to the courts.
If that's the case, it should probably should be codified in law. Copyright has a lot of exceptions, and if the existence of AI models trained with copyrighted data are in public interest then it could be an exception too, but this should not be up to the courts.
No, because it took years of study and producing a new portrait will take weeks. If you were able to study a master's entire portfolio in a matter of hours and then instantaneously generate 10,000 portraits in their distinct style, yes that would be unethical. I don't know enough about copyright to say if it would be illegal and I don't think that's a particularly interesting topic.
You just described some pretty amazing technology that can quite possibly move the human race forward and I’m surprised your first thought is that it is unethical. New technology often displaces old methodologies by disruption. It is not unethical to disrupt, but it may be unethical to let those people be disrupted without compensation, which is where I feel a lot of the anti-technology sentiment comes from.
As a society we ask people to branch out into various expertise… through no fault of their own maybe someone’s expertise is randomly obsoleted by a new technology. Just as copyright laws incentivize new creations, we as a society should incentivize people to embrace new disrupting technologies by safety-netting those displaced by them since it’s a gamble more or less of who is next to be displaced.
As a society we ask people to branch out into various expertise… through no fault of their own maybe someone’s expertise is randomly obsoleted by a new technology. Just as copyright laws incentivize new creations, we as a society should incentivize people to embrace new disrupting technologies by safety-netting those displaced by them since it’s a gamble more or less of who is next to be displaced.
So your primary complaint here is the amount of time it took? Why shouldn't a bike be illegal as it allows you to go places in weeks that used to take years?
If you're influenced no.
If you build your own replica of the bridge of the Enterprise, then yes.
This is pretty settled law.
If you build your own replica of the bridge of the Enterprise, then yes.
This is pretty settled law.
[deleted]
I’d like to add that all creative work stands on the shoulders of predecessors by being trained and based on (even if indirectly influenced) previous works. This really is unavoidable on a human level because we can only do what we see and experience.
> people making the models can get away with "questionable copyright", "legal gray area", etc. kind of talk.
Where are they saying that? Nobody thinks it's a gray area, they think it's explicitly legal and they're probably right too. Claiming it's a copyright violation is actually a rather weak case artists are just saying is obvious.
Microsoft has licensed DALL-E. They're not venturing into legal gray areas.
Where are they saying that? Nobody thinks it's a gray area, they think it's explicitly legal and they're probably right too. Claiming it's a copyright violation is actually a rather weak case artists are just saying is obvious.
Microsoft has licensed DALL-E. They're not venturing into legal gray areas.
Well, I think it is a grey area.
The argument I heard most often goes along the lines of: "people also learn from a lot of examples and other artists, and that is not copyrighted".
It is an interesting argument, but AI is not people and the argument could easily be the opposite (not taking sides here). They might think it is legal, but it is far from clear, IMHO.
IANAL, but it will be interesting to see if this is enforceable and by whom.
The argument I heard most often goes along the lines of: "people also learn from a lot of examples and other artists, and that is not copyrighted".
It is an interesting argument, but AI is not people and the argument could easily be the opposite (not taking sides here). They might think it is legal, but it is far from clear, IMHO.
IANAL, but it will be interesting to see if this is enforceable and by whom.
> Microsoft [...] not venturing into legal gray areas.
Their history as litigation defendants would suggest your optimism may perhaps be unfounded.
Their history as litigation defendants would suggest your optimism may perhaps be unfounded.
The day after movie-to-VR becomes easy to do, movie owners will monitize it, which will end the current free-for-all. Want to walk around The Shire in VR? You better have paid for the Hobbits Online subscription package.
Sam:
This is it.
Frodo: What?
Sam: If I take one more step, I'll be the farthest away from home I've ever been.
Popup: And you can do that for just $29.99 when you buy The Hobbits Shire Expansion DLC!
Frodo: What?
Sam: If I take one more step, I'll be the farthest away from home I've ever been.
Popup: And you can do that for just $29.99 when you buy The Hobbits Shire Expansion DLC!
> The day after movie-to-VR becomes easy to do, movie owners will monitize it, which will end the current free-for-all.
The movie making studios already own those 3d "digital assets" from their films. IDK if there is a real "free-for-all" right now, but the monetisation play has already been planned for.
The movie making studios already own those 3d "digital assets" from their films. IDK if there is a real "free-for-all" right now, but the monetisation play has already been planned for.
The copyright protected material isn't in the model. This can be figured out easily because, if they did, the models are so much smaller than the input data that it'd be literal unobtanium for compression algorithms.
Copyright also has nothing to do with the tool.
What a piece depicts and how it is used are much more important than how a piece was created. If I paint a picture by hand of a particular scene from a Disney movie, that's clearly a derived work and may be subject to copyright protection (it depends, there could be a fair use argument in specific situations, so even "it's copyright!" is not a cut-and-dried argument). If I paint a picture of Hercules in the style of Disney, that's trademark violation. But I can paint Hercules in any other style and Disney can't say shit (though they certainly try).
This is an age-old argument. Some artists think they should have a monopoly on certain ideas. As a society, we've already said no, you only get to own your expression of your ideas.
And as ideas go, "like a trending artist on ArtStation" isn't a particularly strong one.
Copyright also has nothing to do with the tool.
What a piece depicts and how it is used are much more important than how a piece was created. If I paint a picture by hand of a particular scene from a Disney movie, that's clearly a derived work and may be subject to copyright protection (it depends, there could be a fair use argument in specific situations, so even "it's copyright!" is not a cut-and-dried argument). If I paint a picture of Hercules in the style of Disney, that's trademark violation. But I can paint Hercules in any other style and Disney can't say shit (though they certainly try).
This is an age-old argument. Some artists think they should have a monopoly on certain ideas. As a society, we've already said no, you only get to own your expression of your ideas.
And as ideas go, "like a trending artist on ArtStation" isn't a particularly strong one.
> There is no one single entity with enough power to clearly oppose it so people making the models can get away with "questionable copyright", "legal gray area", etc. kind of talk.
Never underestimate the power of a sufficiently motivated rich individual to create legal hell (see Gawker). There are many very rich artists. There just needs to be an obvious large enough irritant which will happen as these generative models start getting used beyond toy amusements. The class action lawsuit against Copilot was filed just as pricing models started.
This is going to be an interesting area to watch. Use in commercial projects at your own risk of exposure to liability.
Never underestimate the power of a sufficiently motivated rich individual to create legal hell (see Gawker). There are many very rich artists. There just needs to be an obvious large enough irritant which will happen as these generative models start getting used beyond toy amusements. The class action lawsuit against Copilot was filed just as pricing models started.
This is going to be an interesting area to watch. Use in commercial projects at your own risk of exposure to liability.
The wrong and easy reaction to this is to limit these services to not include copyrighted content. The more right and really really difficult solution would be to give the original copyright holders partial rights to the output.
It really feels like the textile factory worker problem all over again. We can't let the big players be the benefactors every time.
It really feels like the textile factory worker problem all over again. We can't let the big players be the benefactors every time.
> The more right and really really difficult solution would be to give the original copyright holders partial rights to the output.
That is impossible (assigning credit to training images from the output is arbitrary, not deterministic) and wouldn't help anyone (imagine Spotify except it pays you even less).
> It really feels like the textile factory worker problem all over again. We can't let the big players be the benefactors every time.
Industrialization is better for society because it produces more customer surplus. Even Marx thought the Luddites were wrong.
That is impossible (assigning credit to training images from the output is arbitrary, not deterministic) and wouldn't help anyone (imagine Spotify except it pays you even less).
> It really feels like the textile factory worker problem all over again. We can't let the big players be the benefactors every time.
Industrialization is better for society because it produces more customer surplus. Even Marx thought the Luddites were wrong.
>Industrialization is better for society because it produces more customer surplus.
Also my point.
>Even Marx thought the Luddites were wrong.
Sure they are wrong in that they confuse what the problem is. If workers owned the factory they would be happy to automate the work. But their feeling of being left out has some validity because if they weren't the total value would improve.
The real issue is that the revenue brought back through automation doesn't benefit the worker. If it did, the automation would happen sooner because incentives would align better and the total value for society would thus likely improve faster.
Also my point.
>Even Marx thought the Luddites were wrong.
Sure they are wrong in that they confuse what the problem is. If workers owned the factory they would be happy to automate the work. But their feeling of being left out has some validity because if they weren't the total value would improve.
The real issue is that the revenue brought back through automation doesn't benefit the worker. If it did, the automation would happen sooner because incentives would align better and the total value for society would thus likely improve faster.
What if you train your AI model with three dozen films about ancient egypt and then generate a new 3d world out using the model? Seems fuzzy.
But what if you use only three films? Or two?
But what if you use only three films? Or two?
I think it would be almost trivial to build in a “generate 30% different than” (the legal amount of difference before a work is considered different than the original). That could either sidestep the entire legal question, or throw Hollywood in to a blender since so many movies/shows have been made on the back of that difference.
I think it would be the other way around. A lot of movies(and tv shows) have games these days. You can already experience Star Wars in VR with “Star Wars Squadrons” and I’m sure we will start to see a lot more games with VR support in the future, many being based on Movies. MNCs are there to make money and they will just license it.
> I think it will be commonplace to virtually walk around in movie scenes turned into 3d scenery by AI by the end of next year.
I would take the opposite bet, I doubt most movies have complete enough 3D scene to be AI enhanced, let alone something good enough to walk around, you wouldn't model what you won't film after all and nobody wants to see hallucinated aliens in middle earth.
I would take the opposite bet, I doubt most movies have complete enough 3D scene to be AI enhanced, let alone something good enough to walk around, you wouldn't model what you won't film after all and nobody wants to see hallucinated aliens in middle earth.
No, AI will generate the 3d scene just by looking at the 2d footage.
It can fill in the blanks (i.e. provide textures for areas that are invisible in the film).
It can fill in the blanks (i.e. provide textures for areas that are invisible in the film).
It will make a chuffing good effort but fail hilariously in >99% of cases but then people will parade the fraction that make sense.
It's getting better all the time, check https://www.youtube.com/watch?v=_Y1-KlTEmwk
> Who wouldn't like to wander, fly and swim around Pandora, the world of Avatar
This is a good example of the fundamental limits of headset VR. Jumping into a lake will never feel real. Not by touch, obviously as you can’t get wet. Not by physical movement because you almost certainly can’t move downwards irl. Not by the weight of water around you or the buoyancy you expect to feel. Not by the rotation you would expect to have in water. Probably not by the stroke propeller movement or the correlation of your breath. And of course probably overly smooth movement to avoid motion sickness.
The difference between controlling a flying camera through water and believing you’re swimming is pretty big.
This is a good example of the fundamental limits of headset VR. Jumping into a lake will never feel real. Not by touch, obviously as you can’t get wet. Not by physical movement because you almost certainly can’t move downwards irl. Not by the weight of water around you or the buoyancy you expect to feel. Not by the rotation you would expect to have in water. Probably not by the stroke propeller movement or the correlation of your breath. And of course probably overly smooth movement to avoid motion sickness.
The difference between controlling a flying camera through water and believing you’re swimming is pretty big.
This led me to imagine people who learn to swim in VR and, because breathing is counterproductive underwater, then drown IRL.
Then the family suing as the city doesn't have signs warning of holding your breath while underwater.
> Not by touch, obviously as you can’t get wet [et al]
What you need is a home version of the 4DX cinema setup. If VR lasts another 5 years, I'd put money on someone making one of those in that timeframe.
What you need is a home version of the 4DX cinema setup. If VR lasts another 5 years, I'd put money on someone making one of those in that timeframe.
Exactly. We don't have virtual reality at all. We have virtual remote viewing with sound.
We are basically at the Eliza stage of VR with a bunch of people pretending Eliza is not that far from passing the Turing Test.
Once we have virtual skin sensation and virtual physical movement there is no intellectual debate about any of this. Just instruction tutorials about how to have the experience because no marketing needed.
Once we have this I can think of far more interesting things than swimming around in the Avatar movie.
We are basically at the Eliza stage of VR with a bunch of people pretending Eliza is not that far from passing the Turing Test.
Once we have virtual skin sensation and virtual physical movement there is no intellectual debate about any of this. Just instruction tutorials about how to have the experience because no marketing needed.
Once we have this I can think of far more interesting things than swimming around in the Avatar movie.
>Who wouldn't like to wander, fly and swim around Pandora, the world of Avatar?
Sure, like once or twice? I could see it getting old fast, though.
Sure, like once or twice? I could see it getting old fast, though.
You might get bored with a specific location but not the general concept.
I'd like a version of Tokyo that is hyper detailed and functioning transit system/lines.
Spend a week preparing for a trip on how to navigate the city before actually going.
Spend a week preparing for a trip on how to navigate the city before actually going.
You don't need practice for that, all the signs are in English and you just use Jorudan for route planning. Or Google Maps if you must, though it's gotten worse over time.
nb: you may need practice to navigate Shinjuku station
nb: you may need practice to navigate Shinjuku station
I'd like a version of my city without trash and traffic!
Make them hangout spaces for VRChat and people will spend a lot of time in them.
I don't know but then agree. The general concept sounds boring to me but I can see how this can be interesting for many people. For example as a Starwars fan someone must want to fly the same scenes in VR for a few times.
But I think the better is to train AI to generate scenes based on description such as "dogfight in P-51 with German ME-109 in WW2 for 15 mins" and after some computation the player gets to play this game in VR for 15 mins. The problem is how to make it possible at low cost as player will probably just pay a few bucks for such experience (but you can always resell the popular ones many times)
But I think the better is to train AI to generate scenes based on description such as "dogfight in P-51 with German ME-109 in WW2 for 15 mins" and after some computation the player gets to play this game in VR for 15 mins. The problem is how to make it possible at low cost as player will probably just pay a few bucks for such experience (but you can always resell the popular ones many times)
I disagree. VR is cool, but just walking around with nothing to do is not that interesting now.
Do you feel the same way about real life locations? Exhibitions? Galleries?
Obviously nobody is saying "VR is the same as real life" but surely to some degree it's the content of the experience that determines whether something is interesting. Some locations would be fascinating to walk around even if there's technically "nothing to do".
I'm not much of a gamer so my interest in VR has always slanted towards towards the more passive experiences. In fact quite often don't want my enjoyment spoiled by having to complete tasks or solve puzzles just to get around.
Obviously nobody is saying "VR is the same as real life" but surely to some degree it's the content of the experience that determines whether something is interesting. Some locations would be fascinating to walk around even if there's technically "nothing to do".
I'm not much of a gamer so my interest in VR has always slanted towards towards the more passive experiences. In fact quite often don't want my enjoyment spoiled by having to complete tasks or solve puzzles just to get around.
Once that stops being exciting we could tell AI to create new worlds combining others such as mixing the Smurfs and Pocahontas worlds... I wonder what it could come up with /s
>lead to questions about the copyright
It will lead to questions - but those questions will be answered by the platform owners not the courts. The platform owners will just remove your models even in cases where a court would rule it fair use.
VR of real places is going to be huge - bigger than for imaginary places. My guess is that most VR worlds will be a combination.
Here's a question. What if you model the real world and an architect of a building. I know that copyright doesn't apply to buildings or other functional works. But again, this will be decided by the platform owners and not the courts.
Clearly there are good reasons to not have "platform owners" in the metaverse.
It will lead to questions - but those questions will be answered by the platform owners not the courts. The platform owners will just remove your models even in cases where a court would rule it fair use.
VR of real places is going to be huge - bigger than for imaginary places. My guess is that most VR worlds will be a combination.
Here's a question. What if you model the real world and an architect of a building. I know that copyright doesn't apply to buildings or other functional works. But again, this will be decided by the platform owners and not the courts.
Clearly there are good reasons to not have "platform owners" in the metaverse.
The Trek community created a pretty detailed model of the Enterprise-D, but took it down due to copyright concerns. It's still floating around if you know where to look but it's a real shame that development on it had to stop out of legal fears, well founded or not.
I can see this turning into a real industry. Who wouldn't like to wander, fly and swim around Pandora, the world of Avatar?
Me. It sounds boring. I couldn't give a damn about exploring someone else's fantasy world. This is a bit odd because I love open world video games. I guess I need a reason to spend time there.
Me. It sounds boring. I couldn't give a damn about exploring someone else's fantasy world. This is a bit odd because I love open world video games. I guess I need a reason to spend time there.
Once we can generate the world quickly, how far are we from generating things to do in that world?
I think ChatGPT showed exactly that we are not that far away. ChatGPT fine-tuned on the specific world could generate quests and make every NPC able to get interacted with
if you watch TV, movies, read books, post games, or exist, you're already exploring someone else's world
> It will also lead to questions about the copyright of the resulting models
I very much doubt it. Within the Movie business, the ownership of "digital assets" such as 3d models, who keeps what, and what they can legally do with it, is the subject of long and detailed contracts between the film studio, VFX houses, subcontractors etc. Ownership is well-covered, if not exactly "clear" if you're not versed in the law regarding it.
A third party using any technique (AI included) to re-create a look-alike of such an asset, and to monetise this, would surely attract the "eye of Sauron" of the big studios to turn it's withering gaze upon them, and to send out a well-armed detachment of lawyers.
I very much doubt it. Within the Movie business, the ownership of "digital assets" such as 3d models, who keeps what, and what they can legally do with it, is the subject of long and detailed contracts between the film studio, VFX houses, subcontractors etc. Ownership is well-covered, if not exactly "clear" if you're not versed in the law regarding it.
A third party using any technique (AI included) to re-create a look-alike of such an asset, and to monetise this, would surely attract the "eye of Sauron" of the big studios to turn it's withering gaze upon them, and to send out a well-armed detachment of lawyers.
Why by the end of next year? That seems optimistically short.
Why hadn't it happened this year, or last? Nanite is great, but IMO doesnt change the equation that dramatically.
The cost for a studio to manually decimate game quality versions have been small enough that of the will had been there, we could have been doing it for years.
Why hadn't it happened this year, or last? Nanite is great, but IMO doesnt change the equation that dramatically.
The cost for a studio to manually decimate game quality versions have been small enough that of the will had been there, we could have been doing it for years.
> Why hadn't it happened this year, or last?
Having loosely followed this space, I think I might actually have some input!
First off, you need a VR-ready environment. Not many companies are making these, so the only people equipped to roll this out is Valve, Meta, and possibly Unreal/Epic. Valve already did this alongside Half Life Alyx; they released certain map portions so you could freely walk around them with friends without playing the game. Meta is getting there, but the combination of low-power hardware and headset attrition doesn't leave them with much of an audience. There's also Epic, but they don't really do much in this space yet either.
So, obviously a paradigm shift is needed. And frankly, I think Universal Scene Description will be that paradigm shift. TL-DR: Nvidia came up with a new standard for 3D environments that interoperates game technology with digital effects. In theory, you could take your favorite Andor or Rings of Power scenes, drag their file into another app, and just start walking around in-scene. You may end up adjusting a few things vis-a-vis scale, but once it reaches adoption I can see it being pretty easy to work with and popular.
Maybe 'next year' is a little close, but the stars have definitely aligned already.
Having loosely followed this space, I think I might actually have some input!
First off, you need a VR-ready environment. Not many companies are making these, so the only people equipped to roll this out is Valve, Meta, and possibly Unreal/Epic. Valve already did this alongside Half Life Alyx; they released certain map portions so you could freely walk around them with friends without playing the game. Meta is getting there, but the combination of low-power hardware and headset attrition doesn't leave them with much of an audience. There's also Epic, but they don't really do much in this space yet either.
So, obviously a paradigm shift is needed. And frankly, I think Universal Scene Description will be that paradigm shift. TL-DR: Nvidia came up with a new standard for 3D environments that interoperates game technology with digital effects. In theory, you could take your favorite Andor or Rings of Power scenes, drag their file into another app, and just start walking around in-scene. You may end up adjusting a few things vis-a-vis scale, but once it reaches adoption I can see it being pretty easy to work with and popular.
Maybe 'next year' is a little close, but the stars have definitely aligned already.
Thoughts on the web and the role that WASM, WebGPU, and WebXR have as a distribution channel for these immersive experiences?
WebXR feels like a solid building-block to me. Adoption will be (and has been) shaky, but with Apple being forced to adopt third-party browser engines it's revival seems likely.
WASM is neat but extremely finnecky at the moment. I've seen cool stuff done with it, but I'm not convinced it's funny production ready yet.
WebGPU is sorta in the same boat as WASM, if a little more realistic of a bet. People will want hardware-accelerated browser experiences soon, and WebGPU will probably do most of the lifting there.
WASM is neat but extremely finnecky at the moment. I've seen cool stuff done with it, but I'm not convinced it's funny production ready yet.
WebGPU is sorta in the same boat as WASM, if a little more realistic of a bet. People will want hardware-accelerated browser experiences soon, and WebGPU will probably do most of the lifting there.
The content pipelines aren't there, or visual fidelity.
Have you read "Ready Player One"?
The author predicted that players will play a game where they will be dropped into their favorite movies, and will be given points for exactly mimicking the dialogue, inflection, and movements of the characters.
Spoiler: the first one is War Games.
The author predicted that players will play a game where they will be dropped into their favorite movies, and will be given points for exactly mimicking the dialogue, inflection, and movements of the characters.
Spoiler: the first one is War Games.
"Who wouldn't like to wander, fly and swim around Pandora, the world of Avatar?"
Me, but I doubt it will be the same experience, if the scene is highly optimized by hand like the movie is vs. some AI generation.
Me, but I doubt it will be the same experience, if the scene is highly optimized by hand like the movie is vs. some AI generation.
[deleted]
50 FPS is too low. There's a reason that 90 FPS is the low bar for VR.
90 FPS for redrawing the image according to the motion of your head, but the actual rendering of the scene can be much slower. Motion reprojection is basically the only way to make GPU-intensive games like MSFS playable in VR on < $1000 GPUs.
Retrojection is great but it can't make up for 50 fps.
For something this big and far away you'd think there would be a way to cheat.
It's not as if things are going to move massively if you move your head a bit, you should be able to precalculate things to an extent.
It's not as if things are going to move massively if you move your head a bit, you should be able to precalculate things to an extent.
There is such a way to cheat, and it's pretty clever. Store a depth map and just approximate parallax while you're waiting for a real new frame. https://en.wikipedia.org/wiki/Asynchronous_reprojection
Most VR headsets/runtimes support this but it only goes so far. The closer things are the more occluded sections will need to be in-painted. The farther the scene is the more it might as well be a skybox at infinite distance.
Nah. Not once you are experienced. Agreed as far as beginners.
One thing that sticks out like sore thumb are those relatively low-res rock textures, especially close-up. Would be neat to see some procedural generation there.
For reference here is the scene from the movie https://www.youtube.com/watch?v=EN7kG7Ui7Bg
For reference here is the scene from the movie https://www.youtube.com/watch?v=EN7kG7Ui7Bg
Am I alone in thinking most of this year's Unreal Engine demonstrations are just...not that great? Like, this is what graphics looked like on the Xbox One from nine years ago: https://halo.wiki.gallery/images/1/12/H5G-Waypoint-Swords_of...
That screenshot does not look rendered in game, and no idea what the resolution is - at least not by an Xbox One. If current games are indistinguishable from single-frame renders that take minutes if not hours, I'd say that's a massive win!
This looks more like the actual in-game rendering: http://cdn.themis-media.com/media/global/images/library/deri...
What stands out to me is the low resolution textures everywhere, obvious seams between terrain and assets (runtime virtual texturing resolves that), light seams which have odd seams, and the entire ambience being vaguely similar to how light acts, but not really - primarily the lack of bounce lighting affecting anything. With the move to HDR, the more binary lighting of previous games looks weird too now.
Contrast that with the rich detail in lighting you can see in Fortnite's new update - https://i.redd.it/g2wqe10no44a1.jpg
Nanite, especially foliage, is something you have to experience for yourself though. Foliage pop-in not being so stark and alarming is huge for me, and the rich lighting capabilities improves immersion massively.
Regarding assets, high polygon assets have been created for ages too, but were never actually brought in to games. You bake those details into your normal maps, but with Nanite you can just bring them into games, and get the full detail when the player is up close to them. When games can fully embrace Lumen and Nanite, the developer iteration improvement will be huge.
This looks more like the actual in-game rendering: http://cdn.themis-media.com/media/global/images/library/deri...
What stands out to me is the low resolution textures everywhere, obvious seams between terrain and assets (runtime virtual texturing resolves that), light seams which have odd seams, and the entire ambience being vaguely similar to how light acts, but not really - primarily the lack of bounce lighting affecting anything. With the move to HDR, the more binary lighting of previous games looks weird too now.
Contrast that with the rich detail in lighting you can see in Fortnite's new update - https://i.redd.it/g2wqe10no44a1.jpg
Nanite, especially foliage, is something you have to experience for yourself though. Foliage pop-in not being so stark and alarming is huge for me, and the rich lighting capabilities improves immersion massively.
Regarding assets, high polygon assets have been created for ages too, but were never actually brought in to games. You bake those details into your normal maps, but with Nanite you can just bring them into games, and get the full detail when the player is up close to them. When games can fully embrace Lumen and Nanite, the developer iteration improvement will be huge.
Well that screen capture is taken from a cinematic scene if I am correct. UE demonstrations are real-time playable environments.
Have you tested the Valley of the Ancients demo on your computer? You will see what I mean.
Have you tested the Valley of the Ancients demo on your computer? You will see what I mean.
Nanite gives you high fidelity and Lumen gives you physically accurate lighting, but those things can't replace good art direction.
The quality of nanite heavily depends on your model. You can use photogrammetry scans or CAD exports directly, and nanite will make sure they'll look good. But if you just use existing videogame or fanmade assets (like here), the quality obviously suffers.
The technology at play is about handling scaling detail. You can now use very large and very detailed models so your scenes look good from close and afar. A single shot misses the point.
What makes that shot look good are just the post effects, eg. depth of field and god rays.
What makes that shot look good are just the post effects, eg. depth of field and god rays.
There's no accounting for taste, but another reason why the Unreal advancements are a big deal is improved productivity for artists. The geometry in that Xbox One screenshot was painstakingly optimized by some poor overworked artist, whereas this guy just threw together a couple billion polygons and UE5 didn't even flinch.
edit: not to mention the lighting in the Xbox One screenshot was also pre-baked and probably manually tweaked, compared to real lighting updated in realtime in UE5.
edit: not to mention the lighting in the Xbox One screenshot was also pre-baked and probably manually tweaked, compared to real lighting updated in realtime in UE5.
Halo 9 nine years ago looked like this
https://images.app.goo.gl/wbcDbmHJhcUsxB5YA
https://images.app.goo.gl/wbcDbmHJhcUsxB5YA
Halo Anniversary came out 11 years ago, and it ran on hardware from 2005.
1 is the loneliest number.
This looks so great. When I see this I'm somewhat sad that I don0t have computer powerful enough to try to do something like this by myself.
[deleted]
VR capable computers are quite cheap. Cheaper than a low spec apple macbook pro running m1. For 2-3k you can get a beast of a machine running everything on real high settings.
Considring a brand new top of the line Nvidia GPU will currently cost you almost 2k, your prices feel a little ~4-5 years ago.
Right. You only need a card that's more powerful than the GTX 1060 for VR though, so the actual price-of-admission is more in the $250-400 price range.
The original comment wanted to try a demo like this. A RTX 3080 only runs it at 50 fps. A 1060 isn't good enough.
The parent I'm responding to was interpreting "VR capable" as "top of the line" which is demonstrably false. You probably can't even get this scene in the first place, so it's kinda a moot point.
i don't think you need a 4090 for this. and even if you did get one, its 1600$. another 1000$ for good CPU and ram, etc. and you're well under 3k.
In my experience you can almost always do much better buying a full computer on a good sale than buying components and building it yourself.
As someone who never appreciated VR, this looks great.
Now please do the Ishtar Gate
https://en.wikipedia.org/wiki/Ishtar_Gate
Now please do the Ishtar Gate
https://en.wikipedia.org/wiki/Ishtar_Gate
This is super cool. Performance doesn't really matter, as long as it can keep 45 fps, the experience would be good enough with motion reprojection turned on.
Thinking about it, it could be a very good tool to help cure fear of heights (or vertigo, not sure about the right term in English).
I sometimes fear I shouldn’t play driving games too much, so that I don’t get desensitised against those intrusive thoughts I sometimes get on the road. I wonder how irrational that actually is, because it feels like pretty much the same thing as your idea. I mean it doesn’t seem far fetched that a very realistic VR driving simulation with subtly “easier” physics might make you a worse driver in reality. More closely related to the fear of heights, we also jump from greater heights in video games than we would in real life, and much more frequently. I practically never have a reason to jump down from anywhere, so I could imagine making some wrong estimations when, after years of doing it in VR, I’m somehow faced with a real-life situation.
If you haven't played it, I highly recommend Richie's Plank Experience [0].
The main focus of the game is you take an elevator to the top of a building, there is a plank an you jump off it.
I've found depending on the age of the participant the experience ranges from light fun to harrowing encounter with our own fears of death. I've seen people that, if it's their first time in VR, can take nearly an hour to finally do it.
0. https://www.oculus.com/experiences/quest/1642239225880682/
The main focus of the game is you take an elevator to the top of a building, there is a plank an you jump off it.
I've found depending on the age of the participant the experience ranges from light fun to harrowing encounter with our own fears of death. I've seen people that, if it's their first time in VR, can take nearly an hour to finally do it.
0. https://www.oculus.com/experiences/quest/1642239225880682/
Whoa that's amazing.
Yeah, it's "just" 50 fps, but ... would you just look at it!
4 years ago there was an equally fascinating "Paris apartment" VR demo, but the movement was limited and the amount of work that went into appears to have been orders of magnitude more - https://www.benoitdereau.com/
Go back 10 years and seeing this back then would've been absolutely bananas.
Makes you wonder what we'll have in 10 years from now.