It takes 12 people to use the 18k Sphere camera(petapixel.com)
petapixel.com
It takes 12 people to use the 18k Sphere camera
https://petapixel.com/2023/10/20/darren-aronofsky-says-it-takes-12-people-to-use-the-18k-sphere-camera/
71 comments
For those also curious as I was, some numbers: 4k is ~8 megapixels. 8k is ~35MP. 12k is ~80MP. 18k is ~170MP. Except for Big Sky the resolution is not 2:1 aspect ratio like DCI or other formats, it's 1:1 18k, so that's 340MP.
Big Sky is shooting 120FPS at 10-bit (or 60FPS at 12 bit). That's 408Gbps/51GBps.
A RX 7700 XT has 432Gbps memory bandwidth and a 4070 504Gbps... maybe that could render this? Unless you need to both write to memory and then read from it to render, and now you are up to a flagship RX7900 XTX or 4090 (960/1008Gbps). Which with their 24GB memory can fit a whopping 0.47s of video in memory.
Now consider the NVMe storage array you ll need to get video at 408Gbps. PCIe 5.0 NVMe drives can supposedly hit 14GBps, so "only" 4 drives going full tilt should be adequate to let you play this back at realtime speeds.
Big Sky is shooting 120FPS at 10-bit (or 60FPS at 12 bit). That's 408Gbps/51GBps.
A RX 7700 XT has 432Gbps memory bandwidth and a 4070 504Gbps... maybe that could render this? Unless you need to both write to memory and then read from it to render, and now you are up to a flagship RX7900 XTX or 4090 (960/1008Gbps). Which with their 24GB memory can fit a whopping 0.47s of video in memory.
Now consider the NVMe storage array you ll need to get video at 408Gbps. PCIe 5.0 NVMe drives can supposedly hit 14GBps, so "only" 4 drives going full tilt should be adequate to let you play this back at realtime speeds.
Ones might wager that they aren't going to use consumer-grade GPUs for this.
Here's a pro-level card, for instance: https://www.amd.com/en/products/professional-graphics/amd-ra...
That says 864 GB/s, which is pretty good.
Here's a pro-level card, for instance: https://www.amd.com/en/products/professional-graphics/amd-ra...
That says 864 GB/s, which is pretty good.
My bad, I failed to read the correct units!! The throughput figures in the desktop cards was actually in GB/s, not Gbps!!!
So any of these would be more than fine. That makes a lot more sense, considering what it takes to make a complex 3d scene.
So any of these would be more than fine. That makes a lot more sense, considering what it takes to make a complex 3d scene.
> What’s even more impressive is the camera can run completely separate from this recording technology as long as it is connected through its cable system, this includes distances of up to a reported mile away.
That's not that impressive, Blackmagic cameras have been able to run on fiber since 2018 [1].
The bandwidth for the Sphere cam however, that is insane. 500 Gbit/sec, that used to be the domain of entire ISP's core switches, and now that kind of performance is (effectively) handheld-capable.
[1] https://www.blackmagicdesign.com/media/release/20180201-02
That's not that impressive, Blackmagic cameras have been able to run on fiber since 2018 [1].
The bandwidth for the Sphere cam however, that is insane. 500 Gbit/sec, that used to be the domain of entire ISP's core switches, and now that kind of performance is (effectively) handheld-capable.
[1] https://www.blackmagicdesign.com/media/release/20180201-02
> 500 Gbit/sec, that used to be the domain of entire ISP's core switches
And then you realize that the latest datacenter Ethernet switch chips have 51 Tbit/sec agregate non-blocking capacity (128 ports of 400Gb/s, or similar setups), and a single AI cluster can use hundreds of these switches.
And then you realize that the latest datacenter Ethernet switch chips have 51 Tbit/sec agregate non-blocking capacity (128 ports of 400Gb/s, or similar setups), and a single AI cluster can use hundreds of these switches.
A consumer grade GeForce 4070 has that much memory bandwidth, and there are more expensive models with even more.
It’s an almost unfathomable number but at the same time it is well within the realm of commodity hardware. We are living in the future.
It’s an almost unfathomable number but at the same time it is well within the realm of commodity hardware. We are living in the future.
Memory bandwidth is easier to achieve because of q short distance between elements on the motherboard.
0.5Tbps light to chip!!!!!
That's insane.
That's insane.
> Aronofsky’s 18K resolution film plays back at Sphere at a blistering-fast 60 frames per second, more than twice the speed of the typical 24p motion picture, and is about half a petabyte in size, or about 500 terabytes. Aronofsky explains that the movie is about 32GB of data per second, or nearly 2,000 GB a minute.
I can already see the PC gamers demanding 18K 144hz monitors.
I can already see the PC gamers demanding 18K 144hz monitors.
The human eye can only see at 60 frames per second
edit: Apparently, you don't joke about frames per second.
edit: Apparently, you don't joke about frames per second.
Let’s for a moment accept this as fact. Now consider you have a 60fps screen and 60fps natural vision. The periodicity of video frame updates and “vision frames” are not in any way synchronized. Meaning the image appears less “smooth” than if you had a higher refresh screen.
Lastly, have you ever tried a 144hz and 60hz display side by side? The difference is very obvious.
Lastly, have you ever tried a 144hz and 60hz display side by side? The difference is very obvious.
There's no synchronized refresh in the retina though.
Imagine pixels being randomly updated continuously in time, but each individual pixel has a refresh rate of 60Hz.
The overall perceived 'frame rate' for this parallel random update will be much higher than 60Hz: Suppose 25% of the pixels in an object need to change to register a perceptible difference: then the effective framerate for noticing changes in the object will be ~240Hz.
Imagine pixels being randomly updated continuously in time, but each individual pixel has a refresh rate of 60Hz.
The overall perceived 'frame rate' for this parallel random update will be much higher than 60Hz: Suppose 25% of the pixels in an object need to change to register a perceptible difference: then the effective framerate for noticing changes in the object will be ~240Hz.
The fastest proven seems to be 220hz but 240hz is the maximum theoretical for humans.
if you are referring to this:
> The USAF, in testing their pilots for visual response time, used a simple test to see if the pilots could distinguish small changes in light. In their experiment a picture of an aircraft was flashed on a screen in a dark room at 1/220th of a second. Pilots were consistently able to "see" the afterimage as well as identify the aircraft. This simple and specific situation not only proves the ability to percieve 1 image within 1/220 of a second, but the ability to interpret higher FPS.
that is not a test of FPS, it's a test of a few photons in a dark room and the afterimage affect of the retina (or is it also the optic nerve and the visual cortex?), potentially aka "blurring".
I'm not taking a position on framerates, I'm taking a position on interpreting the data.
> The USAF, in testing their pilots for visual response time, used a simple test to see if the pilots could distinguish small changes in light. In their experiment a picture of an aircraft was flashed on a screen in a dark room at 1/220th of a second. Pilots were consistently able to "see" the afterimage as well as identify the aircraft. This simple and specific situation not only proves the ability to percieve 1 image within 1/220 of a second, but the ability to interpret higher FPS.
that is not a test of FPS, it's a test of a few photons in a dark room and the afterimage affect of the retina (or is it also the optic nerve and the visual cortex?), potentially aka "blurring".
I'm not taking a position on framerates, I'm taking a position on interpreting the data.
Ah, you're right, maybe it's even higher than 240hz then, I mean, they're making 540Hz monitors now so there must be something to it.
> they're making 540Hz monitors now so there must be something to it.
They also keep making placebo pills, sometimes labeled as such, sometimes not. They also keep printing both the bible and the koran, which to my beginner's understanding can't both be true. I wouldn't take the existence of something as evidence that there is something to it!
They also keep making placebo pills, sometimes labeled as such, sometimes not. They also keep printing both the bible and the koran, which to my beginner's understanding can't both be true. I wouldn't take the existence of something as evidence that there is something to it!
[deleted]
What makes it a joke?
"the claim that the eye can only see 60 FPS has been proven false in numerous online threads"
Thanks lol.
Obviously we're a long ways (if ever) from having such theaters available widely enough to be more then location tourist attractions, but VR will almost certainly eventually make such video accessible for home viewing as well. And the march of tech will undoubtedly eventually bring down camera costs to "very high" vs "stratospheric one of a kind high", and in turn mean more works get made with it.
So from an artistic point of view, I'm a little curious long term what film makers might come up with when they get a format where it's impossible to see it all in one go. I mean yes, lots of the best films reward multiple viewings and as a practical matter it can be very hard to notice little details the first time around. Same thing for that matter with static images. But still, you can physically see everything there at once, if multiple people watch it everyone sees the same thing. Whereas in a sphere format with the work exceeds your physical field of view it'll be truly impossible to experience all of the film in a single go no matter how closely you pay attention.
While it'd be very challenging to use well and I'm sure a lot of the format will be documentary type of stuff, nevertheless it seems likely there will be works eventually that really lean into it and they'll genuinely offer different experiences in the same work depending on where you choose to focus. A medium that has already long done this in a way is video games, where it's not at all uncommon for there to be games that offer multiple mutually exclusive paths; picking one means you will not ever see certain content in that run. That can be cheap sometimes, but can also used by a creative producer/writer in interesting ways. Will open a new dimension for film making perhaps.
So from an artistic point of view, I'm a little curious long term what film makers might come up with when they get a format where it's impossible to see it all in one go. I mean yes, lots of the best films reward multiple viewings and as a practical matter it can be very hard to notice little details the first time around. Same thing for that matter with static images. But still, you can physically see everything there at once, if multiple people watch it everyone sees the same thing. Whereas in a sphere format with the work exceeds your physical field of view it'll be truly impossible to experience all of the film in a single go no matter how closely you pay attention.
While it'd be very challenging to use well and I'm sure a lot of the format will be documentary type of stuff, nevertheless it seems likely there will be works eventually that really lean into it and they'll genuinely offer different experiences in the same work depending on where you choose to focus. A medium that has already long done this in a way is video games, where it's not at all uncommon for there to be games that offer multiple mutually exclusive paths; picking one means you will not ever see certain content in that run. That can be cheap sometimes, but can also used by a creative producer/writer in interesting ways. Will open a new dimension for film making perhaps.
Yes as you say, video games do some of that. There is also the nascent art of 360 videos (sometimes erroneously called “VR videos”). They are developing their own conventions and experimenting with what works and what does not.
As you said, the medium makes it easier to make a movie that still has something new to enjoy on the second or third watch.
Another thing is discovering the best clues to direct the viewer to look where you want them to (sound, light, movement, but also having the other characters look that way).
As you said, the medium makes it easier to make a movie that still has something new to enjoy on the second or third watch.
Another thing is discovering the best clues to direct the viewer to look where you want them to (sound, light, movement, but also having the other characters look that way).
Great article. The Sphere is such a ridiculous, hilarious and fantastic project. We should be build one in every city.
> The venue will house the world’s highest resolution LED screen: a 160,000 square-foot display plane that will wrap up, over, and behind the audience
It would be great for watching horror movies.
It would be great for watching horror movies.
How does viewing a movie on the sphere work? From my understanding it’s a sphere that displays things on the outside. Wouldn’t this mean you would only be able to see only a small part of the screen at any given time?
> From my understanding it’s a sphere that displays things on the outside
It's got two screens! One inside and one outside.
The interior screen is 16k by 16k pixels (the 18k camera resolution affords some cropping for image stabilization etc).
The exterior screen doesn't have much in the way of specs other than area, but I'd expect it is comparably lower pixel density.
It's got two screens! One inside and one outside.
The interior screen is 16k by 16k pixels (the 18k camera resolution affords some cropping for image stabilization etc).
The exterior screen doesn't have much in the way of specs other than area, but I'd expect it is comparably lower pixel density.
There is a similar set of screens on the inside
Don't know if this is the right topic to be asking this but here it goes anyways: is there a 360º interactive version of the U2 show at the sphere?
No - it's not 360 degrees. They have a few interesting and cool visuals for something like 5 or 6 songs, more basic or boring uses of the screen otherwise. I didn't sense any haptics off the chair nor could tell if they activated the scents either.
I was there for opening night. Also went to the Sphere Experience. The experience includes the movie mentioned here and has some interesting/cool stuff. It's loud and rumbled a ton (seat haptics) and was almost to the point of "should I put my earplugs in?" but I didn't need to. It was an interesting experience. At first it seemed like they were just projecting a small rectangle movie off of it and I was about to flip out, but they eventually got the whole screen going for it once you begin the journey part.
If you go I highly recommend going in 200 or 300 level. So you get the dome over your head enough; if you're in 400 level it doesn't go all the way behind you. Also, there is an overhang on 100 level so you can be closer to the artists but it's actually considered limited visibility (a lot of people were notified after the fact and quite unhappy)
The venue itself is capable of some amazing stuff. U2 has a lot of social media sharing showing amazing things, but it wasn't that way the whole show. It'll be exciting to see who and what winds up using it and how. I can imagine some pretty amazing EDM shows there.
I was there for opening night. Also went to the Sphere Experience. The experience includes the movie mentioned here and has some interesting/cool stuff. It's loud and rumbled a ton (seat haptics) and was almost to the point of "should I put my earplugs in?" but I didn't need to. It was an interesting experience. At first it seemed like they were just projecting a small rectangle movie off of it and I was about to flip out, but they eventually got the whole screen going for it once you begin the journey part.
If you go I highly recommend going in 200 or 300 level. So you get the dome over your head enough; if you're in 400 level it doesn't go all the way behind you. Also, there is an overhang on 100 level so you can be closer to the artists but it's actually considered limited visibility (a lot of people were notified after the fact and quite unhappy)
The venue itself is capable of some amazing stuff. U2 has a lot of social media sharing showing amazing things, but it wasn't that way the whole show. It'll be exciting to see who and what winds up using it and how. I can imagine some pretty amazing EDM shows there.
This sounds very nice. I'll try to visit by the end of the year. Thanks for sharing
Keep having these vibe that: simulating Sphere feels like a good goal that could show off VR well.
Main hardware comment: 4 way right angled Lemo cables. Must be fancy.
No one has seriously shot 24p in 2 decades but Aaron. Time to catch up man... It takes his crew 12 people lol, this isn't much worse then shooting existing 8 and 12k which at most is a single well trained DIT. I looked at this setup, its not any different, hype article.
I think as a general rule when you’re using technical specs to argue that one of the greatest practitioners of their generation of whatever art you’re describing is doing it wrong, it’s probably worth sitting back and contemplating what point you’re trying to make and whether you’ve maybe missed something important along the way.
No one has seriously shot 24p in 2 decades but Aaron
What is this supposed to mean? Movies are 24fps. Here is a short list of the movies that are not 24fps.
https://en.wikipedia.org/wiki/List_of_films_with_high_frame_...
at most is a single well trained DIT
Is DIT supposed to mean 'Document Image Transformer' here? This is photography, not upscaling.
What is this supposed to mean? Movies are 24fps. Here is a short list of the movies that are not 24fps.
https://en.wikipedia.org/wiki/List_of_films_with_high_frame_...
at most is a single well trained DIT
Is DIT supposed to mean 'Document Image Transformer' here? This is photography, not upscaling.
Haha, let me grab a random list too... yours is particularly bad because you didn't understand my point. Im talking about physical limitations of film, vs digital, vs human perception and the trade offs of budget, crew, and equipment.
Shooting on a camera locked at 24fps is not equal to shooting 24fps. Spend the next 20 years doing it and come back to me.
Shooting on a camera locked at 24fps is not equal to shooting 24fps. Spend the next 20 years doing it and come back to me.
you didn't understand my point
I still don't. You said something that wasn't true in any way, so I'm still not sure what the point is. Your list doesn't seem to make any sense or have any connection to what you're saying either.
Im talking about physical limitations of film, vs digital, vs human perception and the trade offs of budget, crew, and equipment.
If you say so, but what your comment actually said was that no one has shot 24fps in 20 years, which is nonsense.
Then you said something about 18k not being impressive because someone could shoot at 8k and scale it up, but even that wasn't very clear.
I still don't. You said something that wasn't true in any way, so I'm still not sure what the point is. Your list doesn't seem to make any sense or have any connection to what you're saying either.
Im talking about physical limitations of film, vs digital, vs human perception and the trade offs of budget, crew, and equipment.
If you say so, but what your comment actually said was that no one has shot 24fps in 20 years, which is nonsense.
Then you said something about 18k not being impressive because someone could shoot at 8k and scale it up, but even that wasn't very clear.
I think you're expecting 3 decades of cinema tech knowledge to be dumped on you in a comment.
No one shoots true 24. Anymore. Nor does any modern studio want to.
That doesnt mean content isnt finalized at 24...
18k is impressive, thats a-lot of heat. 12 people running a standard high res RAID, is not impressive and puffed up for press.
No one shoots true 24. Anymore. Nor does any modern studio want to.
That doesnt mean content isnt finalized at 24...
18k is impressive, thats a-lot of heat. 12 people running a standard high res RAID, is not impressive and puffed up for press.
No one shoots true 24. Anymore. Nor does any modern studio want to.
This is not true, where are you getting this information? Converting a high frame rate to a lower frame rate is not like resampling a high res image to a lower res image. You have to interpolate or throw out frames, both of which are terrible compared to just shooting in the frame rate you want.
This is not true, where are you getting this information? Converting a high frame rate to a lower frame rate is not like resampling a high res image to a lower res image. You have to interpolate or throw out frames, both of which are terrible compared to just shooting in the frame rate you want.
Unless it’s an integer multiple. Shrug.
Nope, if it's an integer multiple you still have to throw out frames and those frames could have been part of the motion blur of other frames instead. It isn't the same because instead of taking a single exposure for half the time the frame elapses you would have two frames, each exposed for a fourth of that same time.
Or you just interpolate between frames if you want the same motion blur. It’s just like downsampling from 4k to 1080p. The process is clean since you can directly map 2 frames to 1 frame (or, 4 pixels to 1 pixel for resolution conversion).
Absolutly not. Once again, interpolating frames is not "just like downsampling" an image.
Things move and obscure each other, you will get artifacts.
Again, there is also motion blur and missing information. A frame exposure might be for half the time of the frame (which is common). Two frames would then be have their exposures broken up over time. Imagine four units of time. One frame fills up two next to each other and the next two are blank. Two frames would fill up one, the next would be blank, the next would be filled, the next would be blank.
Anyone telling you that you can "just" interpolate frames has never done it before.
Things move and obscure each other, you will get artifacts.
Again, there is also motion blur and missing information. A frame exposure might be for half the time of the frame (which is common). Two frames would then be have their exposures broken up over time. Imagine four units of time. One frame fills up two next to each other and the next two are blank. Two frames would fill up one, the next would be blank, the next would be filled, the next would be blank.
Anyone telling you that you can "just" interpolate frames has never done it before.
> Shooting on a camera locked at 24fps is not equal to shooting 24fps.
Care to explain?
I know quite a bit about camera technology and don't have the slightest idea of what you mean with this sentence.
Care to explain?
I know quite a bit about camera technology and don't have the slightest idea of what you mean with this sentence.
The only generous interpretation I can see is that Aronofsky uses extremely innovative techniques yet has long publicly voiced that he prefers to use analog film.
For instance some of his signature style is short shots taken with cameras mounted on actors in dark conditions which vastly easier to realize with digital yet he has repeatedly chosen to use film.
Whilst it is perfectly consistent that what would change his mind is an extremely powerful digital camera; it is also possible that the resulting PR spin causes some frustration to a generation of professionals who have been working in digital for 20 years and not been given shiny 18K cameras to play with.
He only did that in Pi. Maybe thats all he is known for, but I wouldn't call it his style anymore.
And Requiem but yes that was only one example, he has an amazing and varied output.
It has nothing to do with tech. The same reason Daren shoots film is why I still shoot film, at any speeds. Since almost no one chooses to shoot true 24fps anymore, everything is sampled down and so everything I watch looks like shit. Sharp, but jittery shit.
Use the tool for the situation. I actually respect the tech behind the sphere and 18k cameras, because it's purpose built and I imagine looks lovely in its native speed and fov.
But Daren... this piece is gross man.
Use the tool for the situation. I actually respect the tech behind the sphere and 18k cameras, because it's purpose built and I imagine looks lovely in its native speed and fov.
But Daren... this piece is gross man.
You still haven't defined what "true 24fps" is, and the idea that it "has nothing to do with tech" makes no sense. 24 fps is a technical term, no more and no less.
And everyone here is left baffled because it seems to be some personal notion of yours, as opposed to what it is universally and clearly understood to mean.
I literally have no idea what you mean when you talk about things being "sampled down" or "sharp, jittery shit".
The only thing sampled down in most modern TV/film production is a usually a 4K source down into 1080p or less for distribution. But I can't possibly imagine you're objecting to that. And frame rates aren't "sampled down".
Meanwhile, by "jittery" I can only guess you mean shutter speed, not frame rate? Normal shutter angle is 180°, and that has been standard for many, many decades, identical between film and digital, and which has never been jittery. Sometimes action scenes use a smaller angle (faster speed), which you're free to call jittery, but that's not the norm. Generally just for action scenes, or for occasional other artistic effect.
If you want to have any kind of productive conversation on HN, I suggest you start explaining what you actually mean rather than throwing around terms with idiosyncratic personal definitions you simply invent, that have nothing to do with how the terms are generally understood.
And everyone here is left baffled because it seems to be some personal notion of yours, as opposed to what it is universally and clearly understood to mean.
I literally have no idea what you mean when you talk about things being "sampled down" or "sharp, jittery shit".
The only thing sampled down in most modern TV/film production is a usually a 4K source down into 1080p or less for distribution. But I can't possibly imagine you're objecting to that. And frame rates aren't "sampled down".
Meanwhile, by "jittery" I can only guess you mean shutter speed, not frame rate? Normal shutter angle is 180°, and that has been standard for many, many decades, identical between film and digital, and which has never been jittery. Sometimes action scenes use a smaller angle (faster speed), which you're free to call jittery, but that's not the norm. Generally just for action scenes, or for occasional other artistic effect.
If you want to have any kind of productive conversation on HN, I suggest you start explaining what you actually mean rather than throwing around terms with idiosyncratic personal definitions you simply invent, that have nothing to do with how the terms are generally understood.
I never thought about this, but what if you shoot at 30 fps and want to produce 24? How do you map 30 onto 24 time-wise? In 30, each frame lasts 33.33ms. In 24 it’s 41.66. So if the first frame is synced, the second frame of something captured in 30 and transformed to 24 would show up 8.3ms too late. The third - 16.ms, and so on. It’s not possible to properly sync video to 24fps if it was shot at 30 without interpolation, right?
It’s commonly done the other way, actually. Converting 24fps to 30fps for converting film (typically shot at 24fps) to TV broadcast ready, which is 29.97 fps in the US/NTSC region.
That said, you can go the other way. Basically, you interlace certain frames. Wikipedia has a great article on this.
https://en.m.wikipedia.org/wiki/Three-two_pull_down#2:3
That said, you can go the other way. Basically, you interlace certain frames. Wikipedia has a great article on this.
https://en.m.wikipedia.org/wiki/Three-two_pull_down#2:3
Thanks. Then I suppose it doesn't matter if it's 30fps to 24fps or the other way around, the process is probably similar - either interpolation or interlacing.
And that brings up an interesting point - after such a process, is it possible to produce clear visuals on each frame? The image [0] on the Wikipedia article illustrates my question.
[0] https://en.m.wikipedia.org/wiki/Three-two_pull_down#/media/F...
And that brings up an interesting point - after such a process, is it possible to produce clear visuals on each frame? The image [0] on the Wikipedia article illustrates my question.
[0] https://en.m.wikipedia.org/wiki/Three-two_pull_down#/media/F...
you come across as someone who doesn’t make sense or someone who knows but doesn’t know how to explain themselves across industry borders.
you can assume everyone here is a bit technical, but not experts on film.
you can assume everyone here is a bit technical, but not experts on film.
Here ya go, random list, tho mine is better matched to my point.
https://en.m.wikipedia.org/wiki/List_of_films_shot_on_digita...
https://en.m.wikipedia.org/wiki/List_of_films_shot_on_digita...
Yeah, like that stupid one-eared Dutch painter who just did these kind of low-res pixelated paintings, unlike more realistic work like the old masters such as Rembrandt. What's his name again? I doubt anyone remembers, or if there's anything named after this doze.
Has anyone shot at 42.6666666667x24? Even earliest digital cameras had higher resolution.
[deleted]
https://petapixel.com/2023/06/12/sphere-studios-big-sky-cine...
Bits like a fiber-to-media recording system and custom GPU-based processing systems, really awesome project