Deep learning helps researchers remove clouds from satellite images(blogs.nvidia.com)
blogs.nvidia.com
Deep learning helps researchers remove clouds from satellite images
https://blogs.nvidia.com/blog/2021/09/23/gans-satellite-images/
13 comments
I will never understand why people will write articles like this and not include any pictures.
Most likely because it doesn't work very well.
There was a related article on 'Turning satellite imagery into wall art', that had pictures of easing cloud cover from photos: https://news.ycombinator.com/item?id=28325421
The paper has some images: https://www.sciencedirect.com/science/article/pii/S147403462...
Before everyones jump on the "GAN don't generate ground truth" bandwagon, that's not a concern here.
From what I understand, this is about creating a dataset of [aerial pictures, building mask] pairs, to then train a segmentation model for urban planning and design.
They use Unity and a virtual environment (a la flight simulator I guess) to generate a bunch of samples. They then take any image that has thin cloud covers, so clouds thin enough that you can still see the true outline of the buildings below it, and use the GAN to remove the clouds. Images with thick clouds just get discarded. So in this case the information that wasn't present in the original and that gets generated by the GAN does not matter for the task at hand (such as actual color and details of the building) as only the outline matters, and it was already visible before. That just helps the segmentation model learn by normalizing images in the training dataset.
What is not clear to me, is why didn't they just remove all clouds directly in the game engine since they are running in Unity instead of relying on a GAN.
From what I understand, this is about creating a dataset of [aerial pictures, building mask] pairs, to then train a segmentation model for urban planning and design.
They use Unity and a virtual environment (a la flight simulator I guess) to generate a bunch of samples. They then take any image that has thin cloud covers, so clouds thin enough that you can still see the true outline of the buildings below it, and use the GAN to remove the clouds. Images with thick clouds just get discarded. So in this case the information that wasn't present in the original and that gets generated by the GAN does not matter for the task at hand (such as actual color and details of the building) as only the outline matters, and it was already visible before. That just helps the segmentation model learn by normalizing images in the training dataset.
What is not clear to me, is why didn't they just remove all clouds directly in the game engine since they are running in Unity instead of relying on a GAN.
"we obtain reconstructed images that are more self-consistent"
The future is here. "self consistent" = "realistic" = "believable" = "convincing lies" = "fake"
The future is here. "self consistent" = "realistic" = "believable" = "convincing lies" = "fake"
[deleted]
Except satellite maps are not meant to be not-"realstics", not-"believable", not-"fake". They are meant to help with navigation, property purchase etc. If fake images help with that better than real images, that's fine with me.
You cannot obtain ground truth from neural network guesses, even if those guesses are educated. What fills in the blank is speculation.
If you are planning to catch Osama bin Laden, then you probably can't rely on speculations.
But if you try to plan a good hiking route, guesses might help you spot something interesting which you wouldn't notice otherwise because the image is noisy.
But if you try to plan a good hiking route, guesses might help you spot something interesting which you wouldn't notice otherwise because the image is noisy.
[deleted]
There is already technology that can see through clouds. Check out Radarsat run by the Canadian Space Agency.
Are clothes next?