Overthinking GIS (2024)(scottsexton.co)
scottsexton.co
Overthinking GIS (2024)
https://scottsexton.co/post/overthinking_gis/
44 comments
You still have to send people out into the field. And you had to know what you waned to survey ahead of time. During the war that sometimes meant sending survey teams behind enemy lines. Satellite remote sensing changed all of that.
You also want to set a marker so that when you come back later you can set your theodolite up in the exact same position as you did 20 years ago.
There's a lot of surveyors marks hammered into things out in the world.
There's a lot of surveyors marks hammered into things out in the world.
If you will permit me one cautionary point, given that the deadly Texas flash floods just happened.
The geomorphic reason that there is flat land next to a river is because it is the flood plane. That kind of land looks tempting, but can be deadly. Obviously, GIS analysis could add some kind of distance metric from a river to a usability score, but the analyst needs to know about the problem first.
Just a bit of "ground truthing" from a professional geoscientist.
The geomorphic reason that there is flat land next to a river is because it is the flood plane. That kind of land looks tempting, but can be deadly. Obviously, GIS analysis could add some kind of distance metric from a river to a usability score, but the analyst needs to know about the problem first.
Just a bit of "ground truthing" from a professional geoscientist.
Here in Germany we have open maps like the following that were created by the government to inform of potential flood areas: https://umweltatlas.bayern.de/mapapps/resources/apps/umwelta...
Everyone that buys a house or land knows these maps and checks them since the last deadly flood happened a few years ago. I cannot believe that the US decided to defund the offices that help to gather these kind of information.
Everyone that buys a house or land knows these maps and checks them since the last deadly flood happened a few years ago. I cannot believe that the US decided to defund the offices that help to gather these kind of information.
Those flood maps are common across the first world, sometimes called a "flood overlay" and there is usually also a "bushfire overlay" if you live in Australia. They help describe risk, in the insurance sense, and are presumably used for both insurance and risk management.
In the US, the canonical flood map is maintained by the Federal Emergency Management Agency (FEMA).
https://www.fema.gov/flood-maps
https://www.fema.gov/flood-maps
of course there are flood maps here in the USA. The Federal government does some survey work, but in the US system, it is the county that is responsible for maintaining basic property and land use maps. Every State in the US is composed of county (or parish or some New England thing to be complete). The US Federal government is not defunding "everything". Also note that some parts of the US do not cooperate very much with the Federal government (mostly near Idaho area, others?). Texas regularly discusses becoming their own country, so of course Texas has its own mapping.
Crucially, the thing they was recently de-funded was not mapping but the federal agency that does hydrology. Hydrology being the science of how water flows in the environment around us.
For example, using GIS data combined with meteorological forecasts, a hydrological model may predict water levels of rivers or flood plains. If nobody is around to run the hydrological model on today’s data, you’re not getting your flood warning today.
For example, using GIS data combined with meteorological forecasts, a hydrological model may predict water levels of rivers or flood plains. If nobody is around to run the hydrological model on today’s data, you’re not getting your flood warning today.
Any architect and urban planner is aware of the problems with flatlands near river beds. It's the politicians who decide to ignore the science, known since the times of the Romans.
Yes. Government subsidized insurance bears much of the blame.
In the US, 3DEP is driving towards high coverage of extremely accurate elevation data, in areas where it has coverage you wouldn't need to use a distance metric really.
Working with elevation data in the US is a pleasure compared to most other countries.
Freely available Lidar and derived products (via the USGS API) for nearly the entire country makes working with elevation data a pleasure.
Freely available Lidar and derived products (via the USGS API) for nearly the entire country makes working with elevation data a pleasure.
unfortunately, the thing that needs to be overcome is government regulations and compliance.
The past 30 years have yielded tons of new hazard maps, but communities refuse to implement them in any meaningful way because of capitalism.
So it really doesnt matter how good our analysis and predictive powers are if the policies are never enforced.
The past 30 years have yielded tons of new hazard maps, but communities refuse to implement them in any meaningful way because of capitalism.
So it really doesnt matter how good our analysis and predictive powers are if the policies are never enforced.
Government-subsidized insurance is the opposite of capitalism.
If the market were allowed to operate, no one would be able to afford these houses on the coast. And in many cases that would be a good thing. Instead, the politicians have decided that beachfront property is some kind of fundamental right that we should all collectively pay for.
If the market were allowed to operate, no one would be able to afford these houses on the coast. And in many cases that would be a good thing. Instead, the politicians have decided that beachfront property is some kind of fundamental right that we should all collectively pay for.
I think you are sadly correct. Collectivization of beach house insurance, but not health insurance, seems like a rather apt microcosm of contemporary America.
Yes. We complain about health insurance nonstop, but what we VOTE for is beach house insurance. With both our votes and our dollars. Over and over again.
Everyone wants to believe that they will one day own a house on the beach. Nobody wants to believe that they will one day need the best medical care.
Everyone wants to believe that they will one day own a house on the beach. Nobody wants to believe that they will one day need the best medical care.
s/flood plane/floodplain/
(Sorry for the typo.)
(Sorry for the typo.)
The USGS provides exactly what the author was looking for, free of charges, no computation required, no download required:
https://elevation.nationalmap.gov/arcgis/services/3DEPElevat...
Preview in QGIS: https://ibb.co/B5T49YBj
https://elevation.nationalmap.gov/arcgis/services/3DEPElevat...
Preview in QGIS: https://ibb.co/B5T49YBj
Yeah the moment you reach for pixel level processing tricks in GIS you need to pause and rtfm.
Grade calculations are right in qgis docs and part of their lessons, and are also top Google hits.
https://docs.qgis.org/3.40/en/docs/training_manual/rasters/t...
Even better, you'll learn about DEMs vs topo maps.
Grade calculations are right in qgis docs and part of their lessons, and are also top Google hits.
https://docs.qgis.org/3.40/en/docs/training_manual/rasters/t...
Even better, you'll learn about DEMs vs topo maps.
For more advanced terrain analysis, see GRASS GIS (https://grass.osgeo.org/). It’s a but clunky, but accessible from QGIS via a plugin. Whitebox also has some nice terrain analysis tools (https://www.whiteboxgeo.com/).
GIS is truly impressive from a user's perspective: intuitive, powerful, and full possibilities. However, from a sysadmin's point of view, hosting an ArcGIS Enterprise setup - the market leader in this space - can quickly turn into a complex and frustrating experience.
Truth. It’s a pure lock in, so many agencies are committed and sunk so much cost and effort into it that they’re never seriously going to consider any other vendor/solution. The ESRI documentation for any advanced topics falls apart quickly. I’ve spent days with their support liaisons, walking through the steps they have documented and getting different results.
Fortunately there are a lot of great alternatives to the ESRI lock-in, and FOSS has a long and rich tradition in Geo.
What are the ESRI alternatives?
QGIS for ArcMap/ArcGIS
Geoserver for their REST based servers (ArcGIS Enterprise).
GRASS and GDAL do the leg work of much processing found in the spatial toolboxes and such.
However those tools do not have the polish that ESRI kit does, but at leas you’re not paying the licensing!
- former ESRI consultant
However those tools do not have the polish that ESRI kit does, but at leas you’re not paying the licensing!
- former ESRI consultant
> However those tools do not have the polish that ESRI kit does, but at leas you’re not paying the licensing!
Arguably they have more polish, but less GUI. The open GIS world is more CLI, database, and library driven, which can be an advantage for many users trying to build high reliability or scaling systems.
Arguably they have more polish, but less GUI. The open GIS world is more CLI, database, and library driven, which can be an advantage for many users trying to build high reliability or scaling systems.
Except most GIS Analysts aren't, or don't understand how to.
That's a lot of the reason why Esri still wins a lot of contracts.
That's a lot of the reason why Esri still wins a lot of contracts.
Potentially this is something Clude Code, Cursor and other interactive LLM tools will disrupt. I watched at work a former GIS consultant without software engineering background delve into novel Python stack with Claude and hot damn … they are delivering value.
> hosting an ArcGIS Enterprise setup can quickly turn into a complex and frustrating experience
I’ve heard that sentiment before but never understood. Can you elaborate?
I’ve heard that sentiment before but never understood. Can you elaborate?
Although ESRI officially states that ArcGIS Enterprise runs on both Linux and Windows, the reality is that on Linux it operates through Wine, meaning there is no true native Linux version. As a result, most organizations choose to install it on Windows Server.
When it comes to the installation itself, be prepared for a highly involved process. ESRI typically schedules two full days with your company just for the installation and you will likely need every minute of it. The setup involves a long series of steps, many of which feel reminiscent of software deployment practices from the 90s.
When it comes to the installation itself, be prepared for a highly involved process. ESRI typically schedules two full days with your company just for the installation and you will likely need every minute of it. The setup involves a long series of steps, many of which feel reminiscent of software deployment practices from the 90s.
It's an old version of Photoshop sitting on top of an old version of Access, built by people who stopped thinking about networking in the late 90s.
These days can't you use the hosted versions of all of it?
The hosted version of, for example, ArcGIS Experience Builder is so buggy and limited that you pretty much have to use the Developer Edition to build anything involving even a mild degree of complexity
ArcGIS Online has a bunch of limitations that makes it insufficient for many enterprises. I hope this will change in the future and their SaaS offering will have feature parity at some point.
I only skimmed it, but I want to point out to the author that in his slope equation, the slope is denoted by m, not x.
But yes, using the Laplacian seems natural as a metric for what he wants to measure. Coming from a physics background though, it felt weird being explained in terms of pixels in a picture what a laplacian is...
But yes, using the Laplacian seems natural as a metric for what he wants to measure. Coming from a physics background though, it felt weird being explained in terms of pixels in a picture what a laplacian is...
Found a nice list of GIS datasets here that seems mostly under nice licenses
https://freegisdata.rtwilson.com/
https://freegisdata.rtwilson.com/
Nothing fancy new for a geographer. You do stuff like this in the first years in university
And that is why these articles are great to share, because most people aren‘t geographers and do indeed learn something new
The context of the writer's experience in the relevant field is also arguably important for readers.
I enjoyed the article, but what might not be obvious to readers outside the profession is that the writer does not actually work in this field. They are a clever tech, but a layperson in geospatial terms.
I enjoyed the article, but what might not be obvious to readers outside the profession is that the writer does not actually work in this field. They are a clever tech, but a layperson in geospatial terms.
Neat. Great to see old-school _actual analysis_, I was worried when I read 'LLM' in the text. Instead, I learned something -- the Laplace operator. The OP's explanation is considerably easier to understand than Wikipedia's :)
Cool article! I'm pretty familiar with the GIS offerings and the USGS's site but it was a fun read. I had to block the sidebar that took up half the screen though. I really don't need half of my screen taken up by contact information while I'm trying to read an article. At least UBlock lets me block it fully and then the site just expands normally. Wish the author would've overthought their design..
Can someone recommend a good tutorial on how to work with GIS data? I downloaded a bunch of files from openstreetmap but could not make any sense of it.
GIS is amazing, but the first and second world wars saw a lot of advances in cartography. I'd say #1 advancement goes to the near-perfecting of portable Theodolites. It's a telescope which allows you to look at one point, then look at another, and very accurately understand the angles between them. From that, with a little walking and Pythagoras, you can figure out distances, and thereby create accurate maps with relatively minimal effort and human-error opportunities.
I say "relatively minimal effort" because technique the portable theodolite replaced was sending battalions of civil servants out into the countryside with 50ft long chains and asking them to walk the length, height and breadth of the country.