I work in an organization that asks for innovation, but doesn't allocate any budget for R&D or investigative projects. The company also doesn't have any culture of being ok with taking big swings and accepting misses.
The project deck is fully loaded with status quo type projects, based on client demands. Teams are asked to track all of their hours toward these projects.
The subtext is, "Do the grunt work during your 40 hours that we're sure will generate revenue, and please, please, do something innovative on top of that on your own time/dime".
Garbage removal. If your trash man stops coming to haul away your waste, you've got a problem. You will then ask, "how much do I need to pay to get this service back?"
Craigslist is _the_ case study on how an ugly UI can survive over a long timeline. There are many techies and UI designers that are just disgusted at the UI of craigslist. For some, aesthetic is more important than usability. Luckily, for most, it's not.
Yeah, I never understood who those ads were aimed at. Buying a superbowl ad for a bunch of money to hit millions of consumers in hopes of a single CEO with buy authority watching the ad?
Someone mentioned in an above comment that managers look out for their own interests, just like employees. There are very few people in a company who are really acting in the best interest of the company.
So my take is that the reason companies behave irrationally with regard to talent retention is that no one "owns" that problem. Same reason any number of other important issues fall through the cracks at any given company.
And yes, I think you were spot on when you said that companies don't even know who they should be trying to retain.
Some cities are dominated by insurance/banking/healthcare companies. Their IT needs are quite different than the consumer software space.
It's more like a salesguy just landed a big fish and the big fish is making these demands to close the deal, so bolt on some new functionality to your existing platform. Ad nauseum.