I use mitmproxy or charles proxy with an android emulator, with an older version of Android so it allows for a man-in-the middle, without being finicky about certificates.
>>my goodness, it'd be slow and potentially flaky all via the web UI for those sites
Generally, I've had better luck using undocumented APIs for this kind of stuff. I was a heavy user of Selenium to automate many a task (I work in wholesale construction goods, tons of automations needed everywhere).
But then discovered that using internal APIs (which surprisingly don't change much) is far easier than trying to exception manage changing UIs.
Open up dev tools on your browser and watch the GETS/POSTS as you complete your daily tedium. I use Python (usually just requests and beautifulsoup is enough) to mimic the calls and I have yet to find a use case where I wasn't able to automate something online.
I'm currently developing an SDK for our 18th-century ERP system, which has no API. I've also automated getting shipping container tracking data from a various shipping lines and railroad company websites, many with complicated login processes.
In my one county alone there are 90+ municipalities, each with it's own Planning and Zoning Commission, and most with their own (varying level of) website. I'd say 5-10% don't have a website either.
In your situation, how are you getting data for when land is up for sale/zoning etc.?
currently tracking is limited either by A) type of transportation (ships, rail, trucks) or B) by the Freight Forwarding company.
If you use multiple freight forwarders, you're stuck entering data from PDFs into spreadsheets to create your own custom usable dashboard.
If you use one freight forwarder, you have access to the main journey points, either as a spreadsheet, or if they're more sophisticated, through a web app.
But I've only found one (silicon valley backed) Freight Forwarder [0] that gives the last-mile data -- e.g. last free pickup dates, pickup numbers, last free dropoff dates, return locations etc. -- through their web app.
This is critical for managing warehouse operations, especially for companies that handle their own last-mile (like we do), and it's been an absolute pain as we've scaled.
I have a very small business so can't say how it compares to running a bigger business on qb