OpenWrt founder here; you've pretty much hit the nail on the head.
Due to size constraints most platforms use a split filesystem consisting of a large read only root filesystem and a smaller overlay for changes. The read only filesystem allows for much better compression ratios but it also means if you attempt to replace an individual package you'll end up with two copies, original in the RO and new in the overlay.
Reflashing an OpenWrt device generally consists of migrating a minimal environment over to a ramdisk, rewriting the kernel and read only portions and then writing a backup of the config files to the overlay. On failure you're likely to be stuck in whatever rescue mode the bootloader provides.