RP2050 would work, as it also has the necessary built-in peripherals. ESP32 would add wifi and better ESPHome support. I suggest you stay away from the ESP8266, as it needs to do PWM in software and struggles with the 25kHz output frequency.
From my experience: ESP32/RP2040 work without additional circuitry which works with most fans, but for protection, you want to add level shifters. Not all fans pull up the PWM pin to 3.3V, the spec allows for 5V.
Shameless plug, hopefully this is allowed here.
I built something like that, that allows fan control via WiFi. First I built it only for myself, but since the spare boards from PCBA quickly sold, I decided to keep it stocked.
Short gist: 12V Input, ESP32S2, ESPHome-based. Has 4 PWM-fan outputs, onboard temperature & humidity sensor and Qwiic expansion port.
If you want to build it by yourself, the schematic is in the hardware folder. And if you don't want to use ESPHome - there is no firmware lock, you find the pin assignment on the product page and can write your own firmware if you so desire.
As far as I can find, Plex does not support AMD iGPU for transcoding. Jellyfin will work, but support seems rather spotty. For other AI/ML work, it seems like ROCm is up and coming, but support - e.g. for Frigate object detection - is still a work in progress, especially for newer chips.
The appeal for me was the "it just works" factor. It's a compact unit and setup was easy. Every self-built solution would either be rather large (factor for me) and more difficult to set up. And I think, that's what has kept Synology alive for so long. It allows entry level users to get into the selfhosting game with the bare minimum you need, especially if transcoding (Plex/Jellyfin) is mentioned.
As an anecdote, I've had exactly this problem when buying my last NAS some time ago. It was DS920+, DS923+ vs. QNAP TS-464. The arguments for QNAP were exactly what you write. Newer chip, 2.5G NICs, PCIe Slot, no NVMe vendor lock-in. So I bought the QNAP unit. And returned it 5 days later, because the UI was that much hot garbage and I did not want to continue using it.
Lately, the UGreen NAS series looks very promising. I'm hearing only good things about their own system AND (except for the smallest 2-bay solution) you can install TrueNAS. It mostly sounds too good to be true. Compact, (rather) powerful and flexible with support for the own OS.
As the next player, with mixed feelings about support, the Minisforum N5 Units also look promising / near perfect. 3x M.2 for Boot+OS, 5 HDD slots and a PCIe low-profile expansion slot.
I would say cases like the Fractal Node 304 or something like the HL4 / HL8 from 45homelab would be the best suited candidates.
Regarding mainboards - models from CWWK with lots of SATA ports have been trendy lately. But there are reports of problems. The other options are either using some obscure supermicro mainboards with lots of ports or using a HBA for expansion.
I want to mention a possible middle ground here: UGreen NAS Storage. All but the smallest model come the OS on a seperate M.2 drive. If you disable the watchdog in BIOS, you can use the models like a normal Server This would give you:
The M.2 slots are connected rather slow, but good enough for OS/app drives.
For example, my plan for the next NAS would be the 4-Slot N100 variant with TrueNAS. One M.2 SSD for boot, Two M.2 SSDs for Apps/Server duties in mirroring and the 4 drives in Raid-Z1.
From my experience: ESP32/RP2040 work without additional circuitry which works with most fans, but for protection, you want to add level shifters. Not all fans pull up the PWM pin to 3.3V, the spec allows for 5V.
Shameless plug, hopefully this is allowed here.
I built something like that, that allows fan control via WiFi. First I built it only for myself, but since the spare boards from PCBA quickly sold, I decided to keep it stocked.
Short gist: 12V Input, ESP32S2, ESPHome-based. Has 4 PWM-fan outputs, onboard temperature & humidity sensor and Qwiic expansion port.
The ESPHome code & schematic is on Github: https://github.com/zeroflow/wifi-fancontroller
If you want one, it's available on Elecrow for $35,99: https://www.elecrow.com/wifi-fancontroller1.html
If you want to build it by yourself, the schematic is in the hardware folder. And if you don't want to use ESPHome - there is no firmware lock, you find the pin assignment on the product page and can write your own firmware if you so desire.