Amazing work, I've really wanted to get into knowledge graph generation with LLM's for the last year but haven't found the time. Glad to see someone making good progress on the idea!
I'd imagine they're just going to leverage SDIO and a cheaper ESP32 variant (like a C3) to bring wifi to the chip. Makes sense when you start looking at EMC/EMI with high speed interfaces, adding Wifi just adds another layer to the chaos which might not be needed in a lot of applications
S3 also supports Octal PSRAM vs the single lane SPI PSRAM found on the older ESP32-CAM style boards, should be 8x the bandwidth with all else equal. So far I've only seen the octal PSRAM's available on WROOM modules and it looks likes there only support for Espressif branded Octal PSRAM chips at this point.
How are you going about generating training data?