You may already be aware, but SE marks all commits making those kinds of changes as '[Editorial]', so it is generally trivial to use their tooling to build your own high-quality ebook without any of the editorial changes.
Mods using AI to clone voice actors' voices without their consent or explicitly against their wishes has certainly been controversial in the Skyrim modding community.