create-react-app offers an 'eject' option, which makes all the configuration visible and adjustable. Ejecting can be used as a starting point from which you can further modify and customize a project.
Upgrading from 3 to 4 is not required. Version 3 is still maintained. It's not ideal since there will probably be no future major 3 versions, but no one is forced to make a 'big breaking change' strictly speaking.
> We intend to keep supporting the 3.x branch indefinitely (published separately on npm to aid in migration), although there will likely not be any future major versions based on that code. 4.0 is the future, but we won't leave you hanging if you want to stick with 2.x/3.x.