Exactly, IPFS allows data storage and retrieval for Dapps to also be done through a decentralized network. That's the whole point of a D-app, not falling back on someone's central servers. The performance and scalability limitations of Ethereum make it virtually unusable as a database, application data definitely needs to be taken off-chain.
Also, the IPFS system contains IPNS, which allows users to host mutable content. Rather than navigating to the hash of a file, you're retrieving what's been signed by the publisher's key. So it's more flexible than just static content.
Email your customers, telling them to change their passwords, and link to some info about the leak. (in case they don't visit your website and miss seeing the security alert banner)
Also, the IPFS system contains IPNS, which allows users to host mutable content. Rather than navigating to the hash of a file, you're retrieving what's been signed by the publisher's key. So it's more flexible than just static content.