That's simple. If it is a model that was trained with only that person's data and is therefore connected to it, it is PII (personally identifiable data) and must be deleted.
If the model aggregates the data of multiple people and is disconnected from the individual it does not constitute PII and can be kept.
At least it is not a software I use an like, that is devoured by Oracle this time.
(Of course Oracle makes great stuff. In a company as big as Oracle, there are always niches that create great things. But most offerings are dreadful. And most open source acquisitions did not fare especially well under Oracle)