The problem you’re referring to arose precisely due to lack of formalism. It was a problem back when mathematicians were toying with naive set theory, one not based on axioms, but instead on intuitive terminology. Moving to axiomatic, more formal, set theory solved it.