As you realized, you have to add bunch of qualifiers to make consent works: age-of-consent, condition-of-consent, etc. The defense then shifted into defending those qualifiers.
From system design perspective, it operates on lower level rule-based design. This is weaker. I understand there's debate internally about Humanae Vitae, and it's good. But at least it gets the core design stronger by operating on higher level goal/paradigm constraint (the purpose of procreation)
He is "correct" in a way. It is the inevitable logical extension of current society's view on sexual relationship (consent based). This is just a repeat of what happened in 70s with a bunch of intellectuals: Foucault, Derrida, Althusser, et al argued that age of consent should be abolished too [1]. Seems like the most rational men always come to same conclusion.
Society slowly unravel that consent is such a fragile concept, and they have to resort to some finite number (age) to strengthen that stance. But, as the most rational amongst us found out, those numbers are indefensible too.
A better and more defensible stance is the one Catholic's have (Humanae Vitae).