> Cafe goers tasted the cookies and provided feedback via a survey. Survey results were aggregated and the results were sent back to Vizier.
This, from the paper, is a great start on measuring deliciousness. But if you go to that effort, why not provide a benchmark? Such as the existing chocolate chip cookie recipe or the average scores from 10 random chocolate chip cookie recipes?
As a reader, I might suspect that performance was not that great compared to baselines but that the authors were able to mask it by making this qualitative claim ("the cookies were delicious") -- which I'm sure was still true!
I noted this section when I read the paper as well...
This is cute, but "in the authors' opinions, delicious" does not contribute anything to scientific research. Yes, you can use black-box optimization for cookie recipes. No, you should not make any sort of performance claims nor talk about "significant" improvement unless you back it up.