Jonathan and Elliot chatted with Eben Upton today about what's coming for the CM5 and Pi500, where Raspberry Pi is heading with a possible IPO, and whether the RP1 has a future beyond Raspberry Pi boards.
<shameless_plug>
Check out fwknop, https://www.cipherdyne.org/fwknop/
It's one solution to this problem. Instead of answering with garbage, it allows for keeping the firewall closed/default drop stance. It's port knocking, but with real cryptography instead of just relying on hitting port numbers, and does it with just a single packet.
</shameless_plug>
Full disclosure: I'm one of the Fwknop devs.
Ssh does a fingerprint verification and establishes a secure channel before it does the password exchange. Avoiding replay attacks can be a challeng as well. Putting strong authentication in a single packet is deceptively nontrivial. It can be done, but at that point you'd be reimplementing Fwknopd. Additionally, Fwknop can protect more than just ssh, and do fancy things like providing access to a machine without a public IP address.