Comment by nobunaga
So this person, as a system administrator, wants to be able to sacrifice security for his personal convenience so he can login from anywhere. Does not sound like a system administrator that actually prioritises the right things. Security, especially if its not your own system, should always come first.
You have to balance those two, because the only server that's 100% secure is the one that's powered off. Everyone does that differently. I don't see sshd with key-only auth as dangerous, but password login makes me uncomfortable. Do you drive down to the data center your server is in every time you want to access it?
"I'm using VPN"
Great now you moved the target from sshd to wireguard.