Comment by crote

Of course, but all emails you could read have been sent at one point or another. Unless you only care about local email delivery, you're going to have to get involved with PKI.

And if your threat model is bad enough that PKI isn't good enough for IMAP, why aren't you using a VPN in the first place? Or even an airgapped network? Or PGP?