I’ve recently compared WebP and AVIF with the reference encoders (and rav1e for lossy AVIF), and for similar quality, WebP is almost instant while AVIF takes more than 20 seconds (1MP image).
JXL is not yet widely supported, so I cannot really use it (videogame maps), but I hope its performance is similar to WebP with better quality, for the future.
You have to adjust the CPU used parameter, not just quality, for AVIF. Though it can indeed be slow it should not be that slow, especially for a 1mp image. The defaults usually use a higher CPU setting for some reason. I have modest infrastructure that generates 2MP AVIF in a hundred ms or so.
I tested both WebP and AVIF with maximum CPU usage/effort. I have not tried the faster settings because I wanted the highest quality for small size, but for similar quality WebP blew AVIF out of the water.
I also have both compiled with -O3 and -march=znver2 in GCC (same for rav1e's RUSTFLAGS) through my Gentoo profile.
Maximum CPU between those two libs is not really comparable though. But quality is subjective and it sounds like webp worked best for you! Just saying though, there is little benefit in using the max CPU settings for avif. That's like comparing max CPU settings on zip vs xz!
rav1e has not had an actual update in performance or quality in years since funding got dropped. Use an encoder like aom, or svt-av1.