Comment by boulos

The latest GPUs and TPUs support fp8. It's a big part of the efficiency gain in the latest systems. Blackwell also supports fp4.