Comment by hchja Comment by hchja a day ago 2 replies Copy Link View on Hacker News This is pretty useless in any case that doesn’t involve BFloat16 models
Copy Link spindump8930 a day ago Next Collapse Comment - bf16 is the defacto default datatype and distribution type for LLMs, which are then often eagerly quantized by users with more limited hardware. See the recent Llama releases and e.g. the H100 spec sheet (advertised flops and metrics target bf16). Reply View | 0 replies
Copy Link throwaway314155 a day ago Prev Collapse Comment - So an increasingly smaller number of cases? Reply View | 0 replies
bf16 is the defacto default datatype and distribution type for LLMs, which are then often eagerly quantized by users with more limited hardware. See the recent Llama releases and e.g. the H100 spec sheet (advertised flops and metrics target bf16).