Comment by minimaxir

> Sure, they have huge GPU clusters

That's a really, really big "sure."

Almost every trick to run a LLM at OpenAI's scale is a trade secret and may not be easily understood by mere mortals anyways (e.g. bare-metal CUDA optimizations)