throwmeaway222 14 days ago

That's all different now with agentic which was not really a big thing until the end of 2024. before they were doing 1 request, now they're doing hundreds for a given task. the reason oai/azure win over locally run models is the parallelization that you can do with a thinking agent. simultaneous processing of multiple steps.

nickpsecurity 15 days ago

You hit the nail on the head. Just gotta add the up to $10 billion investment from Microsoft to cover pretraining, R&D, and inference. Then, they still lost billions.

One can serve a lot if models if allowed to burn through over a billion dollars with no profit requirement. Classic, VC-style, growth-focused capitalism with an unusual, business structure.

DoctorOetker 14 days ago

Due to batching, inference is profitable, very profitable.

Yet undoubtedly they are making what is declared a loss.

But is it really a loss?

If you buy an asset, is that automatically a loss? or is it an investment?

By "running at a loss" one can build a huge dataset, to stay in the running.

  • dbbk 12 days ago

    How batched can it really be though if every request is personalised to the user with Memory?

    • DoctorOetker 12 days ago

      Imagine pipelineing lots of infra-scale GPU's, naive inference would need all previous tokens to be shifted "left" or from the append-head to the end-of-memory "tail", which would require a huge amount of data flow for the whole KV cache etc. Instead of calling GPU 1 the end-of-memory and GPU N the append-head, you keep the data static and let the role rotate like a circular buffer. So now for each new token inference round, the previous rounds end-of-memory GPU becomes the new append-head GPU. The highest bandwidth is keeping data static.

gregoriol 14 days ago

With infinite resources, you can serve infinite users. Until it's gone.

93po 15 days ago

they would be break-even if all they did was serve existing models and got rid of everything related to R&D

  • mperham 15 days ago

    Have they considered replacing their engineers with AI?

  • Invictus0 15 days ago

    An AI lab with no R&D. Truly a hacker news moment

    • nl 15 days ago

      The unspoken context there is that the inference isn't the thing causing the losses.

      • gitremote 15 days ago

        Inference contributes to their losses. In January 2025, Altman admitted they are losing money on Pro subscriptions, because people are using it more than they expected (sending more inference requests per month than would be offset by the monthly revenue).

        https://xcancel.com/sama/status/1876104315296968813

    • hn92726819 15 days ago

      I think you maybe have misunderstood the parent (or maybe I did?). They're saying you can't compare an individual's cost to run a model against OpenAI's cost to run it + R&D. Individuals aren't paying for R&D, and that's where most of the cost is.

  • knowitnone2 15 days ago

    they are not the only player so getting rid of R&D would be suicide

    • Lionga 14 days ago

      It is now 3 years in where I was told AI will replace engineers in 6 month. How come all the AI companies have not replaced engineers?