Comment by elzbardico
Comment by elzbardico a day ago
It is more like Apple have no need to spend billions on training with questionable ROI when it can just rent from one of the commodity foundation model labs.
Comment by elzbardico a day ago
It is more like Apple have no need to spend billions on training with questionable ROI when it can just rent from one of the commodity foundation model labs.
Well, they stopped.
Culture is overrated. Money talks.
They did things far more complicated from an engineering perspective. I am far more impressed by what they accomplished along TSMC with Apple Silicon than by what AI labs do.
Is Apple silicon really that impressive compared to LLMs? Take a step back. CPUs have been getting faster and more efficient for decades.
Google invented the transformer architecture, the backbone of modern LLMs.
> Google invented...
"Google" did? Or humans who worked there and one who didn't?
https://www.wired.com/story/eight-google-employees-invented-...
In any case, see the section on Jakob Uszkoreit, for example, or Noam Shazeer. And then…
> In the higher echelons of Google, however, the work was seen as just another interesting AI project. I asked several of the transformers folks whether their bosses ever summoned them for updates on the project. Not so much. But “we understood that this was potentially quite a big deal,” says Uszkoreit.
Worth noting the value of “bosses” who leave people alone to try nutty things in a place where research has patronage. Places like universities, Xerox, or Apple and Google deserve credit for providing the petri dish.
It’s such a commodity that there are only 3 SOTA labs left and no one can catch them. I’m sure it’ll be consolidated further in the future and you’re going to be left with a natural monopoly or duopoly.
Apple has no control over the most important change to tech. They have control to Google.
Really, don't believe benchmarks as gospel. Chinese models are pretty much competitive with offerings from Anthropic, OpenAI or Google. Meta is currently at a disadvantage, but I believe they will find their mojo and soon be competitive again.
Frankly, a lot of times I prefer using GLM 4.6 running on Cerebras Inference, than having to deal with the performance hiccups from Claude. For most practical purposes, I've seen no big penalty in using it compared to Opus 4.5, even the biggest qwen-coder models are pretty much competitive.
Between me and the company I work for, I spend some serious money with AI. I use it extensively in my main job, on two side projects that I have paying customers for, and for graduate school work. I can tell you that there quite a few more SOTA models around than what the benchmarks tell you.
> It’s such a commodity that there are only 3 SOTA labs left and no one can catch them.
No one can outpace them in improving the SOTA, everyone can catch up to them. Why are open-weight models perpetually 6 months behind the SOTA? Given enough data harvested from SOTA models you can eventually distill them.
The biggest differentiator when training better models are not some new fancy architectural improvements (even the current SOTA transformer architectures are very similar to e.g. the ancient GPT-2), but high quality training data. And if your shiny new SOTA model is hooked into a publicly available API, guess what - you've just exposed a training data generator for everyone to use. (That's one of the reasons why SOTA labs hide their reasoning chains, even though those are genuinely useful for users - they don't want others to distill their models.)
I don't know why people automatically jump to Apple's defense on this.... They absolutely did spend a lot of money and hired people to try this. They 100% do NOT have the open and bottom-up culture needed to pull off large scale AI and software projects like this.
Source: I worked there