Comment by swyx 2 days ago
somebody said once we are mining "low-background tokens" like we are mining low-background (radiation) steel post WW2 and i couldnt shake the concept out of my head
(wrote up in https://www.latent.space/i/139368545/the-concept-of-low-back... - but ironically repeating something somebody else said online is kinda what i'm willingly participating in, and it's unclear why human-origin tokens should be that much higher signal than ai-origin ones)
We used a rather low number of atmospheric bombs, while we are carpet bombing the internet every day with AI marketing copy.
The eternal September has finally ended. We've now entered the AI winter. It promises to be long, dark, and full of annoyances.
We're bombing the internet into extinction. But we were way before AI. It got real bad during the SEO/monetization phase. AI was just the final nail.
Multiple people have coined the idea repeatedly, way before you. The oldest comment on HN I could find was in December 2022 by user spawarotti: https://news.ycombinator.com/item?id=33856172
Here is an even older comment chain about it from 2020: https://news.ycombinator.com/item?id=23895706
Apparently, comparing low-background steel to pre-LLM text is a rather obvious analogy.
i didnt claim to invent it.
i claimed swyx heard it through me - which he did
every human generation built upon the slop of the previous one
but we appreciated that, we called it "standing on the shoulders of giants"
> we called it "standing on the shoulders of giants"
We do not see nearly so far though.
Because these days we are standing on the shoulders of giants that have been put into a blender and ground down into a slippery pink paste and levelled out to a statistically typical 7.3mm high layer of goo.
This sounds like an Alan Kay quote. He meant that in regards to useful inventions. AI-generated spam just decreases the quality. We'd need a real alternative to this garbage from Google but all the other search engines are also bad. And their UI is also horrible - not as bad as Google, but also bad. Qwant just tries to copy/paste Google for instance (though interestingly enough, sometimes it has better results than Google - but also fewer in general, even ignornig false positive results).
Deep Research reports I think are above average internet quality, they collect hundreds of sources, synthesize and contrast them & provide backlinks. Almost like a generative wikipedia.
I think all we can expect from internet information is a good description of the distribution of materials out there, not truth. This is totally within the capabilities of LLMs. For additional confidence run 3 reports on different models.
We have two optimization mechanisms though which reduce noise with respect to their optimization functions: evolution and science. They are implicitly part of "standing on the shoulders of giants", you pick the giant to stand on (or it is picked for you).
Whether or not the optimization functions align with human survival, and thus our whole existence is not a slop, we're about to find out.
Every foundation needs some time to settle.
- Sir, this is an elevator.
You may have one point.
The industrial age was built on dinosaur slop, and they were giant.
There's a reason this is comedy:
Listen, lad. I built this kingdom up from nothing. When I started here, all there was was swamp. Other kings said I was daft to build a castle on a swamp, but I built it all the same, just to show 'em. It sank into the swamp. So, I built a second one. That sank into the swamp. So, I built a third one. That burned down, fell over, then sank into the swamp, but the fourth one... stayed up! And that's what you're gonna get, lad: the strongest castle in these islands.
[24] “Everyone then who hears these words of mine and does them will be like a wise man who built his house on the rock. [25] And the rain fell, and the floods came, and the winds blew and beat on that house, but it did not fall, because it had been founded on the rock. [26] And everyone who hears these words of mine and does not do them will be like a foolish man who built his house on the sand. [27] And the rain fell, and the floods came, and the winds blew and beat against that house, and it fell, and great was the fall of it.”
Capitalism, freedom of expression, the marketplace of ideas, democracy: at their best these things are ways to bend the wisdom of the crowds (such as it is) to the benefit of all; and their failures are when crowds are not wise.
The "slop" of capitalism is polluted skies, soil and water, are wage slaves and fast fashion that barely lasts one use, and are the reason why workplace health and safety rules are written in blood. The "slop" of freedom of expression includes dishonest marketing, libel, slander, and propaganda. The "slop" of democracy is populists promising everything to everyone with no way to deliver it all. The "slop" of the marketplace of ideas is every idiot demanding their own un-informed rambling be given the same weight as the considered opinions of experts.
None of these things contributed our social, technological, or economic advancement, they are simply things which happened at the same time.
AI has stuff to contribute, but using it to make an endless feed of mediocrity is not it. The flood of low-effort GenAI stuff filling feeds and drowning signal with noise, as others have said: just give us your prompt.
> for every theory of relativity the is the religious non-sense and superstitions of the medieval ages or today
If Einstein came up with relativity by standing on "the religious non-sense and superstitions of the medieval ages," you'd have a point.
Low background steel is no longer necessary.
"...began to fall in 1963, when the Partial Nuclear Test Ban Treaty was enacted, and by 2008 it had decreased to only 0.005 mSv/yr above natural levels. This has made special low-background steel no longer necessary for most radiation-sensitive uses, as new steel now has a low enough radioactive signature."
https://en.wikipedia.org/wiki/Low-background_steel