Comment by mlsu

Really insightful.

I'm a little more cautious though. I think GPT will be way more integrated, simply because it's useful. Stalinist language was artificial, in the sense that it was basically imposed on you from outside for no good reason. When you wanted to get real stuff done (either talking to close friends, being productive with colleagues, etc) you wouldn't use socialist newspeak because it got in the way. GPT will be imposed by the outside world, but it's actually a useful thing to be able to converse with a language model; you'll do it every day at work, when buying things, when using your phone/PC.

And also, unlike in USSR times, so much of our communication is online and visible. It would not surprise me if we develop a model that can train continuously on the firehose. Text is small. Data rate of every person on earth speaking simultaneously:

- 150 words per minute spoken

- 150 words × (5 characters/word + 1 space) = 150 × 6 = 900 characters per minute

- 1 byte per char = 900 bytes/min = 15 bytes/sec

- 15 bytes / sec * 8,000,000,000 people speaking continuously = 120 gigabytes/second

That's a lot but it's not even the bandwidth of a single consumer GPU.