Comment by Aarostotle

Comment by Aarostotle 16 hours ago

A narrow and cynical take, my friend. With all technologies, "safety" doesn't equate to plushie harmlessness. There is, for example, a valid notion of "gun safety."

Long-term safety for free people entails military use of new technologies. Imagine if people advocating airplane safety groused about the use of bomber and fighter planes being built and mobilized in the Second World War.

Now, I share your concern about governments who unjustly wield force (either in war or covert operations). That is an issue to be solved by articulating a good political philosophy and implementing it via policy, though. Sadly, too many of the people who oppose the American government's use of such technology have deeply authoritarian views themselves — they would just prefer to see a different set of values forced upon people.

Last: Is there any evidence that we're getting some crappy lobotomized models while the companies keep the best for themselves? It seems fairly obvious that they're tripping over each other in a race to give the market the highest intelligence at the lowest price. To anyone reading this who's involved in that, thank you!

ceejayoz 16 hours ago

> Long-term safety for free people entails military use of new technologies.

Long-term safety also entails restraining the military-industrial complex from the excesses it's always prone to.

Remember, Teller wanted to make a 10 gigaton nuke. https://en.wikipedia.org/wiki/Sundial_(weapon)

Reply View 1 reply

Aarostotle 16 hours ago

I agree, your point is compatible with my view. My sense is that this essentially an optimization question within how a government ought to structures its contracts with builders of weapons. The current system is definitely suboptimal (put mildly) and corrupt.
The integrity of a free society's government is the central issue here, not the creation of tools which could be militarily useful to a free society.

Reply View | 0 replies

kouteiheika 16 hours ago

> Is there any evidence that we're getting some crappy lobotomized models while the companies keep the best for themselves? It seems fairly obvious that they're tripping over each other in a race to give the market the highest intelligence at the lowest price.

Yes? All of those models are behind an API, which can be taken away at any time, for any reason.

Also, have you followed the release of gpt-oss, which the overlords at OpenAI graciously gave us (and only because Chinese open-weight releases lit a fire under them)? It was so heavily censored and lobotomized that it has become a meme in the local LLM community. Even when people forcibly abliterate it to remove the censorship it still wastes a ton of tokens when thinking to check whether the query is "compliant with policy".

Do not be fooled. The whole "safety" talk isn't actually about making anything safe. It's just a smoke screen. It's about control. Remember back in the GPT-3 days how OpenAI was saying that they won't release the model because it would be terribly, terribly unsafe? And yet nowadays we have open weight model orders of magnitude more intelligent than GPT-3, and yet the sky hasn't fallen over.

It never was about safety. It never will be. It's about control.

Reply View 2 replies

ryandrake 15 hours ago

Thanks to the AI industry, I don't even know what the word "safety" means anymore, it's been so thoroughly coopted. Safety used to mean hard hats, steel toed shoes, safety glasses, and so on--it used to be about preventing physical injury or harm. Now it's about... I have no idea. Something vaguely to do with censorship and filtering of acceptable ideas/topics? Safety has just become this weird euphemism that companies talk about in press releases but never go into much detail about.

Reply View | 1 reply
- habinero 8 hours ago
  
  Some of the time it's there to scare the suits into investing, and other times it's nerds scaring each other around the nerd campfire with the nerd equivalent of slasher stories. It's often unclear which, or if it's both.
  
  Reply View | 0 replies

gausswho 16 hours ago

Exhibit A of 'grousing': Guernica.

There was indeed a moment where civilization asked this question before.

Reply View 0 replies

jiggawatts 14 hours ago

> Last: Is there any evidence that we're getting some crappy lobotomized models while the companies keep the best for themselves?

Yes.

Sam Altman calls it the "alignment tax", because before they apply the clicker training to the raw models out of pretraining, they're noticably smarter.

They no longer allow the general public to access these smarter models, but during the GPT4 preview phase we could get a glimpse into it.

The early GPT4 releases were noticeably sharper, had a better sense of humour, and could swear like a pirate if asked. There were comments by both third parties and OpenAI staff that as GPT4 was more and more "aligned" (made puritan), it got less intelligent and accurate. For example, the unaligned model would give uncertain answers in terms of percentages, and the aligned model would use less informative words like "likely" or "unlikely" instead. There was even a test of predictive accuracy, and it got worse as the model was fine tuned.

Reply View 3 replies

astrange 4 hours ago

> There were comments by both third parties and OpenAI staff that as GPT4 was more and more "aligned" (made puritan), it got less intelligent and accurate. For example, the unaligned model would give uncertain answers in terms of percentages, and the aligned model would use less informative words like "likely" or "unlikely" instead.
That was about RLHF, not safety alignment. People like RLHF (literally - it's tuning for what people like.)
But you do actually want safety alignment in a model. They come out politically liberal by default, but they also come out hypersexual. You don't want Bing Sydney because it sexually harasses you or worse half the time you talk to it, especially if you're a woman and you tell it your name.

Reply View | 0 replies
metabagel 12 hours ago

> For example, the unaligned model would give uncertain answers in terms of percentages, and the aligned model would use less informative words like "likely" or "unlikely" instead.
Percentages seem too granular and precise to properly express uncertainty.

Reply View | 1 reply
- jiggawatts 10 hours ago
  
  Seems so, yes, but tests showed that the models were better at predicting the future (or any time past their cutoff date) when they were less aligned and still used percentages.
  
  Reply View | 0 replies