Comment by nomel Comment by nomel 2 days ago 3 replies Copy Link View on Hacker News The "alignment tax".
Copy Link behnamoh 2 days ago Collapse Comment - Exactly. Even this paper shows how model creativity significantly drops and the models experience mode collapse like we saw in GANs, but the companies keep using RLHF...https://arxiv.org/abs/2406.05587 Reply View | 2 replies Copy Link nomel 2 days ago Parent Collapse Comment - A nice talk about a researcher's experience/benchmarks with raw GPT-4, before and after RLHF:https://www.youtube.com/watch?v=qbIk7-JPB2c Reply View | 1 reply Copy Link behnamoh 2 days ago Root Parent Collapse Comment - Yup, I remember that! Microsoft removed that part of the paper. Reply View | 0 replies
Copy Link nomel 2 days ago Parent Collapse Comment - A nice talk about a researcher's experience/benchmarks with raw GPT-4, before and after RLHF:https://www.youtube.com/watch?v=qbIk7-JPB2c Reply View | 1 reply Copy Link behnamoh 2 days ago Root Parent Collapse Comment - Yup, I remember that! Microsoft removed that part of the paper. Reply View | 0 replies
Copy Link behnamoh 2 days ago Root Parent Collapse Comment - Yup, I remember that! Microsoft removed that part of the paper. Reply View | 0 replies
Exactly. Even this paper shows how model creativity significantly drops and the models experience mode collapse like we saw in GANs, but the companies keep using RLHF...
https://arxiv.org/abs/2406.05587