Comment by api
The phenomenon being discussed here is a type of overfitting:
https://sohl-dickstein.github.io/2022/11/06/strong-Goodhart....
The last 50 years or so of managerial practice has been a recipe for overfitting with a brutal emphasis on measuring, optimizing, and stack ranking everything.
I think an argument can be made that this is an age of overfitting everywhere.
Interesting that something similar came up recently where an AI being trained might fake alignment with training goals.