‘This Is Nuts.’ An OpenAI Insider Explains Why He Quit.

we saw evidence that they were thinking about an evaluation and how to create an evidence trail that was going to get them a good grade and not necessarily reflect how they were really "thinking"

One of the things that we sometimes see in our chain of thought is the model will say, hum I wonder if I am being evaluated right now, and when we see that, it's really scary because what it implies is that the model might now that is being tested some of time

[...] It's an alien mind. [...] It is a thing we grew. We did not engineer it. We engineered the systems around it that grew it and try to keep it safe, we grew a mind

Ilya was still working at OpenAI and he once briefed the global affairs team [...] about how he believed that our future was merging with machines, that this was the ultimate triumph of capital over labor ...