Meta CTO Andrew Bosworth: Our Path To Frontier AI, Renting Models, Consumer AI's Struggles

…Coding had not been a focus for us on the model, but it actually was a better out of the box at Coding than we had expected it to be. And we found early on through experiments that like actually giving it just a relatively modest number of trained kind of expertly guided examples. And we could post train the model, we could dramatically improve its competitiveness. And so when you start to like run the numbers and the math and it's like, oh, this is an incredible opportunity for us to build a Coding…

This does sound like you can post train with distillation and it works