Meta CTO Andrew Bosworth: Our Path To Frontier AI, Renting Models, Consumer AI's Struggles

And we found early on through experiments that like actually giving it just a relatively modest number of trained kind of expertly guided examples. And we could post train the model, we could dramatically improve its competitiveness.

This does sound like you can post train with distillation and it works