Distillation

Beyond Owls: Subliminal Learning Can Transfer Learned Capabilities and Backdoors

Subliminal learning can transfer more than simple preferences. Distilling on unrelated data, a student model partially acquires a teacher's novel capability, a backdoor, and a propensity to hack in an agentic chess environment.

Read More →

Subliminal Learning: Language models transmit behavioural traits through hidden signals in data

[Nature 4/2026] Distillation can lead to subliminal learning: the transmission of behavioural traits through semantically unrelated data. A student model trained on number sequences from a teacher with some trait learns that trait, even when references to it are rigorously removed.

Read More →