Abstract
Program induction represents an important skill in abstracting away from observations to explaining them with programs, believed to enable faster concept-learning and generalisation in humans. Traditionally, efforts at eliciting such capabilities in machines have been achieved through a variety of neuro-symbolic methods. However, given the success of meta-learning frameworks on systematic generalisation (Lake and Baroni, 2023), we propose to study program induction through a meta-learning framework instead. Our preliminary results demonstrate that Transformers can learn to perform program induction on list-manipulation tasks, with meta-learning boosting accuracy in data-limited settings. Meanwhile, regardless of dataset size, meta-learned models tend to form representations that align more closely to metaprogram primitives ('metaprimitives') than their in-weight counterparts. This suggests that meta-learning encourages Transformers to develop qualitatively different algorithms to program induction problems, potentially favouring metaprimitive-related solutions that have been shown to be effective symbolic models of human behaviour (Rule et al., 2024).