Projects

A large portion of safety and alignment research has focused on closed frontier models, due to their advanced capabilities. However, the gap between open and closed models is narrowing fast, and open-weight models are now being seriously considered for high-stakes applications.

Recent work has hypothesised that large language models may represent some concepts as multi-dimensional manifolds instead of single directions. This has encouraged new steering and probing methods that respect manifold geometry.

I am fascinated by how intelligent behavior emerges from the coordinated activity of networks of neurons. This page highlights my research aimed at uncovering the representations, architectures, and dynamics that are crucial for fundamental cognitive computations.

Neural activations are messy and complicated, so the behaviour of an agent often constrains a system’s computational model more tightly than measurements of its internals (Niv 2021). A large part of my research focuses on modelling the algorithm an agent is running from behaviour alone, and predicting its failure modes.