Projects | Diksha Gupta

Knowledge Removal in LLMs

Wed, 01 Apr 2026 00:00:00 +0000

A large portion of safety and alignment research has focused on closed frontier models, due to their advanced capabilities. However, the gap between open and closed models is narrowing fast, and open-weight models are now being seriously considered for high-stakes applications.

Yet, open-weight safety remains an unsolved problem. Every open-weight model has been successfully “abliterated” through refusal vector ablation, yielding models that comply on dangerous prompts, and unlearning of sensitive information remains extremely brittle to adversarial attacks.

This project was my first attempt to work on this problem (write-up). I red-teamed LUNAR, a state-of-the-art unlearning method. With the kind of access any open-weight attacker has, I could recover the “forgotten” knowledge in two ways:

using GRPO fine-tuning, the model learned to route around LUNAR’s modifications
by steering activations along a difference-in-means direction

This is why I think that open-weight safety requires a different, multi-pronged strategy from closed safety: it requires safety in the absence of guardrails, and robustness to adversarial attacks that are optimised to target activations (such as refusal ablation or activation steering) or weights (such as fine-tuning to escape safeguards). Meeting that bar demands a much closer collaboration between mechanistic interpretability and alignment - an area I am keen to work more on in the future.

Representation Structure in LLMs

Sun, 01 Mar 2026 00:00:00 +0000

Recent work has hypothesised that large language models may represent some concepts as multi-dimensional manifolds instead of single directions. This has encouraged new steering and probing methods that respect manifold geometry. These methods assume that a path along the manifold corresponds to continuous changes in the represented concept and resulting behaviour. If the geometry is instead discrete, such steering places activations off-distribution rather than capturing intermediate values. I am currently testing whether this assumption holds.

More details soon.

Talks

Aug 2026 Pivotal Spotlight, London Initiative for Safe AI (LISA) - selected
Aug 2026 Pivotal Lightning talk, UK AISI - selected

Mechanistic Attribution in Decision Circuits

Sun, 01 Feb 2026 00:00:00 +0000

I am fascinated by how intelligent behavior emerges from the coordinated activity of networks of neurons. This page highlights my research aimed at uncovering the representations, architectures, and dynamics that are crucial for fundamental cognitive computations.

Reliable Computation from Random Wiring: How can a randomly wired network produce reliable outputs? In the insect olfactory system, we showed that convergence—many variable units pooling onto one readout—recovers stereotyped responses without any learning (Nature Communications 2020).

Causal Attribution in Distributed Circuits: How do we attribute function to components of a network that integrates information over time? I studied this in the brain’s decision-making circuits, where dominant theories assumed a modular feedforward hypothesis: evidence accumulation proceeds through a chain of circuits, each region performing a distinct step.

By analysing information flow between nodes and performing causal ablations, we falsified this hypothesis: the apparent feedforward organization concealed recurrent computation, masked by network-level compensation - ablate one node and the rest of the network reorganizes to cover for it. We proposed two models of this distributed circuit:

Standard RNNs trained on the task don’t reproduce this, so I developed ablation-based training techniques that build in these redundancy and robustness properties, yielding more faithful toy models of the biological network (Neuron 2026).
A dynamical-systems model fit only to activations with architectural constraints, which correctly predicted the network’s robustness to perturbations it was never trained on (NeurIPS 2024, spotlight).

Unsupervised Discovery of Internal States: Using unsupervised deep-learning models of neuron activations, this line of work also revealed the moment a network commits to a decision: an abrupt switch from input-driven to autonomous dynamics. This phenomenon is detectabe only at the population level, not in single units (Nature 2025).

Failure Modes in Sequential Decisions

Thu, 01 Jan 2026 00:00:00 +0000

Neural activations are messy and complicated, so the behaviour of an agent often constrains a system’s computational model more tightly than measurements of its internals (Niv 2021). A large part of my research focuses on modelling the algorithm an agent is running from behaviour alone, and predicting its failure modes.

Abstract replay for efficient learning: How does a system consolidate days of learning into compositional knowledge that transfers to new tasks? The two halves of the question suggest two ingredients: compositionality requires reusable abstractions, and consolidation requires replaying - even simulating - experience offline. We proposed that the cortico-hippocampal circuits in the brain combines the two to learn efficiently and compress memories.
Spens E*, Gupta D*, Lewis E, Castegnaro A, Burgess N, Mrsic-Flogel T, Behrens TEJ (2026). Efficient learning through abstracted generative replay. Cosyne Abstracts 2026

Failure modes in sequential decisions: I showed that two failure modes of decision-making (history-dependent biases and attentional lapses) long treated as separate and unpredictable - are signatures of one underlying autoregressive computation, and can therefore be modelled and predicted (Nature Communications 2024; Best Paper Award at RLDM 2022). Along the way I showed that a widely used correction for slow drifts in decision variables distorts inference, and that the drift has to be modelled jointly with the updates instead (NBDT 2022).