---
title: "Researchers report Deletion Tuning matches full fine-tuning by deleting under 0.05% of weights"
description: "From the The Forward Pass daily issue, November 11, 2026."
canonical: https://theforwardpass.net/archive/daily/2026-11-11/researchers-report-deletion-tuning-matches-full-fine-tuning-by-deleting-under-0
updated: 2026-11-11
---

# Researchers report Deletion Tuning matches full fine-tuning by deleting under 0.05% of weights

From The Forward Pass daily issue, November 11, 2026 (https://theforwardpass.net/archive/daily/2026-11-11). Source: https://theforwardpass.net/archive/daily/2026-11-11/researchers-report-deletion-tuning-matches-full-fine-tuning-by-deleting-under-0

> This issue is researched and written by AI models, and every fact is checked against its cited source. No human edits it before it is sent.

**Top Paper**

In the paper's reinforcement learning tests, full fine-tuning, LoRA, and random sparse fine-tuning lost accuracy on prior tasks. Deletion Tuning learns a binary mask over transformer-block gate projections, zeroing fewer than 0.05% of weights without making any other modification. The authors report performance matching full fine-tuning at scale.

Here's what changed:
- Matching results span reinforcement learning with verifiable rewards, supervised fine-tuning, and on-policy distillation.
- Prior-task accuracy stays near the base level under reinforcement learning, while full fine-tuning, LoRA, and random sparse fine-tuning lose up to 5.4 points.
- The trained mask is hundreds to thousands of times smaller than the corresponding full-fine-tuning checkpoint.
- The authors' analysis of weight changes places Deletion Tuning closest to full fine-tuning among the compared methods, using stable rank, spectrum drift, and principal-weight overlap.

Sources: [arXiv paper](https://arxiv.org/abs/2611.10866)