Skip to yearly menu bar Skip to main content


Poster

Dataset Distillation by Automatic Training Trajectories

Dai Liu · Jindong Gu · Hu Cao · Carsten Trinitis · Martin Schulz

Strong blind review: This paper was not made available on public preprint services during the review process Strong Double Blind
[ ]
Tue 1 Oct 1:30 a.m. PDT — 3:30 a.m. PDT

Abstract: Dataset Distillation is used to create a concise, yet informative, synthetic dataset that can replace the original dataset for training purposes. Some leading methods in this domain prioritize long-range matching, involving the unrolling of training trajectories with a fixed number of steps ($N_{S}$) on the synthetic dataset to align with various expert training trajectories. However, traditional long-range matching methods possess an overfiting-like problem, the fixed step size $N_{S}$ forces synthetic dataset to distortedly conform seen expert training trajectories, resulting in a loss of generality—especially to those from unencountered architecture. We referred to this as the Accumulated Mismatching Problem (AMP). And we propose a new approach, Automatic Training Trajectories (ATT), which dynamically and adaptively adjusts trajectory length $N_{S}$ to address the AMP. Our method outperforms existing methods particularly in tests involving cross-architectures. Moreover, owing to its adaptive nature, it exhibits enhanced stability in the face of parameter variations. \keywords{Dataset Distillation \and Task-Specific Dataset Compression \and Dataset Condensation}

Live content is unavailable. Log in and register to view live content