KATANA: Knowledge-Aligned Topology-Aware Neural Agents for RL-Driven Vision-Language Model Compression

Vision-language models (VLMs) achieve remarkable multi-modal comprehension, yet their massive parameter counts impede de-ployment on resource-limited hardware. Traditional pruning techniquesrely on manual heuristics that frequently disrupt delicate cross-modalsynergies, leading to substantial performance declines. To overcome this,we introduce KATANA (Knowledge-Aligned Topology-Aware NeuralAgents), a reinforcement learning framework that autonomously discoversexecutable pruning algorithms. Guided by a synthesized multi-objectivereward signal and safely evaluated within an isolated sandbox, an LLM-driven agent iteratively evolves innovative compression strategies. Ourflagship discovered algorithm, KIRI (Kernel-Integrated ReconstructionIterator), utilizes a cubic sparsity scheduler alongside a novel Dual-NormActivation (DNA) importance metric that dynamically fuses weight mag-nitudes, structural regularizations, and data-dependent activation profiles.Extensive evaluations across four diverse VLM architectures (LLaVA-1.5,BLIP-2, Qwen2.5, and Llama-3.2-Vision) on MSCOCO, Flickr30k, andNoCaps benchmarks demonstrate KATANA’s superiority. At an aggres-sive 70% sparsity, KIRI delivers a measured 2.8× inference speedup whileconsistently outperforming 16 state-of-the-art baselines, including recentadaptive and hybrid methods such as SCOPE and GSOP. KATANA thusintroduces an automated and extensible framework for VLM compressionthat preserves vision-language coherence for practical deployment.