TiltDiff: Tilted Weight-Space Diffusion for Neural Network Generation

Generating functional neural-network weights from trained model collections is a central problem in weight-space learning. We introduce TiltDiff, a performance-tilted latent diffusion framework for neural-network generation. TiltDiff tokenizes network weights, encodes them into a compact latent space with a Transformer autoencoder, and uses a U-Net-based diffusion model to synthesize latent representations that decode into functional parameters. To favor stronger models, we weight the denoising loss by validation accuracy, biasing the learned distribution toward higher-performing regions of weight space. Experiments show that TiltDiff improves predictive performance, robustness to random parameter masking, and representational diversity over prior weight-generation methods. We further combine diffusion U-Net connectivity with attribution analysis to identify class-specific decision pathways. These pathways exhibit emergent correspondence across independently generated models despite differing raw parameters, and pathwaylevel masking verifies their importance for target-class prediction. Our results show that performance-tilted diffusion generates accurate, robust, diverse, and structurally interpretable neural-network weights.