denoiSplit: a method for joint microscopy image splitting and unsupervised denoising

In this work, we tackle the novel and challenging task of joint image splitting and unsupervised denoising. This dual approach is especially critical in fluorescence microscopy, where noise significantly hinders the analysis of the scientific content hidden in the acquired data. Image splitting involves dissecting an image into predefined semantic structures. Our work builds upon uSplit, the current state-of-the-art method for this task. However, we show that uSplit struggles with noise removal, inadvertently distributing the noise across the split output channels. Here we introduce denoiSplit, a new method that preserves the strengths of uSplit while integrating an unsupervised denoising subtask. This integration results in effective semantic image unmixing but ensures useful results even in the presence of image noise. A key innovation in denoiSplit is the use of specifically formulated noise models into our approach and the suitable adjustment of the KL-divergence loss for the high-dimensional hierarchical latent space we are training. We perform qualitative and quantitative evaluations and compare results to existing benchmarks demonstrating the effectiveness of using denoiSplit: a single network to do both splitting and denoising.