The primary objective of Continual Category Discovery (CCD) is to automatically discover novel categories in the continuous stream of unlabelled data without experiencing catastrophic forgetting, which remains an open problem even in conventional, fully supervised continual learning. To address this challenge, we propose PromptCCD, a simple yet effective framework that utilizes Gaussian mixture model as a prompting method for CCD. At the core of PromptCCD is the Gaussian Mixture Prompting (GMP) module, which acts as a dynamic pool updating over time to guide embedding data representation and avoid forgetting during category discovery. Additionally, GMP enables on-the-fly estimation of category numbers, which allows PromptCCD to discover categories in the unlabelled data without prior knowledge of category numbers. We extend the standard evaluation metrics for Generalized Category Discovery to CCD and benchmark state-of-the-art methods using different datasets. PromptCCD significantly outperforms other methods, demonstrating the effectiveness of our approach. Code will be available.