Convolutional neural networks (CNNs) have achieved effective results in image denoising tasks. However, CNN architectures for image denoising tasks are mainly designed manually, which not only relies on CNN-related professional knowledge, but also requires adjustment to different datasets for competitive performance. Algorithms for automatically evolving CNN architectures have been proposed, but most of them are designed for solving image classification tasks and consume considerable computational time and resources. To address these issues, an efficient automatically evolving CNN architecture algorithm for image denoising tasks using genetic algorithm is proposed, which is called fast block-based evolutionary denoising CNN (FBE-DnCNN). In FBE-DnCNN, a genetic encoding strategy based on both deep and wide net blocks is designed to effectively represent the image denoising CNNs for automatic architecture design. With the purpose of solving time-consuming and resource-dependent problems, the partial dataset-based technology is used. A novel refined fitness evaluation method with prior knowledge on parameters of CNNs is designed to improve reliability. For better feature extraction of shallow network layers, convolutional operation, prevention of overfitting, and improvement of the representational capacity, the Feature Block, Transition Block, Dropout Block, and SENet module are introduced in FBE-DnCNN to generate problem-specific search space. With block-specific crossover and mutation, a local search near the good solution is implemented to find better solutions. Experiments show that FBE-DnCNN can evolve distinguished image denoising CNNs with deep and wide architectures in a very short time. FBE-DnCNN achieves competitive performance for the image denoising tasks with different noise levels compared to the traditional approaches, state-of-the-art CNN-based algorithms, and NAS-based methods.