Skin cancer is a dangerous disorder that is caused by an unchecked proliferation of aberrant skin cells that produce genetic mutations on the skin. When ultraviolet (UV) radiation from sunshine or tanning beds damages the skin cells in a way that leads to their rapid multiplication and formation of malignant tumours. Certain forms of skin cancer metastasize along nerves. This results in tingling, pain, itching, numbness, or a sensation like ants crawling under the skin. Skin cancer spreads to deeper tissues, including cartilage, muscle, and bone. The earlier prediction of skin lesions increases the possibility of survival rate. However, during diagnosis, an anomalous finding is made, and the condition is diagnosed as cancer. To overcome these challenges, a novel deep learning (DL) technique is developed in this research article for categorizing skin cancer by employing the proposed Fractional Gannet Humming Optimization_Deep Convolutional Neural Network (FGHO_DeepCNN). Initially, the input skin cancer image is subjected to the image pre-processing phase. The image pre-processing is done by the bilateral filter. Afterwards, skin lesion segmentation is carried out using an encoder-decoder with Dense-Residual block (DRB), which is trained by the Fractional Gannet optimization algorithm (FGOA). Here, the FGOA is formed by the integration of the Fractional Calculus (FC) concept with the Gannet Optimization Algorithm (GOA). Thereafter, image augmentation is done to enlarge the segmented image using geometric and colour space transformation. After that, a feature extraction process is conducted to obtain the significant features, like Completed Local Binary Pattern (CLBP), Gray Level Co-occurrence Matrix (GLCM), Local Vector Pattern (LVP), Significant Local Binary Pattern (SLBP) and CNN features. Finally, skin cancer detection is done using DeepCNN, which is tuned by the proposed FGHO. Here, the proposed FGHO is formed by the combination of FGOA with the artificial hummingbird algorithm (AHA). The experimental outcomes of the proposed FGHO_DeepCNN approach attained a better Positive Predictive Value (PPV), Negative Predictive Value (NPV), True Positive Rate (TPR), and True Negative Rate (TNR), and accuracy with values of 89.80%, 89.40%, 94.50%, 94.00% and 93.40% respectively. The employed FGHO_DeepCNN has acquired excellent performance, thus achieving a PPV of 91.68%, NPV of 88.46%, TPR of 91.68%, TNR of 91.23% and accuracy of 90.67% for dataset 2. In dataset 3, the FGHO_DeepCNN obtained superior performance than other techniques with a PPV of 90.56 %, NPV of 90.36%, TPR of 90.95 %, TNR of 90.87%, and accuracy of 90.15 %. The practical significance of the proposed FGHO_DeepCNN approach is that it is widely used in dermatology clinics and hospitals to detect skin cancer.