Sleep stage scoring is fundamental for the examination and analysis of sleep problems. Sleep experts score sleep by analyzing brain activity, muscle activity, and eye activity. Manual sleep stage scoring is an expert-dependent, tedious, and time-consuming process. Automatic sleep stage classification (ASSC) has gained particular attention due to sleep awareness over the last few years. In this research, ASSC is proposed using deep learning methods using a single-channel electroencephalogram (EEG) signal. EEG signals contain lots of information about brain functions during sleep. The EEG features were extracted using the convolution neural network (CNN) method. Different deep learning architectures are investigated using the raw EEG epochs and their time-frequency spectra using short-time Fourier transform (STFT) and stationary wavelet transform (SWT). The end-to-end classification pipeline classifies 30-s EEG epochs into five sleep stages by extracting features from raw EEG epoch and their time-frequency representations. Deep learning models give good classification accuracy compared to the current state-of-the-art methods. It gives an overall accuracy of (Fpz-Cz: 83.7%, Pz-Oz: 83.5%), (Fpz-Cz: 85.6%, Pz-Oz: 83.6%), and (Fpz-Cz: 85.7%, Pz-Oz: 83.2%) on 20-fold subjectwise cross-validation (CV) of the sleep-EDF-v1 dataset using 1-D CNN, SWT-CNN, and STFT-CNN, respectively. The subjectwise CV performed shows more consistent performance across different subjects. The model size and performance are investigated to develop a less complex and smaller deep learning model.