Accurate solar irradiation prediction is directly linked to estimating the power output of photovoltaic systems, making it essential for optimal operation and management of solar facilities. Over the past few years, there has been an increasing interest in the domain of solar irradiance prediction, where numerous long- short-term memory (LSTM) and convolutional neural network (CNN)-based models have emerged as promising approaches for solar irradiance forecasting. This paper aims at developing a simple and accurate LSTM and CNN-based global horizontal irradiance (GHI) forecasting predictor. More specifically, the present study examines the impact of several parameters on the prediction accuracy, including the forecasting timestep length, dataset size, number of LSTM units, number and size of CNN filters, and the input configuration of the forecasting model. Using the Los Angeles solar Irradiance datasets, several LSTM and CNN-based yearly and seasonal models are explored for predicting one-hour ahead solar irradiance. It has been observed through the simulation results that using seasonal models enhances prediction accuracy and that prediction quality increases in stable datasets free of contradictory examples. In CNN-based forecasting models, accuracy is clearly influenced by the prediction timestep length, kernel size, and max pooling, where, after several tests, a prediction lag N_steps=72hours\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$N\_steps=72\;hours$$\end{document} and a max pooling = 1 have been found to be the optimal parameter values. While stacked LSTM layers improve accuracy, stacking CNN layers does not necessarily lead to improvements. More specifically, the obtained results show that an annual stacked LSTM-based model with three descending layers (32, 16, and 8 units) outperforms all other examined forecasting structures, achieving root mean square error (RMSE) and mean absolute error (MAE) values of approximately 37.08 W/m2 and 15.23 W/m2, respectively. However, a seasonal CNN GHI input forecasting model without max pooling might surpass LSTM-based schemes in terms of efficiency.