Deep Neural network has achieved remarkable progress in artificial intelligence. By employing hierarchical abstraction, weight sharing, and local receptive fields, CNNs are highly effective at extracting features from spatial data and capturing significant patterns and structures. However, CNNs are inherently designed for grid-like structures, like images, they may have difficulty processing sequential or temporal data and may not be able to deal with temporal dependencies well. The ability of LSTMs (Long Short-Term Memory networks) to capture long-term relationships makes them suitable for processing sequential data; additionally, applying suitable activation functions, like sigmoid and tanh, improves their capacity to model complex temporal patterns. The paper presents an innovative LSTM model that incorporates a newly introduced activation function, termed modified_sigmoid, specifically applied to the input and forget gates, offering potential improvements for capturing and preserving essential features in image data. Following that, the recently introduced LSTM model is applied to datasets including MNIST, fashion_MNIST, and brain tumors in DICOM and NIfTI formats. Fourteen activation functions are compared with the modified_sigmoid function across the datasets MNIST and fashion_MNIST. In the end, a hybrid CNN-LSTM network, integrating the modified_sigmoid function, is employed to detect brain tumors by utilizing the higher-level features. The accuracy and validation results are impressive, showing no signs of overfitting. When there are no channels in the input, the model achieves a perfect 100% accuracy, and overall accuracy is very high at 99.87% extracted by CNN-LSTM.