Developing Novel Activation Functions in Time Series Anomaly Detection with LSTM Autoencoder

被引:2
作者
Mercioni, Marina Adriana [1 ]
Holban, Stefan [1 ]
机构
[1] Politehn Univ Timisoara, Dept Comp Sci, Timisoara, Romania
来源
IEEE 15TH INTERNATIONAL SYMPOSIUM ON APPLIED COMPUTATIONAL INTELLIGENCE AND INFORMATICS (SACI 2021) | 2021年
关键词
activation function; anomaly; autoencoder; Deep Learning; detection; hyperbolic tangent; learnable; architecture; ReLU; Talu; timeseries;
D O I
10.1109/SACI51354.2021.9465604
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Our proposal consists of developing two novel activation functions in time series anomaly detection, they have the capability to reduce the validation loss. The approach is based on a current activation function in Deep Learning, a very intensive field studied over time, in order to find the most suitable activation in a neural network. In order to achieve this purpose. we used an LSTM (Long Short-Term Memory) Autoencoder architecture, using these two novel functions to see the network's behavior through introducing them. The key point in our proposal is given by the learnable parameter, assuring more flexibility within the network in weights' updates, in fact, this property being more powerful than a predefined parameter that will bring a constraint due to its limit. We tested our proposal in comparison to other popular functions such as ReLU (Linear Rectifier Unit), hyperbolic tangent (tanh), Into activation function. Also, the novelty of this paper consists of taking into consideration of piecewise behavior of an activation function in order to increase the performance of a neural network in Deep Learning.
引用
收藏
页码:73 / 78
页数:6
相关论文
共 50 条
  • [31] LSTM-Based VAE-GAN for Time-Series Anomaly Detection
    Niu, Zijian
    Yu, Ke
    Wu, Xiaofei
    [J]. SENSORS, 2020, 20 (13) : 1 - 12
  • [32] Forecasting and Anomaly Detection approaches using LSTM and LSTM Autoencoder techniques with the applications in supply chain management
    Nguyen, H. D.
    Tran, K. P.
    Thomassey, S.
    Hamad, M.
    [J]. INTERNATIONAL JOURNAL OF INFORMATION MANAGEMENT, 2021, 57 (57)
  • [33] Enhancing autoencoder models for multivariate time series anomaly detection: the role of noise and data amount
    Sefati, Seyedeh Tina
    Razavi, Seyed Naser
    Salehpour, Pedram
    [J]. JOURNAL OF SUPERCOMPUTING, 2025, 81 (04)
  • [34] Time-based Anomaly Detection using Autoencoder
    Salahuddin, Mohammad A.
    Bari, Md Faizul
    Alameddine, Hyame Assem
    Pourahmadi, Vahid
    Boutaba, Raouf
    [J]. 2020 16TH INTERNATIONAL CONFERENCE ON NETWORK AND SERVICE MANAGEMENT (CNSM), 2020,
  • [35] Bi-LSTM Autoencoder SCADA based Unsupervised Anomaly Detection in Real Wind Farm Data
    Chokr, Bassel
    Chatti, Nizar
    Charki, Abderafi
    Lemenand, Thierry
    Hammond, Mohammad
    [J]. 2024 IEEE INTERNATIONAL CONFERENCE ON PROGNOSTICS AND HEALTH MANAGEMENT, ICPHM 2024, 2024, : 174 - 183
  • [36] Anomaly Detection of the Brake Operating Unit on Metro Vehicles Using a One-Class LSTM Autoencoder
    Kang, Jaeyong
    Kim, Chul-Su
    Kang, Jeong Won
    Gwak, Jeonghwan
    [J]. APPLIED SCIENCES-BASEL, 2021, 11 (19):
  • [37] A Novel Multi-dimensional Time-series Data Anomaly Detection Model Based on Generative Adversarial Network Aided Autoencoder
    Gong, Zejun
    Liu, Qiang
    Ding, Jinliang
    Wang, Xiaobo
    Wang, Peng
    [J]. IFAC PAPERSONLINE, 2023, 56 (02): : 8085 - 8090
  • [38] SA2E-AD: A Stacked Attention Autoencoder for Anomaly Detection in Multivariate Time Series
    Li, Mengyao
    Li, Zhiyong
    Yang, Zhibang
    Zhou, Xu
    Li, Yifan
    Wu, Ziyan
    Kong, Lingzhao
    Nai, Ke
    [J]. ACM TRANSACTIONS ON KNOWLEDGE DISCOVERY FROM DATA, 2024, 18 (07)
  • [39] Unsupervised Multivariate Time Series Data Anomaly Detection in Industrial IoT: A Confidence Adversarial Autoencoder Network
    Shan, Jiahao
    Cai, Donghong
    Fang, Fang
    Khan, Zahid
    Fan, Pingzhi
    [J]. IEEE OPEN JOURNAL OF THE COMMUNICATIONS SOCIETY, 2024, 5 : 7752 - 7766
  • [40] A Novel Method for Time Series Anomaly Detection based on Segmentation and Clustering
    Huynh Thi Thu Thuy
    Duong Tuan Anh
    Vo Thi Ngoc Chau
    [J]. PROCEEDINGS OF 2018 10TH INTERNATIONAL CONFERENCE ON KNOWLEDGE AND SYSTEMS ENGINEERING (KSE), 2018, : 276 - 281