Semi-periodic Activation for Time Series Classification - Medeiros Júnior, José Gilberto Barbosa de; Mitri, André Guarnier de; Silva, Diego Furtado (Anais da Brazilian Conference on Intelligent Systems (BRACIS); 2024: Anais da XXXIV Brazilian Conference on Intelligent Systems, 2024)
This paper investigates the lack of research on activation functions for neural network models in time series tasks. It highlights the need to identify essential properties of these activations to improve their effectiveness in specific domains. To this end, the study comprehensively analyzes properties, such as bounded, monotonic, nonlinearity, and periodicity, for activation in time series neural networks. We propose a new activation that maximizes the coverage of these properties, called LeakySineLU. We empirically evaluate the LeakySineLU against commonly used activations in the literature using 112 benchmark datasets for time series classification, obtaining the best average ranking in all comparative scenarios.