Convolutional channel modulator for transformer and LSTM networks in EEG-based emotion recognition

Taran S, Bajaj V. Emotion recognition from single-channel EEG signals using a two-stage correlation and instantaneous frequency-based filtering method. Comput Meth Programs Biomed. 2019;173:157–65.

Article  Google Scholar 

Damasio A. DESCARTES’ ERROR: Emotion, Reason, and the Human Brain, vol. 3. New York, NY, USA: McGraw-Hill; 1964.

Google Scholar 

Hazarika D, Zimmermann R, Poria S. Misa: Modality-invariant and-specific representations for multimodal sentiment analysis. In: Proceedings of the 28th ACM International Conference on Multimedia, 2020;pp. 1122–1131

Kang H, Hazarika D, Kim D, Kim J. Zero-shot visual emotion recognition by exploiting BERT. In: Proceedings of SAI Intelligent Systems Conference, 2022;pp. 485–494

Feng L, Cheng C, Zhao M, Deng H, Zhang Y. EEG-based emotion recognition using spatial-temporal graph convolutional LSTM with attention mechanism. IEEE J Biomed Health Inform. 2022;26(11):5406–17.

Article  Google Scholar 

Zheng WL, Zhu JY, Lu BL. Identifying stable patterns over time for emotion recognition from EEG. IEEE Trans Affect Comput. 2019;10(3):417–29.

Article  Google Scholar 

Lawhern VJ, Solon AJ, Waytowich NR, Gordon SM, Hung CP, Lance BJ. EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces. J Neural Eng. 2018;15(5): 056013.

Article  Google Scholar 

Zheng WL, Liu W, Lu Y, Lu BL, Cichocki A. Emotionmeter: a multimodal framework for recognizing human emotions. IEEE Trans Cybern. 2019;49(3):1110–22.

Article  Google Scholar 

Song T, Zheng W, Song P, Cui Z. EEG emotion recognition using dynamical graph convolutional neural networks. IEEE Trans Affect Comput. 2020;11(3):532–41.

Article  Google Scholar 

Song Y, Zheng Q, Liu B, Gao X. EEG conformer: convolutional transformer for EEG decoding and visualization. IEEE Trans Neural Syst Rehabil Eng. 2022;31:710–9.

Article  Google Scholar 

Zhong P, Wang D, Miao C. EEG-based emotion recognition using regularized graph neural networks. IEEE Trans Affect Comput. 2022;13(3):1290–301.

Article  Google Scholar 

Zheng WL, Lu BL. Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks. IEEE Trans Auton Ment Dev. 2015;7(3):162–75.

Article  Google Scholar 

Wang XW, Nie D, Lu BL. Emotional state classification from EEG data using machine learning approach. Neurocomputing. 2014;129:94–106.

Article  Google Scholar 

Du X, Ma C, Zhang G, Li J, Lai YK, Zhao G, Deng X, et al. An efficient LSTM network for emotion recognition from multichannel EEG signals. IEEE Trans Affect Comput. 2022;13(3):1528–40.

Article  Google Scholar 

Alhagry S, Fahmy AA, El-Khoribi RA. Emotion recognition based on EEG using LSTM recurrent neural network. Int J Adv Comput Sci Appl, 2017;8(10)

Pascanu R, Gulcehre C, Cho K, Bengio Y. How to construct deep recurrent neural networks. In: ICLR 2014.

Glorot X, Bordes A, Bengio Y. Domain adaptation for large-scale sentiment classification: A deep learning approach. In: Proceedings of the 28th International Conference on Machine Learning, 2011;pp. 513–520

Goodfellow I, Lee H, Le Q, Saxe A, Ng A. Measuring invariances in deep networks. In: Advances in Neural Information Processing Systems, 2009;vol. 22

Sainath TN, Vinyals O, Senior A, Sak H. Convolutional, long short-term memory, fully connected deep neural networks. In: 2015 IEEE International Conference on Acoustics, Speech and Signal Processing, 2015;pp. 4580–4584

Shi X, Chen Z, Wang H, Yeung DY, Wong WK, Woo WC. Convolutional LSTM network: A machine learning approach for precipitation nowcasting. In: Advances in Neural Information Processing Systems, 2015;vol. 28

Li X, Song D, Zhang P, Yu G, Hou Y, Hu B. Emotion recognition from multi-channel EEG data through convolutional recurrent neural network. In: 2016 IEEE International Conference on Bioinformatics and Biomedicine, 2016;pp. 352–359

Zhang D, Yao L, Zhang X, Wang S, Chen W, Boots R, Benatallah B. Cascade and parallel convolutional recurrent neural networks on EEG-based intention recognition for brain computer interface. In: Proceedings of the Aaai Conference on Artificial Intelligence, 2018;vol. 32

Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, 2017;vol. 30

Dai Z, Yang Z, Yang Y, Carbonell J, Le Q, Salakhutdinov R. Transformer-XL: Attentive language models beyond a fixed-length context. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019;pp. 2978–2988

Andayani F, Theng LB, Tsun MT, Chua C. Hybrid LSTM-transformer model for emotion recognition from speech audio files. IEEE Access. 2022;10:36018–27.

Article  Google Scholar 

Hu J, Shen L, Sun G. Squeeze-and-excitation networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018;pp. 7132–7141

Rudakov E, Laurent L, Cousin V, Roshdi A, Fournier R, Nait-Ali A, Beyrouthy T et al. Multi-Task CNN model for emotion recognition from EEG Brain maps. In: 4th International Conference on Bio-Engineering for Smart Technologies, 2021;pp. 1–4

Yang Y, Wu Q, Fu Y, Chen X. Continuous convolutional neural network with 3D input for EEG-based emotion recognition. In: Neural Information Processing: 25th International Conference, 2018;pp. 433–443

Yang Y, Wu Q, Qiu M, Wang Y, Chen X. Emotion recognition from multi-channel EEG through parallel convolutional recurrent neural network. In: 2018 International Joint Conference on Neural Networks, 2018;pp. 1–7

Maaten LV, Hinton G. Visualizing data using t-SNE. J Mach Learn Res. 2008;9(11):2579–605.

Google Scholar 

Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D. Grad-cam: Visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE International Conference on Computer Vision, 2017;pp. 618–626

Li X, Zhang Y, Tiwari P, Song D, Hu B, Yang M, Zhao Z, Kumar N, Marttinen P. EEG based emotion recognition: a tutorial and review. ACM Comput Surv. 2022;55(4):1–57.

Article  Google Scholar 

Park J, Woo S, Lee JY, Kweon IS. BAM: Bottleneck Attention Module. In: BMVC, 2018;pp. 1–14

Zhao Z, Liu Q, Zhou F. Robust lightweight facial expression recognition network with label distribution training. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, 2021;pp. 3510–3519

Zhao Z, Li Q, Zhang Z, Cummins N, Wang H, Tao J, Schuller BW. Combining a parallel 2D CNN with a self-attention Dilated Residual Network for CTC-based discrete speech emotion recognition. Neural Netw. 2021;141:52–60.

Article  Google Scholar 

Woo S, Park J, Lee JY, Kweon IS. Cbam: Convolutional block attention module. In: Proceedings of the European Conference on Computer Vision, 2018;pp. 3–19

Tong L, Zhao J, Fu W. Emotion recognition and channel selection based on EEG signal. In: 2018 11th International Conference on Intelligent Computation Technology and Automation, 2018;pp. 101–105

Tao W, Li C, Song R, Cheng J, Liu Y, Wan F, Chen X. EEG-based emotion recognition via channel-wise attention and self attention. IEEE Trans Affect Comput. 2023;14(1):382–93.

Article  Google Scholar 

Duan R-N, Zhu J-Y, Lu B-L. Differential entropy feature for EEG-based emotion classification. In: 2013 6th International IEEE/EMBS Conference on Neural Engineering, 2013;pp. 81–84

Shi L-C, Jiao Y-Y, Lu B-L. Differential entropy feature for EEG-based vigilance estimation. In: 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 2013;pp. 6627–6630

Wang Y, Zhang B, Tang Y. DMMR: Cross-subject domain generalization for EEG-based emotion recognition via denoising mixed mutual reconstruction. In: Proceedings of the AAAI Conference on Artificial Intelligence, 2024;pp. 628–636

Koelstra S, Muhl C, Soleymani M, Lee JS, Yazdani A, Ebrahimi T, Pun T, Nijholt A, Patras I. DEAP: a database for emotion analysis; using physiological signals. IEEE Trans Affect Comput. 2012;3(1):18–31.

Article  Google Scholar 

Ekman P. An argument for basic emotions. Cogn Emot. 1992;6(3–4):169–200.

Article  Google Scholar 

Russell JA. Affective space is bipolar. J Pers Soc Psychol. 1979;37(3):345–56.

Article  MathSciNet  Google Scholar 

Lee M, Park H-Y, Park W, Kim K-T, Kim Y-H, Jeong J-H. Multi-task heterogeneous ensemble learning-based cross-subject EEG classification under stroke patients. Eng: IEEE Trans. Neural Syst. Rehabil; 2024.

Book  Google Scholar 

Kang H, Choi JW, Kim BH. Cascading global and sequential temporal representations with local context modeling for EEG-based emotion recognition. In: International Conference on Pattern Recognition, 2024;pp. 305–320

Li M, Lu BL. Emotion classification based on gamma-band EEG. In: 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2009;pp. 1223–1226

Lundberg SM, Lee SI. A unified approach to interpreting model predictions. In: Advances in Neural Information Processing Systems, 2017;vol. 30

Huang D, Guan C, Ang KK, Zhang H, Pan Y. Asymmetric spatial pattern for EEG-based emotion detection. In: the 2012 International Joint Conference on Neural Networks, 2012;pp. 1–7

Park HM, Kim G, Oh J, Messem AV, Neve WD. Interpreting stress detection models using SHAP and attention for MuSe-Stress 2022. IEEE Transactions on Affective Computing 2024.

Wang Y, Zhang B, Tang Y. DMMR: Cross-subject domain generalization for EEG-based emotion recognition via denoising mixed mutual reconstruction. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, 2024;pp. 628–636

Wang K, He B, Zhu WP. TSTNN: Two-stage transformer based neural network for speech enhancement in the time domain. In: ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2021;pp. 7098–7102

Gu A, Dao T. Mamba: Linear-time sequence modeling with selective state spaces. arXiv preprint arXiv:2312.00752 2023.

Comments (0)

No login
gif