Cross-Domain Transfer Learning with Vision Transformers for Fatigue Recognition from Facial Expressions

Abstract

Fatigue detection plays a critical role in mission-critical environments such as defense operations, transportation, and industrial control, where sustained attention and alertness are paramount for safety, operational efficiency, and human performance. This study introduces a real-time, non-intrusive fatigue detection system that employs Vision Transformers (ViTs) to identify subtle facial cues associated with fatigue. Unlike traditional methods that rely primarily on overt indicators such as yawning, eye closure, or head nodding, our approach leverages advanced deep learning techniques to capture nuanced micro-expressions and subtle behavioral patterns that are often overlooked. By applying transfer learning from the FER-2013 dataset to the NTHU-DDD dataset, we achieve enhanced model generalization, with ViT-L16 and R50+ViT-B16 architectures attaining classification accuracies of 70.31% and 72.79%, respectively. Visual attention maps reveal that FER-FFR models focus more effectively on critical facial regions, enabling precise feature extraction and interpretability. Furthermore, we introduce a fatigue level indicator that quantifies fatigue progression over consecutive video frames, demonstrating behavior that closely aligns with human fatigue dynamics. The proposed system is robust, scalable, and suitable for deployment in real-world operational settings, providing an automated, reliable, and objective solution for continuous fatigue monitoring, potentially enhancing safety, productivity, and decision-making in high-stakes environments.