Unsupervised Human Fatigue Expression Discovery via Time Series Chain

Abstract

Fatigue manifests as a multifaceted human condition involving both psychological and physiological aspects. It is characterized by a diminished capacity to perform tasks effectively, potentially resulting in negative emotional states, errors in passive or active tasks, and even medical emergencies. There is a growing interest and practicality in continuous monitoring to identify fatigue during extended work periods. Despite the importance of fatigue detection, it is very challenging to build a model in practice due to limited data and a diverse set of sensor modalities. In this paper, we propose an unsupervised pipeline to address the challenge of fatigue detection from video streams in a challenging realistic environment. Specifically, we propose an effective fatigue expression discovery framework by first extracting key landmark points (e.g., shoulder joint, mouth) from video streaming data, then identifying evolving behavior patterns with time series chain, an effective high-order time series primitive, to discover precursors for potential human fatigues. To demonstrate the effectiveness of our proposed framework, we show that our framework can detect signs of fatigue using video data captured in real-world fatigue scenarios