TY - JOUR

T1 - Intelligent Human Operator Mental Fatigue Assessment Method Based on Gaze Movement Monitoring

AU - Kashevnik, Alexey

AU - Kovalenko, Svetlana

AU - Mamonov, Anton

AU - Hamoud, Batol

AU - Bulygin, Aleksandr

AU - Kuznetsov, Vladislav

AU - Shoshina, Irina

AU - Brak, Ivan

AU - Kiselev, Gleb

N1 - The research has been supported by the Bortnik innovation fund #23ГУKoдИИC12-D7/79179.

PY - 2024/10/23

Y1 - 2024/10/23

N2 - Modern mental fatigue detection methods include many parameters for evaluation. For example, many researchers use human subjective evaluation or driving parameters to assess this human condition. Development of a method for detecting the functional state of mental fatigue is an extremely important task. Despite the fact that human operator support systems are becoming more and more widespread, at the moment there is no open-source solution that can monitor this human state based on eye movement monitoring in real time and with high accuracy. Such a method allows the prevention of a large number of potential hazardous situations and accidents in critical industries (nuclear stations, transport systems, and air traffic control). This paper describes the developed method for mental fatigue detection based on human eye movements. We based our research on a developed earlier dataset that included captured eye-tracking data of human operators that implemented different tasks during the day. In the scope of the developed method, we propose a technique for the determination of the most relevant gaze characteristics for mental fatigue state detection. The developed method includes the following machine learning techniques for human state classification: random forest, decision tree, and multilayered perceptron. The experimental results showed that the most relevant characteristics are as follows: average velocity within the fixation area; average curvature of the gaze trajectory; minimum curvature of the gaze trajectory; minimum saccade length; percentage of fixations shorter than 150 ms; and proportion of time spent in fixations shorter than 150 milliseconds. The processing of eye movement data using the proposed method is performed in real time, with the maximum accuracy (0.85) and F1-score (0.80) reached using the random forest method.

AB - Modern mental fatigue detection methods include many parameters for evaluation. For example, many researchers use human subjective evaluation or driving parameters to assess this human condition. Development of a method for detecting the functional state of mental fatigue is an extremely important task. Despite the fact that human operator support systems are becoming more and more widespread, at the moment there is no open-source solution that can monitor this human state based on eye movement monitoring in real time and with high accuracy. Such a method allows the prevention of a large number of potential hazardous situations and accidents in critical industries (nuclear stations, transport systems, and air traffic control). This paper describes the developed method for mental fatigue detection based on human eye movements. We based our research on a developed earlier dataset that included captured eye-tracking data of human operators that implemented different tasks during the day. In the scope of the developed method, we propose a technique for the determination of the most relevant gaze characteristics for mental fatigue state detection. The developed method includes the following machine learning techniques for human state classification: random forest, decision tree, and multilayered perceptron. The experimental results showed that the most relevant characteristics are as follows: average velocity within the fixation area; average curvature of the gaze trajectory; minimum curvature of the gaze trajectory; minimum saccade length; percentage of fixations shorter than 150 ms; and proportion of time spent in fixations shorter than 150 milliseconds. The processing of eye movement data using the proposed method is performed in real time, with the maximum accuracy (0.85) and F1-score (0.80) reached using the random forest method.

KW - eye-tracking

KW - machine learning

KW - mental fatigue detection

KW - Humans

KW - Mental Fatigue/diagnosis

KW - Eye Movements/physiology

KW - Machine Learning

KW - Eye-Tracking Technology

KW - Male

KW - Adult

KW - Algorithms

KW - Female

UR - https://www.scopus.com/record/display.uri?eid=2-s2.0-85208567754&origin=inward&txGid=3c5cd18ba5b82cb13dc98eb6c146691f

UR - https://www.mendeley.com/catalogue/41749f63-0350-3c39-ac88-75aeff6fa130/

UR - https://www.elibrary.ru/item.asp?id=74839663

UR - https://pubmed.ncbi.nlm.nih.gov/39517703/

U2 - 10.3390/s24216805

DO - 10.3390/s24216805

M3 - Article

C2 - 39517703

VL - 24

JO - Sensors

JF - Sensors

SN - 1424-3210

IS - 21

M1 - 6805

ER -