Stress and Mental Workload Monitoring in HMI Systems

The achievement of an ever-increasing level of safety has always been a key element of Human-Machine Interaction (HMI) systems. Thanks to the disruptive growth of the technological market in the last few years, it is increasingly common to find contexts where an operator interacts with a machine that exhibits a certain level of automation. Hence, always guaranteeing adequate safety during an operator’s performance is necessary. These scenarios can be found in everyday life, such as when driving a car and in specific jobs, as in the case of Industry 4.0 operators, air traffic controllers, or pilots.

Our project aims to develop an autonomous Operator Performance Monitoring system to easily integrate into the existing environment. Our tool provides a continuous real-time output regarding the cognitive workload and the operator’s state of health during the performance. It is based on a multimodal approach that considers the most significant physiological signals, such as cardiorespiratory measures, eye tracking, fNIRS, and skin activity, through ad-hoc developed electronics. Together with operational data, these signals feed our AI algorithm, which estimates the operator’s cognitive load. The idea is to infer individual capacity limits by correlating these parameters’ variation with the mental load’s increase or decrease through specific data labelling and classification.

To sum up, we decided to investigate the physiological multimodal approach and the potential of AI algorithms to foster the transition toward the next generation of HMI systems. This is a crucial topic for ensuring safety in tomorrow’s world of work, which is continuously evolving thanks to the exponential growth of the biomedical sensor market in the last few years. The availability of smaller, cheaper, and more reliable wearable sensors allows for investigating and developing technologies that could not been realised so far to enhance safety and push the industry sector to a new era.

Keywords:

References:

• • M. Matessa, T. Strybel, K. Vu, V. Battiste e T. Schnell, «Concept of Operations for RCO SPO,» NASA, Moffet Field, CA, 2017.
  • P. Ayres, J. Y. Lee, F. Paas e J. van Merriënboer, «The Validity of Physiological Measures to Identify Differences in Intrinsic Cognitive Load,» frontiers in Psychology, vol. 12, n. 702538, pp. 1-16, 2021.
  • E. Debie, R. F. Rojas, J. Fidock, M. Barlow, K. Kasmarik, S. Anavatti, M. Garratt e H. A. Abbass, «Multimodal Fusion for Objective Assessment of Cognitive Workload,» IEEE TRANSACTIONS ON CYBERNETICS, vol. 51, n. 3, pp. 1542-1555, 2021.