Examination of Cognitive Workload using Different Modalities
P. Shrisowmya1, K. Adalarasu1*, M. Jagannath2
1Department of EIE, SASTRA Deemed to be University, Thanjavur, Tamil Nadu, India.
2School of Electronics Engineering, Vellore Institute of Technology (VIT), Chennai, Tamil Nadu, India.
*Corresponding Author E-mail: adalarasu@eie.sastra.edu
ABSTRACT:
Cognitive skills are human’s mental skills which help in processing and acquiring knowledge through reasoning, perception and intuition. Cognitive load is associated with an understanding of visualization by enabling us to ingest and interpret. The major influence of the cognitive process is working memory that gets burdened when an individual’s workload increased. Hence, cognitive load can be used as a means to measure the efficacy of visualization. There are numerous physiological signals and imaging modalities available to examine the cognitive workload of an individual. In this study, the various modalities used to examine cognitive performance of the subjects and corresponding classifier model were discussed. The classifier model is intended to link the psychological and physiological data to examine an individual’s cognitive workload conditions. This approach paves a way for the researchers and developers to estimate the cognitive workload in a quantitative platform.
KEYWORDS: Cognitive Workload, Stress, Questionnaire, Physiological Signals, Imaging Techniques.
1. INTRODUCTION:
Cognitive skills are those learning skills by which human brain stores information like attentiveness of students in class or understanding behavior of persons2. Auditory and visual processing and processing speed all comes under cognitive skills. Cognitive skills include the process of brain of fetching this information from memory where large amount of data is stored.
The cognitive workload is defined as the amount of quantifiable psychological activities that are performed by an individual to accomplish one or more cognitive tasks.
It is considered to be a property of an individual rather than a task. The amount of work that needed to be performed (‘the work’), usually within a fixed period of time (‘the load’) is referred as workload and it is generally used to characterize a job.
India has about 1.37 billion people strength and making it the world’s second largest country. Every year 800,000 people’s commits suicide due to stress3. The cognitive workload is widely used to analyse the stress level of the human in her working environmental. Early detection of individual’s functional state is considered as one of the vital phenomena in many emergency situations. The poor cognitive performance results with high monetary and/or life fortunes which represent a high hazard for operational mistakes. The cognitive disability can have been ascribed to time on obligation, sleep misfortune, task delay, high mental outstanding task at hand and psychosocial stress. Analysis of cognitive performance using mathematical model widely used to analyse the workload condition of service people such as traffic polices, pilots, soldiers, etc.
Active cognition during complex and sustained operation is a critical component for success in current military operations. Several unpredictable factors such as battle, fatigue, unseen threats, unplanned railway travel, inability to ensure quality education to children, increased workload tasks lead to stress and eventually increase the level of frustration4. The Goals, Operators, Methods, Selection (GOMS) is a theoretical structure used for examination of workload. Measurement of cognitive load using physiological signals such as Electrocardiogram (ECG), Electroencephalogram (EEG), and Electrooculogram (EOG) has become an interesting field of investigation among many researchers. This study reviewed the various physiological signals and imaging modalities to examine the cognitive workload for the subjects in their workplace environment.
2. MATERIAL AND METHODS:
The following are the cognitive workload outcome measures that are existing in literature.
Alexander et al.5 proposed an attractive method for experimental research using Amsterdam Resting-State Questionnaire (ARSQ) to estimate resting-state cognition for rapid and structured measurements. Based on this ARSQ, the discontinuity of mind, sleepiness and somatic awareness have been identified. The result of this study also provides a brief knowledge about sleeping state of humans. Various techniques have been proposed for classifying workload such as subjective rating, task performance and physiological response6,7. Commonly used subjective assessment technique is NASA Task Load Index (TLX) where workload can be indicated in numerical or graphical scale. The NASA TLX includes a two evaluation measures that consists of both weights and ratings. The weights are associated with workload of a specific task. The ratings reveal the magnitude of the weights in a specified task. The total score of workload is computed by multiplying each rating by the weight given.
Matthews et al.8 developed Dundee Stress State Questionnaire (DSSQ) to define about alertness, ability to perform task. The result of this questionnaire provides information on alertness and ability for attending a high workload tasks. Team Workload Questionnaire (TWLQ) was developed by Sellers et al.9, to explore the task work analysis depending on the task demands. In order to measure sensitive changes during task, TWLQ requires a high degree of interaction and decision making among the participants.
Kazemi et al.10 have proposed Cognitive Failures Questionnaire (CFQ) which includes 25 questions in four fields of failure in human memory, nominal memory, attention, and exercise. The result of this study indicates that cognitive failure makes an individual defenseless against demonstrating awful impacts of stress.
The following physiological signals are considered to be the outcome measures for examining the cognitive workload.
Electrocardiogram (ECG):
Cardiovascular estimates observed to be touchy to task yet that are less delicate to neighboring dimensions of expanding cognitive workload7.The individual’s working memory capacity (WMC) plays a vital association in cardiovascular activities. To measure the participant’s WMC, the operation span (OSPAN) task was performed. During this experiment, many cardiovascular features were extracted in real time. This study shows that significant relation was obtained between workload levels and WMC.
Kilseop and Rohae11 proposed a measure in light of different physiological indices so as to assess the psychological workload in double tasks. In order to examine the mental load, three physiological signals such as ECG, EEG and EOG were monitored. These signals were collected from ten subjects while performing a dual task at different versions composed of tracking and mental arithmetic. The features such as alpha rhythm, eye blink interval and HRV were extracted from three signals. These features are interpreted with cognitive workload. It was found that the processing indices by alpha rhythm were shown significantly different from the processing indices by eye blink interval and HRV.
Electroencephalography (EEG):
Berka et al.12 monitored the task engagement level and mental workload information. The levels of task engagement and mental workload information can also be used to optimize the design of safe workplace, which can increase the worker efficient that in turns lead to increase the productivity. In this study the author explored the feasibility of monitoring engagement and workload while performing the cognitive tasks using EEG signal. The tasks include addition, choice vigilance, learning and memory tests. The EEG signals were analyses with epoch of 1 second for workload. It was found that engagement task was decreased whereas workload was increased with vigilance task. At the same time, both increases during learning and memory tests.
Chaouachi and Frasson13 proposed a model to evaluate the mental workload using spectral features of EEG signals. The workload index was trained using Gaussian process regression model in which the workload level incrementally increases with cognitive task. Similarly, learning activity phase shows that learning index gradually increases from the pre- test to the end of the session. Thus the correlation analysis shows that cognitive workload is linearly related with cognitive task activity.
The mental workload was assessed using statistical features like mean, RMS (root mean square) and correlation based parameter were extracted from EEG signals14. This study revealed that as task level increases, the mean value of all features extracted from EEG also increases. The result concluded that the feasibility and effectiveness of EEG were tested successfully for examining cognitive workload in real time.
To achieve maximum efficiency and productivity it is important to maintain an optimal cognitive load15. The EEG signal was recorded form different lobes of the brain. The result of this study shows that during multitasking prefrontal cortical region are more actively involved when compared to other brain location. This study found that increased external task demand increases the cognitive workload level, and in turn decreases the physiological activities at the same time. During high workload task, participants suffering from sleep loss have exposed decreased task performance. Generally, as mental workload increases, theta waves increase in the frontal lobe and alpha waves decrease in the parietal lobe. The power of frontal theta waves activity increases with the increase in awake time. The power of parietal alpha waves activity decreases slightly while it increases during awake time. The EEG measures are helpful in examining assessing the general commitments of workload which are not recognized by other indices.
Electrooculography (EOG):
Marshall et al.16 proposed a contextual investigation to build up new procedure for distinguishing and contrasting cognitive techniques. In this study, pupil dilation was used to find the relationship between eye movements and cognitive workload. The task performance of the person was correlated with the Index of Cognitive Activity (ICA), which pave a way for alternative method for examining cognitive workload using pupil dilation. The following physiological signals such as EEG, EOG and Electromyography (EMG) were acquired during workload condition and this signal use to find the different cognitive levels. It is found that the variations in pupil dilation identify the timing and location of potential strategy shifts, as measured by ICA. The rapid decrease in the ICA suggested that the person has altered the way during the course of performing the task. This study demonstrates that workload level can be identified from strategy shifts which in turn can be identifiable from EOG signal.
The following imaging modalities are considered to be the outcome measures for examining the cognitive workload.
Magnetic Resonance Imaging (MRI):
Ding et al.17 uses a Convolutional Neural Networks (CNN), a deep learning technique, which is designed to predict Mild Cognitive Impairment (MCI) that occurs at the prodromal stage of Alzheimer’s disease (AD). Mild cognitive impairment (MCI) is referred as the intermediate stage that occurs between normal cognition and dementia. Subject with MCI are at high danger of conceivable to AD which is essential for better treatments. Numerous past examinations use neuro imaging biomarkers to order AD patients at various infection stages or to foresee the MCI-to-AD transformation. Among every one of these investigations, Magnetic Resonance Imaging (MRI) is observed to be a standout amongst the most widely used imaging methodology due to non-invasive and moderate expense.
The study estimates the MCI-to-AD conversion accurately using CNN. Age correction and assisted structural brain image features can also be used as key parameters for enhancing the performance of CNN.
Functional Near-Infrared Spectroscopy (fNIR):
Functional Near-Infrared Spectroscopy (fNIR) is a neuro imaging technology that uses light to measure cortical brain activity, which also highly portable and safe. fNIR technique was used for measuring cognitive workload of certified Air Traffic Controllers under typical and emergent conditions18. The fNIR also measures hemodynamic response of optical brain imaging modality. Subjects perform an n-back task which involves working memory and attention with four stages of difficulty.
Here, subjects were inquired to monitor visual stimuli (single letters) which are serially displayed on a screen and they had to click a button when a target stimulus arrives on the screen. To vary working memory load from zero to three states, four conditions were used. It is found that in n-back results, accuracy and speed of the subjects decrease monotonously as task difficulty increases. It is also found that the average correct click ratio decreases and average response time increases dully. The fNIR results were found to be sensitive to task difficulty precisely at left inferior frontal gyrus region. As measured by fNIR, blood oxygenation increases monotonously with increasing task difficulty that eventually reflects the cognitive workload. Thus fNIR could be used to measure cognitive workload using neuro imaging technology.
3. RESULTS AND DISCUSSION:
Cognitive impairments have been documented in different operational situations. Workload is basically classified into three classes as follows.
Subjective Workload:
To measure the person’s workload when he/she is feeling. It is mainly devoted question-answer type response at various levels of workload.
Table 1. Mental workload classification using physiological outcome measures.
|
Modality |
Eye Based Outcome Measure |
Heart Based Outcome Measure |
|
Magnetic Resonance Imaging (MRI) |
Electrooculography (EOG) |
Electrocardiography (ECG) |
|
Functional Near-Infrared Spectroscopy (fNIR) |
Blink interval |
Heart rate variability (HRV) |
|
Electroencephalography (EEG) |
Blink closure duration |
Heart rate (HR) |
|
Event Related Potentials (ERP) |
Blink rate |
Blood volume |
Performance based Workload:
In this case, by using a primary or a secondary task, the capacity of a person can be measured. The mental workload can be estimated by increasing the difficulty and duration of the tasks.
Physiological Workload:
The quantification of physiological variables is determined by the mental demands that lead to increased physical activities. Table 1 shows the physiological outcome measures that are considered to be effective tools to examine cognitive workload. These outcome measures are classified into three categories as a function of organs are involved: (a) brain related, (b) eye related and (c) heart related.
4. CONCLUSION:
The aim of this review is to highlight the significance of physiological signals, and imaging techniques available to examine the cognitive workload analysis. The discussed articles also provide a clear comparison among different features of the physiological signals (ECG, EEG and EOG) and imaging modalities used for predicting cognitive workload. This detailed review enables researchers to develop a cognitive workload model with higher gain accuracy and efficiency.
5. ACKNOWLEDGEMENT:
Authors would like to thank all researchers who have contributed their work in the field of cognitive workload.
6. CONFLICT OF INTEREST:
The authors of this manuscript declare no conflict of interest.
7. REFERENCES:
1. T.M. Lahtinen, J.P. Koskelo, T. Laitinen and T.K. Leino, Heart rate and performance during combat missions in a flight simulator, Aviation, Space and Environmental Medicine. 2007; 78: 387–391.
2. K. Mohanavelu, S. Poonguzhali, R. Banuvathy, K. Adalarasu and M. Jagannath, Biosignal processing approaches for detecting mental fatigue, Indian Journal of Public Health Research & Development. 2018; 9(10): 236–241.
3. A. Mukhopadhyay (2019, February). When Will India Address Its Student Suicide Crisis? Fair Observer Available Online:https://www.fairobserver.com/region/central_south_asia/india-student-suicides-mental-health-south-asia-news-75495/
4. H.R. Lieberman, P. Niro, W.J. Tharion, B.C. Nindl, J.W. Castellani and S.J. Montain, Cognition during sustained operations: comparison of a laboratory simulation to field studies, Aviation, Space and Environmental Medicine. 2006; 77: 929–935.
5. B.A. Diaz, S. Van Der Sluis, S. Moens, J.S. Benjamins, F. Migliorati, D. Stoffers, A. Den Braber, S.S. Poil, R. Hardstone, D. Van’t Ent, et al. The Amsterdam Resting-State Questionnaire reveals multiple phenotypes of resting-state cognition, Frontiers in Human Neuroscience. 2013; 7: 446.
6. E. Julien, H.L. Nigel and Z. Pega, Beyond subjective self-rating: EEG signal classification of cognitive workload, IEEE Transaction on Autonomous Mental Development. 2015; 7(4): 301–309.
7. B. Geethanjali, K. Adalarasu, M. Jagannath and R. Rajasekaran, Influence of pleasant and unpleasant music on cardiovascular measures and task performance, International Journal of Biomedical Engineering and Technology. 2016; 21(2): 128–144.
8. G. Matthews, M Zeidner and R.D. Roberts, Emotional intelligence: A promise unfulfilled?, Japanese Psychological Research. 2012; 54: 105–127.
9. J.M. Sellers, Team Workload Questionnaire (TWLQ): Development and Assessment of a Subjective Measure of Team Workload, Thesis of Master of Science in Applied Psychology, University of Canterbury, 2013.
10. R. Kazemi, S. Karimpour, M. Shahriyari and S.N. Hossain, A survey of the relationship between the mental workload and cognitive failure in taxi drivers, Journal of Health Science Surveillance Systems. 2017; 5(4): 188–192.
11. R. Kilseop and M. Rohae, Evaluation of mental workload with a combined measure based on physiological indices during a dual task of tracking and mental arithmetic, International Journal of Industrial Ergonomics. 2005; 35(11): 991–1009.
12. C. Berka, D.J. Levendowski, M.N. Lumicao, A. Yau, G. Davis, V.T. Zivkovic, R.E. Olmstead, P.D. Tremoulet and P. L. Craven, EEG correlates of task engagement and mental workload in vigilance learning and memory tasks, Aviation, Space and Environmental Medicine. 2007; 78(5): B231–B244.
13. M. Chaouachi and C. Frasson, Exploring the relationship between learner EEG mental engagement and affect, in Proceedings of the 10th International Conference on Intelligent Tutoring Systems, Springer Berlin Heidelberg, 2010, pp. 291–293.
14. P. Zarjam, J. Epps and F. Chen, Evaluation of working memory load using EEG signals, in Proceedings of APSIPA Annual Summit and Conference, 2010, pp. 715–719.
15. R. Shriram and D.M.S. Nivedita Daimiwal, EEG based cognitive workload assessment for maximum efficiency, IOSR Journal of Electronics and Communication Engineering, 2006, pp. 34–38.
16. S.P. Marshall, C.W. Pleydell-Pearce and B.T. Dickson, Integrating psychophysiological measures of cognitive workload and eye movements to detect strategy shifts, in Proceedings of the 36th Hawaii International Conference on System Sciences, Big Island, HI, USA, 2003, pp. 130b–135b.
17. Y. Ding, J.H. Sohn, M.G. Kawczynski, H. Trivedi, R. Harnish, N.W. Jenkins, et al., A deep learning model to predict a diagnosis of Alzheimer disease by using 18F-FDG PET of the brain, Radiology. 2019; 290: 456–464.
18. A. Hasan, W. Ben, B. Scott, A.S. Patricia, I. Kurtulus, H. Sehchang, D. Atul and O. Banu, Cognitive workload assessment of air traffic controllers using optical brain imaging sensors, Advances in Understanding Human Performance: Neuroergonomics, Human Factors Design, and Special Populations, 2010, pp. 21–31.
Received on 07.05.2019 Modified on 26.06.2019
Accepted on 11.07.2019 © RJPT All right reserved
Research J. Pharm. and Tech. 2019; 12(10):4647-4650.
DOI: 10.5958/0974-360X.2019.00800.X