Study and analysis of human mental stress detection using galvanic skin response and heart rate sensors in wired and wireless environments

 

Saroj Kumar Panigrahy*, Sanjay Kumar Jena, Ashok Kumar Turuk

Department of Computer Science Engineering, NIT Rourkela

*Corresponding Author E-mail:  skp.nitrkl@gmail.com

 

ABSTRACT:

It is an infeasible task for psychoanalysts to monitor the chronic mental stress of patients. Due to topical and innovative developments in wireless sensor technology and physiological sensors, researchers have been attracted to pursue research in wireless body area sensor networks (WBASN) or wireless body area networks (WBSN). As a result, the wireless sensor technology has enabled medical science in enhancing real-time health monitoring and maintaining a better healthcare scenario. In our previous work, the mental stress sensor presented was able to detect the stress condition based on the Galvanic Skin Response (GSR) only with an average accuracy of about 76%. In this article, a system has been presented for sensing human mental stress by acquisition, processing and analysing the GSR and heart rate or pulse rate signals from human body. This system is proficient in detection of the mental stress condition using the GSR and heart rate signals assimilated by wearable physiological sensors in wired or wireless environments. For evaluation of the system performance, well-known supervised binary classifiers have been selected for classifying the GSR and heart rate data from the physiological dataset available and the results are discussed. This multi-sensor stress detection system has average accuracy more than the previous stress detection system using GSR.

 

KEYWORDS: Mental Stress, Biological Sensor, Physiological Sensor, Electrodermal Activity, EDA, Galvanic Skin Response, GSR, Wireless Body Area Sensor Network, WBASN, Wireless Body Area Network (WBAN)

 

 


INTRODUCTION:

Human mental stress is defined by Hans Selye1 as “the non-specific response of the body to any demand for change”. Chronic stress can create risk of cardiovascular diseases and melancholy. Hence, there is a crucial requirement of stress management as a part of mental healthcare of human being. Psychoanalysts and individuals face difficulties in keeping track of stress level changes for a prolonged period.

 

Thus, a system, capable of monitoring the stress levels of an individual, constantly for a longer period (e.g., full day, some days, weeks, or months), is highly required for human mental healthcare. The system should send alert messages to the individual or the physician responsible for treatment of a mental patient. Accordingly, the physician can suggest some stress management procedure for the mentally ill patients.

 

There are various physiological sensors available for measuring human physiological signals. Some of them are listed in Table 1. Contemporary research advances in non-invasive wearable physiological sensors and wireless sensor networks, make it easier to remotely monitor the human physiological signals continuously for a prolonged period.

 

Table 1: List of Physiological Sensors

Sl.

Physiological Sensors

1

Blood Pressure

2

Pulse Rate (PR) or Heart Rate (HR)

3

Electrocardiogram (ECG)

4

Electroencephalogram (EEG)

5

Electromyogram (EMG)

6

Galvanic Skin Response (GSR)

7

Foot Motion

8

Body Temperature

9

Glucose or Blood Sugar Monitor

10

Peripheral Capillary Oxygen Saturation (SpO2)

 

A.      Wireless Body Area Sensor Network:

A wireless sensor network (WSN) is a wireless network consisting of spatially distributed autonomous devices using sensors to monitor physical or environmental conditions. A WSN system incorporates a gateway that provides wireless connectivity back to the wired world and distributed nodes2. WSNs have significantly developed with the faster progresses in semiconductor and wireless technologies. There are many application areas of WSN. One such application in healthcare is the Wireless Body Sensor Network (WBSN) or Wireless Body Area Network (WBAN) or Wireless Body Area Sensor Network (WBASN). A WBASN is a sensor network for a specific purpose which operates independently to connect various sensors and actuators, located in and/or on human body3. A WBASN consists of miniaturized, lightweight, low-power, invasive/non-invasive wireless sensor nodes that monitors the human body vital signs and its environment4.

 

Figure 1. A typical WBASN

 

In WBASN, different sensors/actuators are attached to clothes and the body (non-invasive) or implanted inside the body (invasive) of a human being. A sensor sometimes called sensor node gathers data, processes them and then sends them to a control unit (another sensor node or a computer) wirelessly. A sensor consists of a power unit for supply of power, a processor for data processing, a memory unit for temporarily storage of data and a transceiver for sending and receiving data. Figure 1 shows a typical WBASN.

 

WBASNs can be a part of procedures such as diagnosis of diseases from analysis of vital signs, taking care of chronic conditions, and handling emergency events5. Usually, a WBASN consists sensor networks, which can be of in-body and/or on-body types. In an in-body sensor network, the communication occurs between implanted/invasive devices/nodes and the body central unit (BCU). In case of on-body sensor network, the communication takes place between wearable/non-invasive devices/nodes and the BCU. A WBASN sensor can connect to a network (after searching and finding a suitable one) for communication and transmit sensed data to a server (remotely located) or central control unit (CCU) for analysis and storage as shown in Figure 2.

 

Some of the challenges of WBSN is user acceptance, privacy and security of medical data. The acceptance by the patients is also an important factor for the successful execution of a new technology such as WBSN in healthcare. For that, the patients should have a clear understanding of the technology to be used for healthcare6.

 

Figure 2. A WBASN connected to another network

 

Due to huge amount of medical data collected in the due course of time, the real-time data from the sensors can be uploaded to the cloud through Internet connection, so that real-time monitoring and alert system can be realized. When the patient data is published and stored to cloud/Internet, there is always a risk of privacy and security of medical data. For this, some security measures such as maintaining secrecy7 and cryptographic techniques8,9 has to be used during implementation.

 

B.      Physiological Signals Used in the System:

In this work, two human physiological signals have been used for mental stress detection. These are (i) GSR and (ii) HR. These are explained briefly.

 

i.         Galvanic Skin Response:

The skin conductance also referred to as galvanic skin response is one important sensitive indicators of emotional arousal. GSR modulates the secretion of sweat from sweat glands present in the skin. Sweat glands are found in varying densities throughout the body, being the highest in the palm region (approximately 370 sweat glands per cm2) according to Gray's estimates10. The skin conductance changes in hand and foot regions due to secretion of sweat. Skin conductance is moderated autonomously by sensitive activity which initiates human behaviour, mental and emotive states on an intuitive level. GSR is one of the sensing modality for stress detection which is minimal obtrusive and has better advantage than other physiological variables because the skin is innervated, exclusively by sympathetic nervous system. Hence, GSR is highly sensitive to emotive stimulation (e.g., anger, fear, and startle response)11.

 

ii.       Heart Rate:

Heart rate or pulse is the number of times human heart beats per minute (bpm). It can vary according to the physical needs of body such as need to absorb oxygen and excrete carbon dioxide. Normal heart rate ranges from 60-100 bpm. Central nervous system stimulants increase heart rate. In contrast, central nervous system depressants or sedatives decrease the heart rate. A stressful situation sets off a chain of events. As a result, human body releases adrenaline, a hormone that temporarily causes breathing and heart rate to speed up and blood pressure to rise12.

 

C.      Types of Physiological Sensors:

There are various types of physiological or biological sensor developed by researchers over the past two decades. One category of sensors falls into invasive or implantable type, i.e., the sensors are implanted inside the body. The other category falls into non-invasive type, i.e., these are on-body sensors. Earlier, the non-invasive type physiological sensors were wired in nature, i.e., the connection to body was through long wires, but were robust and there was no requirement of separate power supplies. But the drawback was, the user had to deal with hanging wires which was creating hindrance in mobility. A varied wireless technology for communication have been available over the last few years, from wide area networks (e.g., cellular, Internet) to short-range networks (e.g., Bluetooth13, ZigBee14). Wide area networks may be used to communicate servers in a remote location (web-based healthcare15) and provide its users the freedom in mobility to a higher degree.

 

D.      Hardware for Acquisition and Transmission of Physiological Sensor Data:

As discussed in previous section, the physiological sensors are capable of collecting data from human body. But for processing and analysis of those data, the sensors have to be connected to specific hardware for recording of the data. This can be achieved by a number of ways through wired or wireless mediums. The hardware consists of microcontrollers16 for processing like conversion of data from analog to digital form and transceivers for transmission of data to a computer. Examples of widely used such devices are Arduino, Raspberry Pi, and so on17. In a wired environment, biological sensors collect data from human body and send to a computer directly connected to the sensors for processing and analysis. However, in a wireless environment, the physiological sensors collect data from human body and send to a computer, which is in a remote place within the range of wireless medium. The recorded data can be stored in a computer for processing and further analysis for real-time monitoring as well as offline applications.

 

STRESS SENSOR BASED ON GSR and HR:

In our previous work18, a stress sensor with only GSR data is capable of detecting stress levels with an average accuracy of about 76%. In this paper, a stress sensor with GSR and HR data has been designed. It can record the GSR and HR signals of human body in both wired as well as wireless environments and provides alerts if there is a change in stress level detected beyond a threshold.

 

A.      Hardware Setup:

In this section, brief description of the hardware used for our experimental study has been provided. The hardware requirements for our experimental study are ¾ (i) a Grove GSR sensor19 for sensing GSR signal from human subject, (ii) a Grove fingercilp HR sensor20, (iii) an Arduino UNO21 board (microcontroller) used for data acquisition, and (iii) two Zolertia Z1 motes22 (wireless sensor nodes having microcontroller and wireless transceivers) used for data acquisition and wireless transmission, and (iv) a computer for storing, processing and further analysis of collected data. Figure 3 shows the hardware used for our study.

 

Figure 3. Hardware used in the experimental study

 

In our experimental study, for collecting the GSR and HR signals from human subject, the subject has to wear the GSR sensor on the index and middle fingers and the HR sensor on the thumb of non-dominant hand. The GSR and HR sensors are connected to the data acquisition device and the data is stored in a computer through wired or wireless medium. Figure 4 depicts the general idea of our experimental setup.

 

Figure 4. General experimental setup

 

There are two scenarios for collecting the GSR and HR signals from human body: wired and wireless. For the experimental setup in wired environment, the hardware connection has been done as shown in Figure 5. The outputs of the Grove GSR and HR sensors are connected to the Arduino Uno board which is connected to a computer. The analog GSR and HR signals are transformed to digital form by the Analog to Digital Converter (ADC) unit present in Arduino board, and then transmitted to the computer through the USB port for storage.

 

Figure 5. Hardware setup in wired environment

 

In case of wireless environment, two Zolertia Z1 motes are used (instead of an Arduino board) for both data acquisition and wireless transmission. The outputs of the Grove GSR and HR sensors are connected to one of the Z1 mote which is placed/tied to hand of the subject and another Z1 mote is connected to the computer. The two Z1 motes communicate wirelessly using IEEE 802.15.4 network standard23.The general hardware setup has been depicted in Figure 6.

 

Figure 6. Hardware setup in wireless environment

 

B.      Experimental Procedure:

Experimental data from two conditions¾ mental stress and relaxed state were used (not recorded due to denial by the participants for recording and storing) from volunteer participants for the experiment (all subjects provided written informed consent for the study). The participant/subject has to sit and wear the GSR and HR sensors as shown in Figure 7.

 

 

 

 

Figure 7. Snapshots during experiment in (a) wired environment with Arduino Board (b) wireless environment with two Zolertia Z1 motes: one tied to the hand of the subject and the other connected to a computer.

 

 

The mental stress condition imposed by conducting a sequence of tests: arithmetic operations (e.g., repeated subtraction of 9 from 500 for a duration of two minutes), memory search (a list of words to be shown in sequence for 30 seconds for memorising, then the participant has to identify them among a set of confusing words under time constraint of 1 minute), reading the sliding texts on the screen for 1 minute, Stroop colour word test24 (the participants were shown one out of five words such as violet, red, green, blue, and yellow in different colours and they had to identify the colour by clicking one of the five buttons according to the colour) for 1 minute. The relaxation condition consists of sitting calm and breathing deeply.

 

C.      Stress Detection:

At the start of experiment, during the relaxation phase (2 minutes), the mean and standard deviation of GSR and HR values are computed. During stress conditions (conducting series of tasks) the GSR and HR values tend to change significantly. This significant change in GSR and HR values beyond a threshold value as compared to the precomputed mean and standard deviation of GSR and HR values respectively helps in stress detection.

 

The GSR and HR data from the dataset25 have been used for classification using supervised classifier. It consists of GSR and HR values of 10 subjects during a playing computer game (a variant of PACMAN). The GSR and HR values correspond to frustration and normal conditions of the subjects during the play. The GSR and HR data are preprocessed in such a way that the duration of the non-activity (values -1) were removed and divided into blocks of samples using window sizes of 50 depending upon the frustration conditions (0-no, 1-yes). Then the mean and standard deviation of the blocks and their corresponding frustration condition are stored. These are the features used for classification of the frustration condition (stress) of the subjects.

 

There are various supervised binary classifiers available and used for classification of different datasets26-29. Three well-known and faster supervised binary classifiers¾ BayesNet27, J4828, and Decision Table29 have been used for detecting the mental condition of the subject whether frustrated (stressed) or not. A 10-fold cross validation has been used for finding the accuracy of the classifications.

 

RESULTS:

This section presents the results obtained.

 

The accuracies of various classifiers for all the 10 subjects using the classifiers¾ BayesNet, J48 and Decision Table are presented in Table 1.

 

 

Table 1: Classifier Accuracy (%) for 10 subjects using BayesNet, J48 and Random Forest

Subject ID

Classifier Accuracy (%)

BayesNet

J48

Decision Table

S1

85.96

91.7

89.67

S2

74.25

82.42

76.86

S3

81.88

85.99

84.02

S4

88.86

90.55

89.34

S5

69.16

75.74

68.97

S6

68.31

78.90

71.66

S7

76.21

84.40

81.80

S8

77.26

80.04

77.02

S9

83.28

89.66

87.46

S10

91.14

94.46

92.95

 

 

 

The average accuracies of all the classifiers is shown in the Figure 8.

 

Figure 8. Average accuracies of all the three classifiers used.

 

DISCUSSION:

From the Table 1, it is clear that the classification based on J48 tree outperforms among the three classifiers used. In our previous work, stress detection using GSR data, J48 tree based classifier was having accuracy of about 76%. But when the system uses both the GSR and HR data, it gives better accuracy of about 85%.

 

In this work, a stress detector using GSR and HR signals has been presented. Its performance analysis has also been shown. As mental stress is also related with the emotional behaviour of a person, depending on the analysis of emotional intelligence30 of a person the physician can suggest a stress management procedure to the patient. In our future work, we may design an emotion sensor using physiological values such as EEG, GSR, HR and EMG data of human being.

 

REFERENCES:

1.        What is stress? The American Institute of Stress. Available from URL: http://www.stress.org.

2.        What Is a Wireless Sensor Network? National Instruments White Papers. Available from URL: http://www.ni.com/white-paper/7142/en.

3.        Khan JY and Yuce MR. Wireless body area network (WBAN) for medical applications. In ch. 31, New Developments in Biomedical Engineering, Edited by Domenico C. Signals and Communication Technology, pp. 591-627, InTech, 2010.

4.        Ullah S, Higgins H, Braem B, Latre B, Blondia C, Moerman I, Saleem S, Rahman Z, Kwak KS. A comprehensive survey of wireless body area networks. Journal of medical systems. 2012 Jun 1; 36(3):1065-94.

5.        Jovanov E, Milenkovic A, Otto C, De Groen PC. A wireless body area network of intelligent motion sensors for computer assisted physical rehabilitation. Journal of NeuroEngineering and rehabilitation. 2005 Mar 1; 2(1):16-23.

6.        Sorathiya V, Kapupara P, Shah K. Technology Based Patient Education Information System. Research Journal of Pharmacy and Technology. 2016; 9(1):101-107.

7.        Anjaneyulu GSGN, Sisodiya YS, Kumar S. Novel Composite Encryption for Secrecy in Cloud Computing. Research Journal of Pharmacy and Technology. 2016; 9(9):1501-1504.

8.        Vijayaragavan R. Proxy Blind Signature Scheme based on Non-Commutative Division Semi-rings. Research Journal of Pharmacy and Technology. 2016;9(7):913-917.

9.        Thyagarajan K, Thangavelu A. Design and Analysis about Preventing Distributed Denial of Service Attacks in Mobile Ad-Hoc Network Using Blocking IP Broadcast Method. Research Journal of Pharmacy and Technology. 2016; 9(8):1229-1244.

10.     Gray H. The Organs of the Senses and the Common Integument. Anatomy of the Human Body (20th ed.). Philadelphia: Lea and  Febiger. 1918.

11.     Dawson ME, Schell AM, Filion DL. Handbook of psychophysiology. Cambridge University Press, Cambridge. 2000.

12.     Stress and Heart Health - American Heart Association. Available from URL: http://www.heart.org/HEARTORG.

13.     Bluetooth Technology. Available from URL: http://www.stress.org.

14.     ZigBee Technology. Available from URL: http://www.zigbee.org

15.     Amsa P, Pandey GK, Bhowmick M, Sonkar S, Karthikeyan D, Kumar TS. The Scope of Web-based Clinical Research in Pharmacy. Research Journal of Pharmacy and Technology. 2008; 1(3):121-123.

16.     Sundaresan YB, Durai MAS. An Experimental Prototype for Comparative Analysis of Intel, Atmel and PIC Microcontrollers. Research Journal of Pharmacy and Technology. 2016; 9(9):1367-1374.

17.     Twelve Best Development Boards for DIY Projects. Available from URL: https:// www.mepits.com/project/236/DIY-Projects/12 -Best-Development-Boards-for-DIY-Projects.

18.     Panigrahy SK, Jena SK and Turuk AK. Study and analysis of human stress detection using galvanic skin response (GSR) sensor in wired and wireless environments. Research Journal of Pharmacy and Technology. 2017; 10(2):545-550.

19.     Grove GSR sensor. Available from URL: http://wiki.seeed.cc/Grove-GSR_Sensor.

20.     Grove Finger-clip Heart Rate Sensor. Available from URL: http://wiki.seeed.cc/Grove-Finger-clip_Heart_Rate_Sensor.

21.     Making Stuff with Arduino. Available from URL: http://nborko.github.io/vc-arduino.

22.     Zolertia Z1, Low-power wireless module for IoT and WSN. Available from URL: http://zolertia.io/z1.

23.     IEEE 802.15.4. IEEE Standard for Low-Rate Wireless Networks. Available from URL: http://standards.ieee.org/getieee802/download/802.15.4-2015.pdf.

24.     Golden CJ, Freshwater SM. Stroop color and word test. Revised 2002 Adult Manual for Clinical and Experimental Uses. 2002.

25.     Reuderink B, Nijholt A, and Poel M. Affective Pacman: A frustrating game for brain-computer interface experiments. In Proceedings of the 3rd International Conference on Intelligent Technologies for Interactive Entertainment (INTETAIN 2009), volume 9 of Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, pages 221–227, May 2009. Springer Berlin Heidelberg.

26.     Chandrasekar Ravi, Khare N. Review of Fuzzy Rule Based Classification systems. Research Journal of Pharmacy and Technology. 2016; 9(8):1299-1302.

27.     Friedman N, Geiger D, Goldszmidt M. Bayesian Network Classifiers. Machine Learning. 1997;29(2-3):131-63.

28.     Quinlan JR. C4.5: Programs for Machine Learning. Morgan Kaufmann Publishers, 1993.

29.     Kohavi R. The power of decision tables. InEuropean Conference on Machine Learning 1995 Apr 25 (pp. 174-189). Springer Berlin Heidelberg.

30.     Deepak A, Krishnamoorthy K. Emotional Intelligence-A Questionnaire Based Study. Research Journal of Pharmacy and Technology. 2016;9(7):772-725.

 

 

 

 

Received on 31.01.2017             Modified on 21.02.2017

Accepted on 28.03.2017           © RJPT All right reserved

Research J. Pharm. and Tech. 2017; 10(4): 1168-1173.

DOI: 10.5958/0974-360X.2017.00211.6