Human factors study on usage of BCI headset for 3D CAD

Accepted Manuscript
Human factors study on usage of BCI headset for 3D CAD modeling
S. Sree Shankar, Rahul Rai
PII:
DOI:
Reference:
S0010-4485(14)00007-4
http://dx.doi.org/10.1016/j.cad.2014.01.006
JCAD 2159
To appear in:
Computer-Aided Design
Please cite this article as: Sree Shankar S, Rai R. Human factors study on usage of BCI
headset for 3D CAD modeling. Computer-Aided Design (2014),
http://dx.doi.org/10.1016/j.cad.2014.01.006
This is a PDF file of an unedited manuscript that has been accepted for publication. As a
service to our customers we are providing this early version of the manuscript. The manuscript
will undergo copyediting, typesetting, and review of the resulting proof before it is published in
its final form. Please note that during the production process errors may be discovered which
could affect the content, and all legal disclaimers that apply to the journal pertain.
Human Factors Study on Usage of BCI headset for 3D CAD Modeling Sree Shankar S and Rahul Rai* DART LAB Department of Mechanical and Aerospace Engineering University at Buffalo, Buffalo, NY -‐14260 Contact Author*: [email protected] Abstract Since its inception, computer aided 3D modeling has primarily relied on the Windows, Icons, Menus, Pointer (WIMP) interface in which user input is in the form of keystrokes and pointer movements. Brain-‐
computer interface (BCI) is a novel modality that uses the brain signals of a user to enable natural and intuitive interaction with an external device. In this paper we present a human factors study on the use of an Emotiv EEG BCI headset for 3D CAD modeling. The study focuses on substituting the conventional computer mouse and keyboard based inputs with inputs from the Emotiv EEG headset. The main steps include 1) training the headset to recognize user specific EEG/EMG signals and 2) assigning the classified signals to emulate keystrokes which are used to activate/control different commands of a CAD package. To assess the performance of the new system, we compared the time taken by the users to create the 3D CAD models using both the conventional and BCI based interfaces. In addition, to exhibit the adaptability of the new system, we carried out the study for a set of CAD models of varying complexity. 1. INTRODUCTION Computer Aided Design (CAD) systems form the basic foundation of any design and innovation processes. Modern day CAD systems rely heavily on the conventional windows, icons, menus, pointer (WIMP) user interfaces (UIs). A Lack of natural interaction in WIMP UIs inhibits the creativity of the designer in the design process and this in turn hampers the quality of design solutions. There is a need for intuitive and natural interfaces that allow designers to interact with computers in the same way that they interact with people [1]. In other words, human computer interaction (HCI) should encapsulate the characteristics of human-‐human interaction (HHI). Extensive work has been carried out in developing new modalities that complement existing UIs. Brain computer interface (BCI) is a novel technology that imbibes HHI specific characteristics. The BCIs' potential to become an important modality of natural interaction for 3D modeling is almost limitless and unexplored. The study presented in this paper lays out a preliminary foundation to allude to BCIs capability for 3D CAD modeling. Typically a BCI system is used to detect and relate patterns in brain signals to the users' thoughts and intentions. In this regard, over the years EEG based systems have been used for a variety of applications [2,3,4]. Another form of input that can be used for human computer interaction is electromyogram (EMG) signals. EMG corresponds to electric signals generated by skeletal muscular activity. Unlike reading mental activity, these detections are very fast (10ms) thereby conveying a decisive advantage and rendering them suitable for fast paced applications like gaming. Gomez-‐Gil et al [5] demonstrated the reliability of EMG signals when it was used to control a tractor. The use of the Emotive headset for the study involves utilizing both EEG and EMG based signals in tandem. The user specific signals are converted into intended commands which are consequently used to create 3D CAD models. All the CAD models constructed for the study are based on the following three operations: (1) create geometrical shapes, (2) edit shapes by resizing or by geometric operations such as booleans, sweeps and extrusions, and (3) move shapes by rotations and translations. As every individual has a unique thought pattern when it comes to performing the same task, human factors study have been carried out to assess the reliability and intuitiveness of the BCI based system developed in this research. A preliminary human factor study on five subjects was carried out. The preliminary human factors study demonstrates that the developed system has the potential to have a lower learning curve and high usability. The paper is organized as follows. In Section 2, relevant work carried out in the field of multimodal interfaces for CAD applications is discussed. Section 3 outlines the basic hardware and overall system used in the study. The sample 3D models and results of the study are presented in Section 4. Results and conclusions have been summarized in Section 5. 2. RELATED WORK In this section, the related literature in key research areas is discussed. 2.1 BCI in CAD The use of EEG signals to support CAD modeling is relatively new. Esfahani et al used BCI to distinguish between primitive shapes that are imagined by a user [6,7]. The users’ brain signals are used to extract distinguishing features which are then used to classify the users’ intent which maps to one of the shape primitives. BCI has also been used as a support modality for interfacing with a touch based CAD system that deals with object manipulation [8]. 2.2 Deploying Brain Signals Brain activity associated with any thinking process has to be quantified and converted to tangible intentions and commands. The very first advancements in this respect were made in the domain of virtual reality. Pfurtscheller et al. [9] demonstrated for the first time that it is possible to move through a virtual street without muscular activity when the participant only imagines feet movements. Leeb et al. [10] also carried out similar studies where the user was able to navigate in a virtual environment by using his/her EEG brain signals. Fabiani et al. [11] and Trejo et al. [12] took this one step further with their work on cursor movement using BCI. These studies illustrate the use of BCI in a variety of applications [13]. Similar advancements have also been made in the world of virtual gaming [14] where brain signals are used to control the user's avatar. 2.3 Natural and Intuitive CAD Interfaces Intuitiveness can be a key parameter in deciding the success of an interface. People tend to use interfaces that are more intuitive and easily understood/used. Generally this necessitates the inclusion of human-‐human interaction (HHI) aspects in the interface [1]. HHI relies on the concurrent use and perception of behavioral signals (cues) such as speaking, moving, gazing, and gesturing, which convey various messages (communicative intentions). For example, to describe an object, one can talk about it and at the same time; move one's hands to explain the different features of the object such as its size or shape. MozArt, a Multimodal Interface (speech and touch) for conceptual 3D modeling is an ideal example in this regard [15,16]. MozArt prototype interface explores how multimodal input, coupled with the appropriate hardware may be applied to simplify conceptual 3D modeling for novice CAD users. 2.4 Development of Non-‐invasive BCI BCI's advancement can be largely attributed to its non-‐invasive nature [17]. In addition, factors such as reduced costs and portability have also helped to increase its popularity among researchers [18,19]. In BCI, field work carried out by companies such as Emotiv and Neurosky have led to the development of headsets that give users a lot of freedom. Reduction in the cost of the hardware has also enabled this upcoming technology to compete with traditional WIMP based interfaces. If developed to a greater extent, BCI based UIs have the potential to replace existing WIMP based CAD interfaces. In summary, BCI interfaces are being used in a wide variety of applications ranging from its traditional use for recording EEG signals to video gaming. However, BCI's potential to be used as a natural and intuitive interface for conceptual 3D CAD modeling remains largely unexplored and this forms the foundation of the study presented here. The next section details the system components used for the study. 3. SYSTEM COMPONENTS AND METHODS In the study carried out we use electrical signals based on user's brain activity, i.e. electroencephalography (EEG), and facial muscles, i.e. electromyography (EMG) for interfacing with a CAD system. As shown in Figure 1 the bio-‐signals are recorded and analyzed using an Emotiv EEG EPOC headset. These signals are then used to train the interface to understand the user's thought patterns and intent. Furthermore, we use an inbuilt Emotiv API as the main interface engine to link the BCI headset with the CAD application. Google SketchUp is used as test CAD platform because of its simplicity and ease of use [15]. Figure 1: Basic framework used for the study 3.1 The Emotiv EEG Interface The Emotiv EEG is a low cost Human-‐Computer Interface (HCI) that is comprised of: (1) a neuroheadset hardware device to acquire EEG and EMG signals and (2) the software development kit (SDK) to process and interpret these signals. The placement of the channels is based on the international 10-‐20 convention. The neuroheadset acquires brain signals using 14 sensors placed on the user scalp. It also includes 2 internal gyroscopes to provide information about the head position of the user. The BCI headset communicates with the computer system wirelessly by means of a USB dongle. The Emotiv headset can capture and process brainwaves in the Delta (0.5–4 Hz), Theta (4–8 Hz), Alpha (8–14 Hz) and Beta (14–26 Hz) bands [5] using an inbuilt API’s. The user's affective emotions and cognitive actions can be detected using these signals. However expressive actions which correspond to the user's face movements relate to EMG signals. The Emotiv control panel facilitates 13 different cognitive actions, i.e. push, pull, left, right, up and down directional movements, clockwise, counter-‐
clockwise, left, right, forward and backward rotations and a special action that makes an object disappear in the user's mind. Additionally, the angular velocity of one's head can be measured in the yaw and pitch (but not roll) directions using the inbuilt gyros. 3.2 Software Components The Emotiv toolkit includes the a C++ API, which allows (1) communication with the Emotiv headset, (2) reception of preprocessed EEG/EMG and gyroscope data, (3) management of user-‐specific or application-‐specific settings, (4) post-‐processing, and (5) translation of the detected results into an easy-‐
to-‐use structure (also called as EmoState) [18]. Figure 2 describes the integration of Emotiv API and EmoEngine. The EmoEngine is an inbuilt logical abstraction of the Emotiv API that performs all the processing of the data from the Emotiv headset. The Emotiv EEG, by means of the Emotiv API, provides external applications information about the event type that emanates from the user's brain and reports the event power, which represents the certainty of the event occurrence. Alternatively, a neutral event is reported when no actions are detected [5]. Figure 2: Integration of EmoEngine and Emotiv API with an application The following inbuilt functions of the Emotiv API are used in the study: 1. Emotiv Control panel 2. Emotiv Key The Emotiv Control panel serves as a training platform for novice users. It has three suites, namely the expressive, affective, and cognitive suites of which we use the expressive and cognitive suites for the study. The expressive suite uses EMG signals as inputs i.e. signals that are generated from the movement of facial muscles. The EMG component consists of 11 recognizable inputs, all of which can be calibrated to best suit the user. The cognitive suite has a special test bench that acquires relevant EEG signals. During the cognitive training, the user has to imagine manipulating a cube in the virtual environment. This training action includes imagining pushing, pulling, rotating, lifting, and dropping the cube in the virtual environment. It is to be noted that the signals recorded during the training session is not filtered for artifacts. The EPOC also includes "EmoKey" software used to emulate keystrokes based on combinations of thoughts, feelings, and facial expressions. Any EPOC detection can be paired with keystrokes or string of keystrokes through a simple user interface by the end user. The assignment of EMG/EEG signals is accompanied by conditions such as `greater than', `equal to', `lesser than', and `occurs'. These conditions have to be fulfilled in order to execute a given command. The EmoKey can thus be used as a controller to simulate surface selection and manipulation tasks of 3D CAD modeling. As mentioned earlier, we use Google SketchUp software as the testing platform for our study [20]. Google SketchUp provides an easy to use 3D modeling environment that has a large online repository of model assemblies for efficient analysis. The graphical user interface (GUI) of Google SketchUp is used as platform to display the current state of 3D models. Based on visual feedback from the current state of 3D models, the user generates a new set of input brain signals. The new brain signals are then recorded and processed by the Emotiv headset and sent to Google SketchUp for modifying the generated models. 4. USER STUDY In this section we describe the experimental setup and present the results from the study. 4.1 Experimental Setup To obtain an objective assessment of our approach, we conducted an institutional review board (IRB) approved formal user study using five participants with an average age of 26 (SD=1.5) with no known mental disorders. The participants were chosen such that they had little to no experience in CAD modeling. Furthermore the experiment was spread over three days to account for maximum variability associated with the participants’ mental state. The CAD models used for the experiment varied in complexity. Within this experimental framework complexity is defined with respect to the number of commands required to create the models. Hence a simple model would require fewer commands in comparison to a more complex model. Figure 3 shows sample 3D CAD models created during the study. Figure 3: 3D CAD models created by different users using the developed BCI based CAD system The experiments started with training sessions where each participant’s thought patterns were tuned to the desired actions such as push, pull, and eye blink etc. On an average the calibration process took about 30-‐35 minutes. Table 1 shows a sample mapping that was used for the construction of the 3D CAD models. The magnitudes for the conditions that have been shown in the table are based on trial and error. These values are unique to each user and are generally determined during training. The EmoKey is then used to map these values to specific keystrokes. A demo video of the study can be found using the following link: http://www.youtube.com/watch?v=GATQUVgLLYY Table 1: Sample mapping of keystrokes using Emotiv Key 4.2 Results In order to validate the performance of presented BCI interface, a human factors study consisting of novice CAD users was performed. The aim of the study was to: 1. assess individual differences/similarities of the participants while creating the 3D models 2. assess the learning curve of the users Figure 4 shows the steps, the participants had to perform for the study, sequentially. We observe that the complexity of the tasks increases progressively. The main parameter used for the study is the time taken to complete the tasks assigned. Therefore our study is based on the assumption that with increase in complexity, the time taken to complete the tasks should also increase. Figure 4: Tasks assigned to participants Figure 5 displays a distribution of the participants' completion time with varying task difficulty. As anticipated, the completion time of the tasks was proportional to the task difficulty level. There was a steep increase in the completion time of task 3 compared to tasks 1 and 2. This was largely due to the involvement of two thought processes (push and pull) in tasks 3 and 4. Even though no additional information was provided to participants during the execution of task 4, participants took lesser time to complete the task. It should be noted that task 4 was of the same difficulty level as task 3 and this indicates that the participants’ were able to adapt to the interface once they finished the first three tasks. Figure 5: Participants' performance for different tasks To evaluate the performance of individual participants across all four tasks, their completion times were compared as shown in Figure 6. Despite variation in the thought patterns and facial expressions, individual participants’ performances were not statistically different from each other. This indicates that the BCI has good potential as a generalized medium for CAD modeling and the exploration interface. Figure 6: Comparison of individual performance 5 CONCLUSION AND FUTURE WORK In this paper we evaluate a BCI based system to create 3D CAD parts. The study involved recording the subject’s brain activity and EMG signals corresponding to facial movements using an Emotiv EEG headset. An inbuilt API classifies the above mentioned bio-‐signals to ascertain the users’ intent. This information is then used to carry out 3D CAD modeling in Google SketchUp. In order to test the performance of interface, a human factors study comprising of 5 participants was performed. Each participant was given sufficient time to get acquainted with the system (~30 min), after which they were asked to create a series 3D CAD models (training) of increasing complexity. The participants were then asked to create a test model (unsupervised), to assess the adaptability of the system. The results of the study showed a learning curve that displayed a peak at the most difficult task (model 3). However 80% of the participants demonstrated significant improvement which was indicated by a reduction in time to construct the test model (model 4). It is important to note that the complexity of the test model was at par with that of the most difficult training model. Observations indicate that despite the lower learning curve and users' adaptability to the interface, EEG based signals were sometimes hard to classify. This could be eliminated by creating more robust classifiers. The reliability of the system degenerates when more than two cognitive actions are involved. The participants also experienced fatigue that could be ascribed to the headset configuration and design. . This could be rectified by using BCI headsets that are more ergonomic like the Neurosky mindwave or InteraXon Muse. The human factors study results presented in the paper were based on a small group of users. In future, a more extensive user study should be performed to get a better understanding of the interface usability. Another future research direction will be to improve the usability of the presented interface by integrating BCI with other modalities such as speech and gestures. References [1] H. Gürkök, A. Nijholt, Brain–computer interfaces for multimodal interaction: a survey and principles, International Journal of Human-‐Computer Interaction, 28 (2012) 292-‐307. [2] G. Pfurtscheller, C. Neuper, C. Guger, W. Harkam, H. Ramoser, A. Schlogl, B. Obermaier, M. Pregenzer, Current trends in Graz brain-‐computer interface (BCI) research, Rehabilitation Engineering, IEEE Transactions on, 8 (2000) 216-‐219. [3] M. Cheng, X. Gao, S. Gao, D. Xu, Design and implementation of a brain-‐computer interface with high transfer rates, Biomedical Engineering, IEEE Transactions on, 49 (2002) 1181-‐1186. [4] G.R. Muller-‐Putz, G. Pfurtscheller, Control of an electrical prosthesis with an SSVEP-‐based BCI, Biomedical Engineering, IEEE Transactions on, 55 (2008) 361-‐364. [5] J. Gomez-‐Gil, I. San-‐Jose-‐Gonzalez, L.F. Nicolas-‐Alonso, S. Alonso-‐Garcia, Steering a tractor by means of an EMG-‐based human-‐machine interface, Sensors, 11 (2011) 7110-‐7126. [6] E.T. Esfahani, V. Sundararajan, Classification of primitive shapes using brain–computer interfaces, Computer-‐Aided Design, 44 (2012) 1011-‐1019. [7] E.T. Esfahani, V. Sundararajan, Using brain computer interfaces for geometry selection in cad systems: P300 detection approach, in: ASME IDETC Conference Proceedings, 2011. [8] R. Bhat, R. Rai, A. Deshpande, E.T. Esfahani, BCI-‐Touch based system, a multimodal CAD interface for object manipulation, in: ASME IMECE Conference Proceedings, 2013. [9] G. Pfurtscheller, R. Leeb, C. Keinrath, D. Friedman, C. Neuper, C. Guger, M. Slater, Walking from thought, Brain research, 1071 (2006) 145-‐152. [10] R. Leeb, R. Scherer, F. Lee, H. Bischof, G. Pfurtscheller, Navigation in virtual environments through motor imagery, in: 9th Computer Vision Winter Workshop, CVWW, 2004, pp. 99-‐108. [11] G.E. Fabiani, D.J. McFarland, J.R. Wolpaw, G. Pfurtscheller, Conversion of EEG activity into cursor movement by a brain-‐computer interface (BCI), Neural Systems and Rehabilitation Engineering, IEEE Transactions on, 12 (2004) 331-‐338. [12] L.J. Trejo, R. Rosipal, B. Matthews, Brain-‐computer interfaces for 1-‐D and 2-‐D cursor control: designs using volitional control of the EEG spectrum or steady-‐state visual evoked potentials, Neural Systems and Rehabilitation Engineering, IEEE Transactions on, 14 (2006) 225-‐229. [13] A. Kübler, B. Kotchoubey, J. Kaiser, J.R. Wolpaw, N. Birbaumer, Brain–computer communication: Unlocking the locked in, Psychological Bulletin, 127 (2001) 358. [14] A. Lécuyer, F. Lotte, R.B. Reilly, R. Leeb, M. Hirose, M. Slater, Brain-‐computer interfaces, virtual reality, and videogames, Computer, 41 (2008) 66-‐72. [15] A. Sharma, S. Madhvanath, A. Shekhawat, M. Billinghurst, MozArt: a multimodal interface for conceptual 3D modeling, in: Proceedings of the 13th international conference on multimodal interfaces, ACM, 2011, pp. 307-‐310. [16] C. Hondrou, G. Caridakis, Affective, natural interaction using EEG: sensors, application and future directions, in: Artificial Intelligence: Theories and Applications, Springer, 2012, pp. 331-‐338. [17] K.-‐R. Müller, B. Blankertz, Toward non-‐invasive brain-‐computer interfaces, Signal Processing Magazine, IEEE, 23 (2006) pp. 125-‐128. [18] http://www.emotiv.com/, (August, 2013) [19] http://www.neurosky.com/, (August, 2013) [20] http://sitescontent.google.com/google-‐sketchup-‐for-‐educators/home, (August, 2013)

Download Report