Paper Reading #27 : Sensing Cognitive Multitasking for a Brain-Based Adaptive User Interface

Reference

Authors and Affiliations:

Erin Treacy Solovey,Francine Lalooses,Krysta Chauncey, Douglas Weaver,Margarita Parasi,Matthias Scheutz,Robert J.K. Jacob at Tufts University, Computer Science,161 College Ave., Medford, MA

Angelo Sassaroli, Sergio Fantini, at Tufts University Biomedical Engineering 4 Colby St., Medford, MA

Paul Schermerhorn,Indiana University Cognitive Science Bloomington, IN

Audrey Girouard,Queens University School of Computing 25 Union St., Kingston, ON K7L 3N6

Presentation: CHI 2011, May 7–12, 2011, Vancouver, BC, Canada

Summary

Hypothesis

The paper describes two experiments leading toward a system that detects cognitive multitasking processes and uses this information as input to an adaptive interface.

Authors then present a human-robot system as a proof-of-concept that uses real-time cognitive state information as input and adapts in response.They hypothesize that the prototype system serves as a platform to study interfaces that enable better task switching, interruption management, and multitasking.

Contents

The authors describe a preliminary study and two experiments using neural data in which they identified four mental processes that may occur during multitasking and have direct relevance to many HCI scenarios. These processes are almost indistinguishable by examining overt behavior or task performance alone. However, using their non-invasive brain-sensing system with functional near-infrared spectroscopy (fNIRS), they were able to automatically distinguish these four states. The authors show an example of this with a proof-of-concept adaptive human-robot system that can change behavior based on brain signals received.

Methods

12 healthy volunteers (10 male), between the ages of 18 and 34 were asked to

complete ten 40-second trials for each of the three conditions of multitasking(branching, delay, dual mode), which were presented randomly. The tasks were based on the Human robot task procedures done previously to classify rocks on mars.

Each task was repeated until the participant achieved greater than 80% accuracy in the task. For the second experiment,

12 participants completed eighteen trials of random and predictive branching to distinguish specific types of branching taks following on the procedure of the first experiment.

Results

Experiment one found that statistical significance in response time between delay and dual, delay and branching, but not between dual and branching . Similarly, they found statistical significance in accuracies between delay and dual, delay and branching, but not dual and branching. Also, correlations between accuracy and response time for each task were not statistically significant.T

otal hemoglobin measures were overall higher in the branching condition than in the dual-task or delay condition.

There was no statistically significant difference in response time between random and predictive branching.There also was no statistically significant difference in accuracy between random and predictive branching. Also, correlation between accuracy and response time for random branching was not statistically significant, but there was a statistically significant correlation in the predictive branching condition.

Deoxygenated hemoglobin levels were higher in the random branching type than the predictive branching type for the first half of trials, but reversed for the second half.

Discussion

We studied that we are not capable of true multitasking in one of our reading book(Why we make mistakes?) and I was amazed to see this study showing different types of multitasking. Later it was clear that these are not true multitasking activities. I was surprised by the similar results for the dual and branching activities. The research provides a foundation for developing HCI devices or any other devices which tend to get going in a multi tasking environment. Actually, it might be able to help in designing devices that are less prone to be mishandled or cause less error/mistakes by human users.