record

Thesis Info

LABS ID
00871
Thesis Title
A Reactive Brain Computer Interface: a novel sonification and visualization approach evoked by illusions
Author
Marlene D. D. Mathew
2nd Author
3rd Author
Degree
PhD
Year
2020
Number of Pages
University
University of California, Santa Barbara
Thesis Supervisor
George Legrady
Supervisor e-mail
glegrady AT ucsb.edu
Other Supervisor(s)
Tobias Hollerer, Alenda Chang, Andres Cabrera
Language(s) of Thesis
English
Department / Discipline
Media Arts and Technology
Copyright Ownership
Author
Languages Familiar to Author
English, Dutch, Papiamento
URL where full thesis can be found
Keywords
Brain Computer Interface, Visual Evoked Potential, Optical Illusion, Sonification, Visualization, Visual Perception
Abstract: 200-500 words
The Brain Computer Interface (BCI) as a communication method has become very popular in the past several decades. This type of technology has also become more active in Human Computer Interaction for multimodal interaction and research. In this research the intersection between art and neuroscience primarily from the perspective of a Media Artist is explored. Neural activity creates electric and magnetic fields in the human brain and Brain Computer interfacing (BCI) gets that brain’s activity, which in turn can be used to control an application or environment. A reactive BCI is when an application generates stimuli that someone needs to focus on, which creates changes in that person's brain activity. BCI technology can give us a useful entry point into examining how biological data shapes the aesthetic attributes of our personal and biomedical information and the significance communicated by an artwork that has a technical interface as an essential component. By embracing BCI technology as a methodology, such a work makes it visible to a public that has little direct contact with the works of scientists. The scientific precision of artistic BCI has been scrutinized, but the key value of artists’ use of this technology lies in the exploration of the emotive and educational impact of these types of technologies. This research looks into the effects that art may have on the brain, looking into visual illusions also known as optical illusions. Optical illusions are distortions of the senses, which are caused by the visual system. Visual perception is the brain's ability to make sense of what the eyes see. Electroencephalography (EEG) is a non-invasive method used to measure and evaluate the electrical activity in the brain. Very few visual perception studies explore the link between neuroscience and the arts. Visual processing in the brain creates several types of visual evoked potentials (VEP) in our brainwaves. A VEP is an evoked potential caused by a visual stimulus, which is measured by the electrical response of the brain’s primary visual cortex to a visual stimulus. Another goal of this research is to develop an interface using visual perceptive EEG data to create a computational language, that is, come up with a framework that will provide sonic and/or visual output of this neurofeedback information. This is done primarily from an artist/composer (Art) standpoint while looking into cognition and perception (Science). The proposed program works with brainwave data, using VEP features triggered by optical illusions to manipulate audio/visuals. The research project exposes some significant developments in the use of BCI technology for artistic purposes, like how to precisely collect and process EEG data aesthetically, as well as what license I can use with the data in order to create meaning or an environment for the audience themselves to bring meaning to the artwork. The interest lies in seeing how visual perception can inform and offer new forms of expression. This dissertation looks at artistic explorations and narratives that comes out of the BCI data, drawing on insights from fields like cognitive neuroscience, neurofeedback, biology, Brain Art and Op Art. It also presents a novel approach in creating media artworks using VEP features and multimodal interaction to explore visual and sonic output. It also documents development of Visum and Aspecta, two bio artworks by concentrating on the conceptual design, approach, methods and challenges. The overall goal is to offer pathways within the field of human computer interaction by introducing novel sensory methods of interfacing with computer systems that aim to amplify human qualities.