Neural and Hybrid Computing Laboratory (NHC)
University of Florida
24 April 2007
I use mathematical modeling for recognizing patterns in electrical activity that was recorded from the eyes. The aim is to recognize a pattern and co-relate it to actual eye movements (left, right, blink etc.). This setup is then expanded to include a command module that allows a subject to use eye movements to control a video game. I start with a random signal. Using a BiQuad filter, I filter out some of the high frequencies. I then use a Discrete Fourier Transform along with interpolation to get a relatively smooth waveform. Finally I perform analysis on this waveform to detect eye movements.
Designing an interface that allows the user to control an external device using only their brain activity is one on the recently flourishing areas of neuroscience that aims to allow paralyzed individuals to have control of some basic activities in their lives, such as eating, turning the light on in a room, reaching out and grasping something. One of the more basic experiments designed to demonstrate how to work through coordinating such a scheme is an Electro-Oculography Brain Machine Interface, and the EEG BMI (Electroencephalography Brain Machine Interface) is much more complicated. Using Matlab 6.5. and the Tucker Davis Technologies data acquisition system and a TCP/IP connection to Quake, a 3-D video game on a separate desktop, I have created an interface that allowed the user to control the movement of a player in Quake II (video game) using only the electrical activity that was recorded from the eyes. There is a lot of research that is going on about BMI interfaces that will eventually help people that are paraplegic to regain some control in their lives.BACKGROUND ON EOG- (Electro-Oculography)
Electro-Oculography measures the potential of the retina. In creating an EOG-BMI, I used the differences in potential to create an interface controlling the video game Quake . The retina has a negative charge associated with it and the cornea has a less negative charge, referred to as the Cornea Retinal Potential (CRP), approximately 10-30 mV. Movement of the eye generates an electric charge that is measured by the electrodes that were placed on either side of the eye. EOG recordings are also used in the areas of Psychology and Psychiatry to gain insight into various psychological disorders1 and also in Ophthalmologic diagnosis he EOG is used in to assess the function of the pigment epithelium2.1 http://ee.ucd.ie/~smeredith/EOG_index.html 2 http://en.wikipedia.org/wiki/Electrooculography
The initial setup used the cap with the 10-20 electrode placement system, but I then switched to a simpler setup using just electrodes and wires to the TDT system. The system that I design is much more complex than it appears. Multiple problems arise in programming, using Matlab, however I found the programming to be surmountable and the problem lied in finding the correct algorithm.
My model utilizes the signals generated from the eye movement of a person for controlling a video game. I take a random signal and look for periodic patterns (or approximations to these patterns). If such a pattern is detected (peak detection) then I forward the information to a control module which processes the information and issues commands to the video game. Users can control the game in real-time with visual feedback from the game PC.
Here is a flow chart outlining my solution:
I am using hardware from Tucker Davis Technologies Inc. (Medusa Preamp) to record EOG signals. My circuit reads from channel one, two and three of the TDT PreAmp. The following is a snapshot of it:
Signals from saccadic eye movement are weak, and noise tends to drown that signal. For example, the figure below shows the kind of signal I am dealing with: Various strategies can be used to make this signal stand out against the noise. I can use Fourier analysis to detect patterns in my random sample. Also, I can employ Biquad filters.
Discrete Fourier transform The sequence of N complex numbers x0, ..., xN−1 is transformed into the sequence of N complex numbers X0, ..., XN−1 by the DFT according to the formula: where e is the base of the natural logarithm, is the imaginary unit (i2 = − 1), and π is pi. The transform is sometimes denoted by the symbol , as in or . The inverse discrete Fourier transform (IDFT) is given by:
Note that the normalization factor multiplying the DFT and IDFT (here 1 and 1/N) and the signs of the exponents are merely conventions, and differ in some treatments. The only requirements of these conventions are that the DFT and IDFT have opposite-sign exponents and that the product of their normalization factors be 1/N. A normalization of for both the DFT and IDFT makes the transforms unitary, which has some theoretical advantages, but it is often more practical in numerical computation to perform the scaling all at once as above (and a unit scaling can be convenient in other ways).
(The convention of a negative sign in the exponent is often convenient because it means that Xk is the amplitude of a "positive frequency" 2πk / N. Equivalently, the DFT is often thought of as a matched filter: when looking for a frequency of +1, one correlates the incoming signal with a frequency of −1.)
In the following discussion the terms "sequence" and "vector" will be considered interchangeable.
In signal processing, a digital biquad filter is a second-order recursive linear filter, containing two poles and two zeros. "Biquad" is an abbreviation of "biquadratic", which refers to the fact that in the Z domain, its transfer function is the ratio of two quadratic functions:
High-order recursive filters can be highly sensitive to quantization of their coefficients, and can easily become unstable. This is much less of a problem with first and second-order filters; therefore, higher-order filters are typically implemented as serially-cascaded biquad sections (and a first-order filter if necessary).
I use a BiQuad filter to get rid of various unwanted frequencies. Using Discrete Fourier Transform and interpolation I generate a smooth signal:
- I. As the eye moves left I get a positive spike. Since this is a saccadic movement, the eye then comes back to its original position, thus generating an immediate negative spike.
- II. Since I am subtracting the data from channels one and two, the waveform for the right eye movement is an inverted form of the left eye movement.
I correlate the presence of positive or negative peaks with certain intervals of time to discern eye movement. The strength of my approach lies in its ability to adapt to various individuals who generate waves of various amplitudes.
Sequence of Left Right Movements
Note: Right is a positive peak, followed by a negative peak, and the opposite is true for left movements.
Sequence of Left, Right Movements Interleaved With Blinks.
Note: My setup (subtracting signals from left and right eye electrodes) insures that we get minimum interference from natural eye blinking.
Detecting Eye Blinks
Note: Using two additional electrodes positioned above and below the right eye we can detect an eye blink. Although this is possible with just one electrode placed above the right eye, we are using two to boost my signal. The smaller peaks are from the left and right movements of the eyes. Even when the subjects performs saccadic movements as fast as possible the effect is negligible on this channel ( electrodes placed above and below eye).Current Setup for Movement Detection
- I analyze data in “windows” of time.
- The resolution of each window is just slightly larger than half the width of a saccadic movement. The idea is to capture at the most one positive or negative peak in a window.
- We keep track of the last three windows.
- If a negative or a positive peak is detected in the current window then I back track and analyze the last three windows.
- This analysis tells us whether we have a random spike or one that is a part of a right/left movement.
I can make the character in quake move left or right In addition my setup can be modified to include up/down and fire movements as well.
I aim at being able to predict eye position by analyzing the electrodes placed above and below the eyes and on either side. This will require a lot more work.