Research Task 2, Part 1 (Isabelle Greenberg)

Mind-Controlled Keyboard:

Introduction:

• Although the subjects for this project are relatively immobile, their eye movements may help to give more possibilities for the research. In the previous papers, extra movements like blinking would move the the next slide and mean something for the program.
• This will rely more on neurophysiologic signals as an access method, or the neuron system in general
• Factors that have to be considered are the users, the environment, and the algorithm behind the process
• There are many ways to make an EEG controlled keyboard and one of them is the RSVP (Rapid Serial Visual Presentation) system. It allows the user to study options one after another by showing the alphabet one letter at a time until the user chooses one
• The headgear must recognize the user's intent to select one of the signals with some form of EEG data
• One of the waves is called P300 (this is probably described more later)
• The language will be in Thai, and the target group is the disabled

2.1 Brain-Computer Interface

• The BCI connects directly between the human brain and the computer, so it will be useful for people with locked in syndrome or ALS
• The user's habits, homes, and their environments will be considered, and they may or may not be considered in our project(the environment may be the most useful in our situation)

2.2 Mind-Controlled Keyboard Interfaces

• Rapid Serial Visual Presentation: Shows one letter at a time and when the expected letter arrives, the positive intent value stands out in comparison to the other letters.
• Matrix Speller: Letters are in rows and columns like a chess board, and moves over the columns one by one. When the expected column arrives, the positive intent signal will distinguish it, and then the speller goes letter by letter in that column
• Hex-o-Spell: This is more visual with groupings of letters in circles. It highlights each circle and loops until the positive intent signal, with which it further breaks down that circle.
• The same idea is used repeatedly but with different setups and appearances.

2.3 Emotiv EPOC

• 14 sensors that track the user's EEG
• Accurately detects when given the user's gender, age, handedness, intentional control, vividness of visual imagery, and mental rotation ability (the ability to move a 2D or 3D image in your mind)
• Test bench is going to be used mostly with this project, and I believe was described more thoroughly in an earlier paper

2.4 EEG Data

• There is a constantly changing electric field on the scalp because of the signals fired by neurons in the brain
• There is 4 types of EEG data:
• P300:
• Detectable peak in activity that occurs 300 ms after some stimulus is presented. This signal helps choose the specific highlighted value when the computer hovers over it
• Slow Cortical Potential:
• Shown by changing voltage in the brain which can be controlled after a long period of time
• Sensorimotor Rhythms:
• Rhythms detected when relaxing while not thinking about movement.
• Steady State visual evoked potential:
• If flashing stimulus is presented to the user, brainwave modulations of the same frequency as the flashing rate of stimulus is detected in the visual cortex.

4.1 Objective and Outputs

• They want to study the signal activities in the brain and get useful info from it, make software with the headset, create new algorithms, and make a new method to the keyboard system
• The outputs will be a program, a report, a new algorithm, and the result of the experiments.
• The new algorithm has to give the fastest solution and the most accurate result

4.2 Benefits

• People with these project can communicate in a new way that's better than before, and the headset will make the setup relatively inexpensive
• In practice the technology we have is insufficient, slow, and sometimes not reliable

Literature Review

• Brain Computer Interfaces is a possible assistive technology (AT) for the disabled and it is classified as an augmentative and alternative communication (AAC) possibility.
• Previous AAC tech has been the joystick, mouse, binary switch, eye gaze and head control, however this does not help a good portion of the disabled
• BCI research for this purpose has 5 components
• Input modalities for the device
• Processing demand of the device
• Language representation
• Output modalities
• Functional gain of the device
• BCI technology has various components
• Stimulus presentation paradigm (check 2.4)
• Signal acquisition (Info received from the headset)
• Preprocessing (filtering out the noise)
• Dimensionality (reducing random variables)
• EEG evidence
• COntextual Evidence
• Joint interference

Available Tech

• Matrix Speller
• first goes column by column then by each value
• Works as a loop until the sentence is completed
• The matrix can be rearranged like how a keyboard is set up in a specific way
• 7.8 characters per minute with 80% accuracy
• Study Case for the Matrix Speller
• This is called the Brain-Computer Interface Virtual Keyboard for accessibility
• 95 keys that had groups then sections then the values themselves, using the drill-down approach
• The accuracy was 61.25%, thus showing how a massive amount of variation in the matrix makes users spend more time selecting characters
• Ways to improve: many groups with the same type of keys inside each group, thus there is less variation
• Rapid Serial Visual Presentation (RSVP)
• A symbol is presented one at a time in the center of the screen rapidly and seemingly randomly (it is not actually random, it just appears to be)
• Depends less on eye gaze control like the previous, but only shows one character at a time
• 5 characters a minute, which cannot be used for conversations
• Must be improved
• A good filter in the program to take out the extra noise is essential (but how will we make a good filter?)
• Symbol selection can increase more after use
• Balanced-Tree Visual Presentation
• Groups in circles balanced in probability according to a Huffman tree
• When the group with the desired symbol is selected the symbols distribute themselves, and there is a "back" symbol in case a mistake is made
• The matrix speller was chosen for this project, but now the algorithm suggests probable words after a word is started.

6.1 Approach

• The approach begins with the input which is obtained with the Emotiv EPOC headset in this situation. There are 14 channels that are communicated over bluetooth to the computer
• The denoise is the first part of the software. The data received has to be cleared of the clutter that affects signal quality.
• Data mining then detects the user's intention, but many instances of the intention have to be recorded so that a relative pattern is found. Afterwards, the pattern triggers the command of the virtual keyboard in real time
• the GUI is made to suit people with disability in Thai and has a word guess function
• The output is shown and the user decides if it is okay, or decides to pass

6.2 Tools and Techniques

• They used a compiler called Eclipse and Java because they are more suited with it
• The testbench and control panel were a major help

The last portion of the paper were examples of previous works and images of those projects.

The next two sets of notes will be posted tomorrow, my internet isn't working that well right now.

Sorry!