+Brain/Thought Controlled Music+

[1:11]

This is a thread where i blog my experiences working with the Emotiv EPOC Headset for music related applications.

Apparently i was a Good Boy this year, because Santa ordered an Emotiv EPOC Headset just in time for Xmas. Santa sure knows me well!!

i've always longed for a device that could read musical activity in peoples brains and present them to the world. While the current technology is nowhere near that dream, we're beginning to see the first steps towards accomplishing just that with devices like the EPOC. There's a good chance that this technology could be as much of a game changer as the synthesizer, but time will tell.
Emotiv is having somewhat of a crisis figuring out a business model that works best for them. They've gone the way of Proprietary and seem to want to have a firm grasp on the applications made for the Headset. It doesn't matter much how i feel about the situation, because i'm not familiar with Business practices, so i couldn't make an informed decision. But i will say that Emotiv probably won't be writing programs for the types of things i'd like to do with the EPOC headset. The possibilities are endless for applications based on the hardware. Again, time will tell.

i will be receiving the Consumer level product, which means i won't have access to full EEG data streams through their software, but can be accessed through other software. Also it means i'm not supposed to create software for it. No problem, since i am not a programmer. Thankfully there are already programs to convert the EPOC data stream into OSC, so i can use Max for Live to customize a set up for performances.

Right now, many of the current users of the EPOC are using it to control lighting during shows. There is a substantial time lag from processing the brainwaves that makes it unreliable for musical timings. This timelag should continually become smaller as the tech becomes more refined. But even controlling your lighting by thought is a fantastic start.

But the EPOC isn't just about brainwaves. The headset has a gyroscope to detect the position of the head, and the sensors can also detect facial expressions and eye blinks. The Gyro allows for fluid real time control without using your hands NOW, while waiting for more dynamic processing of the other features. i am a BIG fan of Korgs Kaoss Pad as a musical interface, and the Gyro can easily be mapped in Max/Live to behave just the same, but with an added dimension of control. For example, assigning Filter parameters to the coordinates of the Gyro to use your Head to do on the fly Filter Sweeps with no hands.

Here is an example of using the EPOC to start and stop Live using brainwaves. This is simple of course, and it'd be easier to just press the spacebar, but this is a developing technology still much in the experimental stages.
http://www.youtube.com/watch?v=J_wbUIoMAVA

---

images.jpg
Description:
Filesize:	8.06 KB
Viewed:	8751 Time(s)

---

[BobTheDog]

Nice!

Please keep us updated with your progress. These look like interesting devices.

Andy

[pyrosonic]

Yes,please do keep us posted as to the results of your experiments.This is exciting stuff!

[1:11]

Shipping was faster than expected, ordered on Monday and sitting at my door on Thursday. i appreciated that they got it here in time for Xmas Eve, gotta love that!!

This is what you get in the Consumer Level product:

-MK3 Headset
-USB PC Dongle
-Saline Hydration Fluid
-Electrode Pads in a Hydrator Case
-Rubber Ear Pad with Electrode Body
-Charger
-EPOC Installation Disc/Software

Here are pictures of above, including some pics of the Guide and Box.

---

IMG_0001.jpg
Description:
Filesize:	2.4 MB
Viewed:	227 Time(s)
This image has been reduced to fit the page. Click on it to enlarge.

---

---

IMG_0003.jpg
Description:
Filesize:	2.72 MB
Viewed:	254 Time(s)
This image has been reduced to fit the page. Click on it to enlarge.

---

---

IMG_0004.jpg
Description:
Filesize:	1.96 MB
Viewed:	229 Time(s)
This image has been reduced to fit the page. Click on it to enlarge.

---

---

IMG_0006.JPG
Description:
Filesize:	2.2 MB
Viewed:	237 Time(s)
This image has been reduced to fit the page. Click on it to enlarge.

---

[BobTheDog]

Have you tried it yet?

[nobody]

I'd also love to hear about what you come up with.

Maybe it's my current mood, but I feel like I'd rather control my musical devices physically, simply because it sounds to me like it's more fun to physically interact with my instruments.

But I won't deny the huge geek factor to this.

[Inventor]

[BobTheDog]

I would like one for mind control over small furry animals and those pesky kids.

[nobody]

[1:11]

Haven't started working with it yet, won't be until after the New Years Eve Show. All my brainpower (which isn't much ) is going to that. But come January first....it'll be set up!
i also will probably need to get a haircut to allow proper contact of the sensors.

AudioD: For now, you still have to play instruments physically. Probably have another 5-10 years until the tech is refined enough to become usable as a stand alone controller.

Geek Factor? Does this guy look like a geek to you?

---

emotiv2.jpg
Description:
Filesize:	87.38 KB
Viewed:	8524 Time(s)

---

[nobody]

[nobody]

[1:11]

[1:11]

Last night, i finally started to work with the EPOC, which was no simple affair...but that was expected, just in a different way.

Before you work with the headset, you must wet the sensors with contact fluid, and attach each sensor before turning it on. Then comes the hardest part, putting the headset on.

These headsets really aren't made for people with hair!! Immediately after attempting to place it on the head, hair was tangling with the headset and popping contact felts off. Which then, you got to wrestle with it again to get it out off your hair, and look on the floor for the popped off felt. i went through this 4-5 times throughout the night.
Popping felts distract from the experience in a big way. If you don't have all the contacts proper, the system won't work right. They don't attach the felts because you must take them off to avoid molds from growing in the contacts.
Also, the felts only stay wet for around in hour in my conditions....subtract about 20 minutes for set up time, 5 minutes replacing felt, and another 5-10 minutes adjusting the headset, you only get about half the time of actual HeadWork that you put into it. Hair soaks up the solution, and before you know it, they are too dry to be usable.....and you then have to repeat the process. This is a MAJOR hurdle in making the unit usable for regular use. The average person on the street won't go through all this.

Emotiv has been saying they are developing a dry headset. This would be essential to the company's success. The wet set is just NOT for regular users.
Having both mental problems and cortex damage, i've had many EEG's. Usually the electrodes are attached with a gel and a strong adhesive by a professional. The design of the headset does a good job of getting the electrodes in proper alignment, and getting proper contact is just a little harder than regular EEG techniques. The problem again, is in keeping the contact. Just taking a drink can alter the position, and you're back to fiddling.

Most of the difficulty is with having hair......so i will be cutting it. My girlfriend is a top notch stylist, so this is no problem. Though, a simple Mohawk would work well.

On to the software......(see next post)

[1:11]

The software works wonderfully!! Yes, there are some kinks...but overall, this system works better than i imagined, but in a different way. The strength of this system isn't in brainwave analysis as the company promotes, it's in the facial expression recognition system.

You're led to believe from demonstrations of this product, that the software can learn certain thoughts/mental states to control virtual objects and such. Like, think "Forward" and the software filters/analyzes the EEG data and triggers an Event. It CAN work this way to some degree, but not effectively or reliably. At present there are too many variables to effect the results. Turning a light or seeing the cat come in can completely change your brainwaves...but people still can achieve reliable results, how?

From the Facial Recog. When you think of task, there are minute facial movements that you are usually unaware of. The sensors pick up on these quite well, and end up in the EEG stream. So when you think, "push", and tense up your jaw, or blink while thinking, the headset learns these physical actions, and then triggers the Event. Since these behaviors are usually subliminal, it's easy for a person to believe the headset pick up on their thoughts, triggering the event.
In order to Truthfully trigger an event by thought, you must consciously make an effort to have a blank face at all times, and have meditation skills of a Hare Krishna.
The idea of using thoughts to control in a concert situation is an unlikely one. The sensors are very very sensitive, just touching the headset causes the sensors to overload....so if you're performing with high Sound Pressure Levels, it's going to send the EEG out of wack, making it unusable (in that way)

So WHAT IS THIS THING GOOD FOR?
next post

[1:11]

What the software is good at is analyzing facial expressions, General levels of brainwave activity (Theta, ect.), emotional states, and head position. Once the brainwave analysis/filtering becomes more refined, it will become more feasible as tool. But the other features mentioned are definitely usable for a musician NOW.

The demonstrations of the product you see on the internet and TV are accomplished by raising/lowering an emotional state, rather than thinking it. Different emotional states can be used to trigger different events. Which is obviously why the company is called Emotiv.

It would be very easy to trigger stage lighting, alter BPM, ect. by emotional state. For example, have the Tempo raise or lower depending on how excited or calm you are. Or triggering Blue lights while feeling down, or red lights when excited or scared/angry.

Events could also easily be programmed to be triggered by clenching your teeth, smiling, blinking, ect. Providing an added aspect of control without using your hands.

Head position is determined by a gyro on the headset. This works excellent, much better than expected. There is software provided to allow you to move a cursor on the screen via head position. i was surprised at how well the cursor was positioned and controlled. You look at something on the screen, and the cursor is pretty much there. This is great for working in a DAW and playing Live. You can get to the buttons you want to push much quicker than with a mouse. You can program a jaw clench or other facial expression to act as the mouse click, that way...you just look at where you want to click and clench your jaw. Very useful for de/activating tracks quickly.

Another useful feature using emotional state as the focus, is analyzing media. There's an example of a guy using the headset to analyze emotional content of Videos, and then being able to search through videos using emotion.

This can be useful in providing feedback to musicians on how their music effects a person, and can provide insight to what parts excite, or at what parts the music loses the interest, and regains, ect. This could prove to be invaluable.

i will be demonstrating this soon with music from this site. i will be demonstrating the effects on emotional states from pieces that put me in a relaxed state (in the zone), ones that raise my energy or excites/scares, and ones that trigger a range of emotions.

i have work to do........until next time!!

[nobody]

Wow! That sounds like a lot of fun! I'm looking forward to hearing your demos. Maybe you could make a video of you using it, too.

[brontosaurus19]

What a cool concept! You'll definitely have to post some demos to give us an idea of how it works.

[1:11]

i have some videos demoing the headset, but dont have any harddrive space to render the vid from the camera.
Have the effects of Drones and Pop on brainwave activity.

i haven't forgotten about this thread, it will be continued, as soon as time allows.