DJ Mindwave - Crystalise your thoughts

Project Owner

  • Talented Student Program (TSP)

Project Credits

  • Jim Cook (TechLab, ICT)
  • Jai Honeybrook-Carter (TechLab, ICT)
  • Stuart Esdaile (Sydney Admin, ICT)
  • Daniel Burn (Applications Manager, ICT)
  • Philip Poronnik (Professor of Biomedical Sciences, School of Medical Sciences)
  • Margot Day (Associate Professor Physiology, School of Medical Sciences Bosch Institute)
  • Anthony Masters (Associate Professor Chemistry, School of Chemistry)
  • Janet Mao (TSP Group Lead)
  • Safat Sufian (TSP Student)
  • Rosemary Menzies (TSP Student)
  • Rowena Kok (TSP Student)
  • Vincey Au (TSP Student)
  • Laura Sayers (TSP Student)

Start Date

4th April 2014

Status

  • TSP Showcase including a live demonstration completed on the 28th May 2014.
  • Submitted to the ACUR 2014, and accepted 8th July 2014
  • Project was used by the student experience office for open day, 30th August 2014
  • Presented at Posters in Parliament, Canberra, 22nd September 2014
  • Awarded Prize for the Best Poster by peer vote at Posters in Parliament

Problem Statement

The TechLab and School of Physiology joined forces to offer a TSP project for Semester 1 2014.

The Pitch was: The brain is an amazing collection of cells that generate electrical signals. We can see these signals using EEG technology, but can you use your brainwaves to perform remote tasks? Crystallize your thoughts into a concrete physical outcome. The recent advent of cheap portable Bluetooth EEG headsets allows us to output our brainwaves as digital data streams that can then be used to control remote devices.

For this project you will work with physiologist and IT engineers to develop an app that uses EEG headsets to perform a task of your own design. What about a Sphero soccer game? Or Mindtunes?

To succeed you will need to complement your creative streak with some neurophysiology and programming skills. Interested? Check out emotiv.com for inspiration. This project is in conjunction with the ICT TechLab, which means you will use the coolest and latest devices to help you develop your app.

Once in the Lab for the initial kick off, the Students landed on the concept of using an EEG headset to control a Novation Launchpad

Can we control music creation and mixing with only Neural activity and blinking?

Final Brief

Our 8x8 square pad is divided so that each column represents a separate instrument group (i.e. drum, bass) and each row represents a different genre (i.e. jazz, funk, hip hop). We move up through columns by level of concentration, whereby reaching an “eSense Attention” threshold of 75 (out of 100) will program a continuous change from one square to the next through that column. A blink engages the row and an increase in concentration (beyond 75) will cause the program to move right continuously through the chosen row. Another blink will engage the desired square, which will then turn on/off the desired track.

Furthermore, we defined visual feedback – i.e. a red square to show where we are on the Launch Pad before we make a selection and for the desired square to change colour upon blinking (selection). In addition, the first row of the Launch Pad is blank to allow us to cancel out all sounds in the row. We also incorporated the pentatonic scale into the pad in the final column. The pentatonic-associated squares will fire according to the following thresholds set for our eSense Attention:

  • 1st note: 0-14
  • 2nd note: 14-28
  • 3rd note: 28-42
  • 4th note: 42-56
  • 5th note: 56-70
  • 6th note: 70-84
  • 7th note: 84-100

Challenges & Learnings

  • C1. Migration from Scratch to Python required a lot of refactoring. It was decided that a reliance on online resources would reduce the ability of the project to travel with no Internet connection.
  • L1. Luckily the team was able to bring in external resources to assist with this refactoring.
  • C2. Ableton does not have a robust API which made initial tests difficult.
  • L2. This was overcome by having python control the “keyboard” and assigning hotkeys for actions in Ableton, while considered a “hack” it was effective enough for a 12 week 1st year undergraduate project.

Languages / Framework

Python, MIT Scratch, Neurosky Mindwave EEG, Abelton Pro

Links to Resources



Tags: