While researching on music visualization I came across colour scales, studies from the beginning of the 18th century till the present day of all of the possible correspondences between the elements of color (hue, saturation and value) and those of sound (pitch, amplitude, and tone color), the most often proposed mapping is of pitch to hue. Many such mappings have been proposed and some were built into light instruments.

Brainstorming after tutorial

After the tutorial we decided to change the direction we were heading in and go back to our original idea of creating music with movement, trying to leave the narrative behind and concentrate on the Orchestra metaphor.

This lead us to the reason behind the project, the why? Our reasons would be trying to change an auditory experience, such as music, in to a physical an visual one. We would like to create and modify sound by movement.
We haven’t completely dished the idea of controlling video but we are putting it on hold as it can be added at a later date to make the experience more complicated.

During the following days we will conduct some research on the orchestra interaction, the user’s tastes to create a user profile, we will continue to investigate the technology behind Kinect, and on visualisation of music.   


Max for Live, Ableton Live, Kinect integration. Possibility that complex gesture could be trackable by stable motion tracking function. 


Tutorial with Mazie

During our tutorial with Mazie it was brought to our attention that perhaps a news of the world content would not be suitable since it might confuse our users. It is difficult to create an art installation using a current event, they might think it is a new way of presenting the news.

She suggested we should do some more preliminary research by approaching our users and asking what they might be interested in seeing. This would help us with deciding on the content of our installation.

She also suggested we should implement Wizard of Oz testing at an early stage even if we do not have a working prototype or content just to prove and see if our user understands and engages with our project.


Max/Msp Video effect by sound



Kinect gesture tracking test with Processing


Our second idea

Finally at the end of the day after throwing all sorts of concepts around we came up with another possible idea. It evolved from being followed by in-animated objects, to small balls that would gather and follow you while you weren’t looking, to being able to control the balls with gestures. The concept and meaning behind this idea needs to be determined but we did think that social stigmas such as follow the leader could be applied to it.


Today we tried to define and rethink the “wand” idea as we were preoccupied about the meaning and concept. The idea of music evolved to become a journey similar to storytelling where you could travel trough time using the movements of the wand.

We suggested several themes such as evolution, space or a more current topic, phone hacking. The idea morphed to data visualisation and how using movement you can skip through the events. 

We tried to develop the idea and map out what we thought the interaction should be like.

The conductor idea consisted in the possibility to be able to manage the volume of different instruments through the movement of the arms and wand. It would be classical music and the movement and sound would be represented on the wall from a projector. 


During the last two days we have been brainstorming using different methods presented by our Usability specialist Tj.

We have participated in a session of body brainstorming where me did several exercises to come up with a user. Starting by defining their age, occupation and interests, we later moved on to create a persona, we imagined what this person would be feeling like, what they would be saying, doing, hearing… the purpose of this exercise is to allocate a need the user might have. During one of these sessions Viktor came up with the metaphor of a conductor (the context we were in was a user that likes classical music) and we derived to create an experience they would find appealing.