Music video for Catalina, a track off the Chocolat EP by Moullinex, released in January on Gomma Records. Please see it in HD!
Done with Kinect + Processing + Cinema 4D + After Effects.
How it was done
The 3d footage was captured using a a Microsoft Kinect, using Processing to store the sequence files and Cinema 4D R12, with some built-from-scratch python tag scripts to read these sequence files to Cinema 4D in real-time.
Kinect + Processing
We started with the Kinect interface library developed for Processing and made available by Daniel Shiffman. Check his page for instructions on how to get that part running. It’s dead easy!
Some modifications were introduced, to get our 3D data into Cinema 4D:
Recording of the sequences to plaintext files. Each file represents one frame with a coordinate map (point index, x, y, z lines) in plaintext (we know it’s not the optimal solution, but it did the job and allowed us to debug more easily - plus, we are lazy). Hitting the key ‘3’ would start recording the sequence, while storing it in memory, and ‘4’ would flush this recording to sequential files in the hard drive.
A threshold filter was added to enable us to filter out any points that were too far (‘too far’ defined by a variable threshold that you can adjust with keys '1’ and '2’). This way we could remove the living room entirely from the sequence.
Download the source here.
Cinema 4d + Python
Manipulating the sequences was gonna be the fun part, but first we had to make C4D talk with our sequence files. Importing them manually would be painful/impossible, so we wrote a little python script (supported by Cinema 4D R12 tags) that read these files in real-time, allowing us to tweak playback speed, resolution, etc. Also, we needed to exclude all our zero points, created by the threshold filter in Processing. The python script updates a selection tag which takes care of that. Read the how to file in the project folder if you want to know more.
You can download the C4D project (which includes the Python script) here, with detailed instructions and an example sequence from our recordings.
Editing and Post
After importing all our scenes into Cinema 4D, and getting the look we wanted, the rendered sequences were taken into After Effects and Premiere. Business as usual from here on.
Concept: Luis Clara Gomes, Luis Calçada
Visuals: Luis Calçada
Programming: Luis Clara Gomes
Editing: Francisco Costa
And a big thank you to Joana Nobre and Paulo Raimundo.
C4D project + Python script
More from us
Supporting the EP would be great. Thanks!
Buy the EP here, or listen to the tracks on soundcloud.