Reordering Video Frames based on Audio Frequency Spectrum Similarity

Experiment by @mario-klingemann processes short video scene and rearranges frames in order of an audio spectrum, often placing very similar short sounds together one after the other - very surreal and possibly epilepsy-inducing:

The audio frequency spectrum of a video’s frames are clustered by their similariy using a t-sne embedding, Then the new order of frames is calculated using a nearest-neighbor traversal so frames that sound similar are following each other.

Another experiment takes a music video and orders the parts of similar sounds by their pitch, using ‘Never Gonna Give You Up’:

After calculating a Constant-Q chromagram with librosa the frames of the clip are regrouped by their pitch classes.

Installation for open house at my studio, The Yawn Shop, in Portland OR, fall 2013.

It saddens me that I may have to let this excellent space go for practical reasons, but that may not happen if i can find a studio mate or two.  If you see this post, live in the Portland area, and are dying for a space to work please email me at  The space has awesome natural light, 24/7 access, has amazing views of the city, and plenty of other amazing artists to meet.  Its $130 a month or less if i find two partners. please get a hold of me!


Outfit of the day, plus m*th*rf*ck*ng fr*me cl*st*r progress. Everything I am wearing today is from the Target clearance section (minus the shoes), and yes, I am one of those people who cannot just say “thanks” when someone likes my outfit, but must expound on the very price of said articles of clothing. Which is why I hide at home a lot.