Comments Sync

This is a web-base videoclip, using data from SoundCloud, created during the Music Hack Day in Barcelona, June 2012.

I first wanted to fetch the comments for a track and the location for each user posting a comment, and create a slide show playing the song and the comments, and the panorama associated to each comment. It turns out around 4% of soundcloud users have a usable location, so I had about six panoramas por a three-minute song. Not too promising.

I decided to keep the idea, but sync the movement of the camera in the panorama to the comments. The more comments in a given moment, the more motion the camera has. The panoramas are still there, and change every few comments.

I check the comments for a given track, check if the user posting has its location in their profile, resolve the address with google maps geocoding service, fetch the street view panorama for that location using GSVPano.js and create a spherical panorama using three.js and WebGL. That creates the slides used as backdrop. Each panorama is renderd with a shader that applies some tonemapping.

I then play the song with soundcloud's streaming api, and listen to the comments. Everytime there's a comment, I shake the camera a bit. You start noticing that in some tracks, comments are clustered on the best parts, which make for a good "sync" without accesing the audio data.

Cool, thanks.