echoprint

Announcing Echoprint

We’re ecstatic to announce the release of Echoprint, a fully free and open source music fingerprint and identification service powered by The Echo Nest. We’ve partnered with our dear friends at Musicbrainz and 7digital to bring developers a music fingerprint for free so they can build the future of music. Read the press release for more details!

The Echo Nest has been focusing on a crucial component of the oncoming music cloud for some time: we spend a lot of time and engineering resources on music resolving. This extends from mapping a query for a band name to its ID, to uncovering mentions of songs on blogs, to identifying the song in an audio stream without any metadata — otherwise known as fingerprinting. The Echo Nest’s existing fingerprint technology, "The Echo Nest Musical Fingerprint" aka ENMFP, has been in wide use privately and via our API for 18 months. Today we are unveiling a new fingerprint technology called “Echoprint,” whose main feature is its complete openness — everything from the program to analyze the audio to the server and data to make the match are available for anyone to use, under a permissive open source license, for free.

Music resolvers such as Shazam, Soundhound, Gracenote and Audible Magic work by “listening” to the music and then searching for features in the audio in a large database of content. They find a match in a canonical song ID, and return that to the user. Some, like the mobile apps of Shazam and Soundhound work “over the air” — where audio can be identified in noisy settings over a microphone. And others work only on files, by computing hashes on mp3s stored on a hard drive. All of them share three components:

  • the “code generator” — the piece of software that takes an audio signal and computes features or “codes” to look up,
  • the server — the hardware and software that can quickly look up codes in large database of reference songs and find the answer, and
  • the reference data — the database of millions of songs that can be resolved.

Echoprint includes all three. And, for the first time, all three are being released under open source or permissive licenses. The code generator is open source and free to download and build into apps for any platform — Mac, Windows, Linux or mobile (Android, iOS or Windows Mobile). The server is open source and available for anyone that wants to host their own resolving service. And the data, at first provided by our content partners at 7digital, is available for free download provided that any new additions are shared back. This data sharing approach ensures that over time, the Echoprint data commons will grow to every song fingerprinted by all users, and that data will be openly available to any developer. The Echo Nest will run and maintain the first Echoprint resolving server but we expect others to host their own.

Why an Open Source Fingerprint

We believe that resolving music should be something that belongs to the community. It should be a high-quality, relied-upon internet service, like DNS or email.

The catalog of current commercial music resolvers is unnecessarily limited — you cannot submit your band’s music to Shazam, for example — and we consider this to be a fault correctable by opening all parts of music resolving and recognition.

Today, application developers are forced to spend more time than they want to on song identification and basic metadata issues. We hope by opening up this resource, developers can focus their time and effort building creative new music experience for fans, not cleaning up metadata.

We think the potential of music experiences can be vastly improved with free and open access to music resolving and recognition. Allowing listeners the freedom to identify any song they hear out in the world or to clean up their collection or sync it with cloud services opens up a new class of music apps. This will become even more important during the upcoming period of the ‘big move’, when millions of fans will be moving music (often with messy metadata) from a collection of hard drives kicking around their house to cloud and locker services.

Using Echoprint alongside other Echo Nest APIs such as Rosetta Stone and Taste Profiles allows developers to build apps that can “convert” a local catalog of music into any streaming or music service. For example, Echoprint can take any audio source — a phone’s microphone or a file on a laptop — and identify the artist’s Facebook page, Rd.io ID or build a playlist on 7digital.

What we hope people will build

Any developer can now build music recognition into any app for free, with no licensing fees and no need to host their own server or collect their own data. We include sample fingerprint app source code for iOS. Just by including the provided open source projects into any app, a developer can now make their music experiences “resolving aware.”

  • a social music app could scan a users’ local music collection
  • a music game could detect the song playing in the room and show metadata from other Echo Nest APIs.
  • a hardware box could go inbetween your turntable and your stereo to “scrobble” your vinyl on last.fm or Facebook
  • your phone can figure out which movies or television is playing automatically from the soundtrack

Features

Like Shazam or Soundhound, Echoprint works on files or “over the air” in noisy environments.

The code generator can scan audio at roughly 250x real time, i.e. an entire song can be scanned in half a second on a single processor. Enough audio to make a query can be scanned in less than a tenth of a second.

The identifier needs only 20 seconds of audio to make a match, from anywhere in the song. Our reference database contains full songs and the queries can be much smaller and from anywhere in the song.

The server component is fast and scalable. A single server-class hardware box can support 60 queries a second and can horizontally scale.

Where to get it

— Brian, Dan, Alastair and the rest of the Echo Nest

The Echo Nest create Echoprint to bring Shazam-style recognition to other apps

"the opportunity here is for this music fingerprinting technology to make its way into a wide range of apps, sitting alongside other features. The Echo Nest cites social networks and location based services as two categories that may benefit, for example.

A streaming music service like Spotify might use Echoprint to bring song identification into its own app, rather than relying on its current partnership with Shazam. You can even imagine some developers creating games around the technology.

The key point is that developers will now be able to play with audio fingerprinting technology resolving to a database of 13 million tracks, complete with 7digital’s store for track purchases, and streaming service APIs from the likes of Spotify and Rdio.”

READ more on The Guardian  or Hypebot

READ Echo Nest’s blog to see how it all works

This week at GeneralAssemb.ly (review) wdi

So this week has been all about databases — specifcally, PostGreSQL and how and why it’s superior to something like SqLite3. It boils down to being more efficient and faster, and much better at handing large datasets. (Saying Article.length in ruby with a large dataset would take a very long time compared to PostGres.) 

Dan had us working on a huge Wikipedia dataset (he downloaded pretty much all of Wikipedia — about 20 gigabytes’ worth) and challenged us to create a parser for it. The parser would go through the entire dataset, pull the title and body of the article through XML markups, and create a SQL command in a separate file of all elements we wanted. I’m still working through smaller datasets before I tackle that behemoth, but it’s been a fun challenge so far. PostGreSQL and parsing is definitely going to be one of those tough subjects that’ll take some slogging through, but give the benefit of big rewards. 

We also worked with Carrierwave which lets us upload files through the browser (say, for pictures or audio). Then we worked with SoundManager2 which is a full-fledged HTML5 and flash audio player. It detects your browser and if it can’t handle HTML5 it will render the audio in flash. Echoprint was another awesome program that could output a text string of an audio file (its soundwaves), match that with all the songs in their database, and return information like artist and track name for the song. I was able to upload an mp3 without giving it any info and Echowave was able to tell me its artist and track name each time — very cool. 

HTML5 has some pretty cool features like their <audio> tag but unfortunately, it doesn’t work with Firefox. Not wishing to alienate a large percentage of users, we went with SoundManager2 since that’s compatible with pretty much all browsers. We are now working on creating a quiz-like app that fetches similar artists for the track from Echoprint and shows a mini-quiz asking the user which artist sang that track. I’ve got the similar artists bit working so far and am now working on making it more quiz-like. 

It’s really something to see a website do cool, interactive, useful things and being able to say, “I made that.” The sense of achievement is extremely rewarding and fulfilling. 

How Echoprint Is Making Political Ads Transparent

When we announced Echoprint, the world’s first open-source audio fingerprinting engine, we knew it would help developers build the ability to recognize songs into apps, but we never suspected that it would play a role in political transparency too. That’s exactly what it is doing, much to our surprise.

Ad Hawk, a free app for iPhone and Android from the Sunlight Foundation, allows television viewers to identify any American political advertisement as it airs, by fingerprinting its audio with Echoprint. Then, the app tells the viewer exactly who funded the advertisement.

No longer can political advertisers hide behind a cloak of anonymity. Ad Hawk, powered by Echoprint, is there.

Once the app identifies the add, it tells the viewer not only who funded it…

…but also how much the organization has spent; how much cash it still has; where the ad is airing; the organization’s ratio between positive and negative advertising; and whether its ads have targeted Democrats or Republicans:

In order to pull off this politically-clarifying magic, The Sunlight Foundation monitors media reports and YouTube for new advertisements, then adds their audio fingerprints to the Echoprint audio fingerprinting database. The app then references that database to identify the advertisement.

The Sunlight Foundation is “a non-partisan non-profit that uses cutting-edge technology and ideas to make government transparent and accountable.” We’re pleased that Echoprint is making their job a little easier.

(Oddly, this isn’t the first time our technology has gotten tangled up with politics. Our new Taste Profiles technology offers musical insights regarding Democrats and Republicans.)

State of the Union: Echoprint, the Open Source Audio Fingerprinting System

Last year, The Echo Nest unleashed our completely open-source audio fingerprinting system, free for any developer to integrate into their applications on any platform: Echoprint.

Before that, it took years of research and development to build a system to identify music based on how it sounds. In fact, that’s basically what we did before releasing all of that work into the wild under an open-source license. Some people want to teach the world to sing; we want to teach computers to hear.

Our goal with Echoprint was to offer developers and commercial music services a more open alternative to closed commercial offerings, but with comparable performance. Hundreds of developers and several companies have already taken us up on it. Many popular music destinations now use Echoprint behind the scenes — either to de-duplicate music posted on blogs, get ad revenue to the correct rightsholders from streaming audio, or sync music fans’ local collections to a cloud service. In a turn we didn’t predict, three separate hot social TV startups (that we know of) are also using Echoprint to detect shows or commercials automatically, based on the audio detected by a mobile device.

After spinning off Echoprint as open-source software, The Echo Nest Team enlisted a clever man with a PhD in London — Andrew Nesbit — help us administer the thing. But the real stars of this situation are all the developers who have been using and improving Echoprint. We think it’s just such a great idea, because it lets developers recognize songs without spending years to reinvent the audio fingerprinting wheel. It’s our open-source gift to the world.

Nesbit has been crushing it, providing excellent support to The Echo Nest’s ongoing efforts to grow the developer community around Echoprint. This is crucial, due to its open-source license: The more people use it, the better it gets.

There’s no real limit to the number of applications Echoprint can enable.  Here are a few we think will be popular:

Mirroring Music to the Cloud

You know how Apple’s iTunes Match can scan your iTunes library, match it against Apple’s online repository of music files, and allow you to play replicas of your songs from your locker in “the cloud” — all without actually uploading the music? Echoprint can do that, either for your own catalog or matching your personal audio to our Rosetta Stone ID space partners.

Monetizing User-Generated Content

When people upload a video, slideshow, or other content whose soundtrack is a popular song, online video companies need a good way to understand which song it is, so they can make sure the appropriate copyright holder gets compensated under whatever license they have. Echoprint can help with that.

Acoustic Identification

As it stands now, Echoprint has limited over-the-air capabilities for identifying songs in near real-time, which allows users to hold up a smartphone in order to identify whatever is playing. This feature still has a ways to go. Some Echoprint developers have been experimenting with it, and have already made significant improvements, but eventually, any app developer should be able to build this functionality into their apps for free.

We think any app should be able to identify songs for free, because it makes the world a better place when music fans and computers can understand what a given song is. As such, Echoprint will always remain completely open-source on both the client and server sides.

It’s been quite a year. We look forward to more great things from Echoprint over the years to come.