Hey Alexa, meet Music
A look behind the scenes of our winner entry for Alexa India Hackathon
Introduction
This story will talk about our experience at Alexa India Hackathon, held at Amazon during my tenure there as an intern in June 2019. I teamed up with two of my experienced colleagues Srini and Abhishek for the hackathon. There was no problem statement defined, but the idea proposed had to involve developing a skill for Alexa. Enough intro, onto the fun part now!
Inception of the idea
A lot of us learn or want to learn music out of hobby. A major setback of traditional music learning experience is that an instructor has to guide you not only through your learning phase, but also in the practicing phase. This part is time consuming and has the potential to be automated by Alexa.
Through our proposed Alexa skill, we wanted Alexa to act as the feedback provider during your practicing phase. In short, Learn music from a tutor, and practice it with Alexa! (This idea was a brainchild of Srini)
Overview of the proposed Solution
CAVEAT : Alexa is not allowed to listen to audio of more than 8 seconds in length. Also, Alexa cannot store any of the audio in any form due to privacy policy of Amazon.
Therefore, Abhishek made a companion app for this project. All a user needs to do is to play music with the companion app activated and press the upload button in app. This was done for the purpose of demonstration in hackathon. The production (hopefully xD) version will probably support direct recording user’s music on Alexa enabled device with due permission.
Here is what your personalized conversation with Alexa will look like -
The video demonstration is also available here — AlexaMusicAcademy
Scoring Mechanism
There are a whole lot of complexities in coming up with a similarity criterion for audio. Direct comparison will not work here ( what if your audio is 100% similar, but there is a time lag of a millisecond? Yes, it will be scored as 0% similar). Also, factors like noise and difference in calibration of instruments has to be taken into account. This was a problem which left music producer companies scratching their head for a long time. They simply could not afford a manual workforce to moderate all the copyrighted music floating around internet!
Storytime. In 1999, 3 friends with zero engineering background teamed up to solve this problem. They found another guy — a genius with 4 degrees from Stanford, and started a company called Shazam. Shazam invented the technology Acoustic Fingerprinting, which became an overnight hit among music lovers and companies as well. This was later acquired by Apple for 400 million USD!
We found an open source library called chromaprint which uses the same technology to compare audio files. The boilerplate code was taken from this brilliant medium article by a fellow Amazonian Shivam Aggarwal. Under the hood, it compares the audio files for different offsets and returns the result from offset with best match.
We have created a simple online audio similarity scoring app using the same, in which you can compare two audio files for similarity. Here is the link to webpage. You can try out the webapp HERE.
Happy Ending :)
About a week and a half after the demo presentation, we received a mail stating we were the WINNERS of the hackathon!
The hackathon was a great journey, involving a blend of joy of creating, enriching experience of working with a great team and a sleepless but fruitful night. I want to thank my team members Srini and Abhishek for giving me the opportunity to team up. This has definitely made hackathons my newfound interest, so hope to see myself participating in one soon!
