[cancelled; rescheduled] SoundSignature: What Type of Music Do You Like?

Brandon Carone (NYU) joins us to discuss his PhD research.

SoundSignature is a music application that implements various Music Information Retrieval (MIR) Python packages to analyze the acoustic characteristics of users’ favorite songs. It integrates a customized OpenAI Assistant, which combines the extracted acoustic/musical features of each song with its extensive knowledge of the artists and songs. Capitalizing on this combined knowledge, SoundSignature provides users with personalized insights into what characterizes their musical preferences, akin to a musical preference personality report. Users can then interact with the chatbot to explore deeper inquiries about the analyses and their musical taste. This interactivity transforms the application, acting not only as an informative resource about familiar songs, but also as an educational platform that enables users to deepen their understanding of musical features, music theory, acoustic properties commonly used in signal processing, and the artists behind the music. Beyond general usability, the application also incorporates several open-source musician-specific tools, such as a fundamental frequency tracker (CREPE), a chord recognition algorithm (CREMA), a source separation algorithm (DEMUCS), and an audio-to-MIDI converter (basic-pitch). These features allow users without coding skills to access advanced, open-source music processing algorithms simply by interacting with the chatbot (e.g., can you give me the stems of this song?). In this paper, we highlight the application’s innovative features and educational potential, and present findings from a user study that evaluates its efficacy and usability.

NOTE: this event is part of the DL4MIR workshop series (ccrma-mir.github.io); guest speaker talks are open to the broader CCRMA community.