Concepts and Control: Understanding Creativity in Deep Music Generation

Abstract: Recently, generative AI has achieved impressive results in music generation. Yet, the challenge remains: how can these models be meaningfully applied in real-world music creation, for both professional and amateur musicians? We argue that what’s missing is an interpretable generative architecture—one that captures music concepts and their relations, which can be so finely nuanced that they defy straightforward description. In this talk, I will explore various approaches to creating such an architecture, demonstrating how it enhances control and interaction in music generation. Ultimately, we may find that the heart of this interpretability challenge lies in understanding content and style, a timeless question that resonates across art and life.

Ziyu Wang is a PhD candidate in Computer Science at the Courant Institute of Mathematical Sciences, New York University, and holds an affiliation with NYU Shanghai. Currently, he is a visiting scholar in the Machine Learning Department at MBZUAI. His research is conducted under the supervision of Prof. Gus Xia in Music X Lab, where he explores the intersection of music and machine learning. In 2019, he earned his undergraduate degree in Mathematics from Fudan University. Beyond his academic pursuits, he is a passionate conductor, pianist, and Erhu (a traditional Chinese string instrument) player. He has previously served as the conductor of the NYU Shanghai Jazz Ensemble and as the director of the Fudan Musical Club. For more information, please visit his website (https://zzwaang.github.io/).

Zoom Link: https://stanford.zoom.us/j/7733389381?pwd=Z2xkVUEvNTA3dTRSclBHUlNRZGFZdz09
Meeting ID: 773 338 9381
Passcode: 601462