Will AI replace the musician

AIVA: Artificial intelligence composes the music of the future

Ashkhen Zakharyan, Birgit Götz

A contribution about musical photographs and the future, in which every person can have their own soundtrack, based on their own life story and personality.

EnlargeAIVA composes emotional soundtracks - individually for each story, each person.

AI, creativity and personalization

Artificial intelligence is undoubtedly the technology of the 21st century that will change the most, but it can also improve human possibilities in different ways.

Many companies are already pushing to integrate AI systems into their products and services in order to personalize their customer relationships and thus survive in the new digital revolution, but also because this disruptive technology can help companies to increase productivity and profitability .

The ability of AI is often seen as helping people in manufacturing processes, driving self-driving cars, suggesting the most relevant search results on Google, or beating human champions at chess or the ancient Chinese game of Go.

Few think about artificial intelligence in the creative field, for example to paint new portraits in the style of Rembrandt or to compose emotional music for soundtracks.

AIVA: Learned from 30,000 scores

AIVA - Artificial Intelligence Virtual Artist - is an AI that learned how to compose music by reading over 30,000 scores by some of the greatest composers in music history.

AIVA was created by Pierre Barreau, Vincent Barreau and Denis Shtefan, inspired by the science fiction film "Her" by director Spike Jonze: The sensitive Theodore, who writes personal, soulful letters for other people, communicates with Samantha, a super-intelligent AI its operating system. Since she cannot take physical form and cannot appear in photos, she decides to write a piece of music to capture a moment in her life, just like with a photo.

AIVA's aim is not to replace human creativity, but to expand it and promote cooperation between man and machine. The demands on music are changing: every day, new content is created at an exponential rate, and new forms of entertainment now require high-quality soundtracks on a large scale.

Video games, for example, can contain hundreds of hours of stories, but they are only accompanied by two hours of original music. Players are annoyed by the repetitive music, a problem that prevents this art form from rivaling cinema in all areas. But with artificial intelligence, hundreds of hours of individually adapted music can be composed for games.

Are you skeptical? Then listen to one of AIVA's creations, played by 78 musicians from the CMG Orchestra in Hollywood: Among the Stars

This is how AIVA works

AIVA is based on deep learning: large amounts of data are made available to a computer system so that it can learn and create its own mathematical model of this data collection. All of these inputs are processed through a series of neural networks, an artificial representation of how our own brain works.

By reading the 30,000 scores by Mozart, Bach or Beethoven, AIVA was trained to understand music theory and the patterns behind the scores of geniuses. The AI ​​perfects its sense of what music is by looking at a few bars of existing scores and predicting which notes should come next in those tracks. AIVA's accuracy in this prediction will determine how quickly she will learn the art of composing.

But this art is very subjective and it is not enough to teach AIVA how to define music. To ensure that the AI ​​is composing the right music for the right person and the right project, we've trained our algorithms on what makes each score emotionally unique. We have also added over 30 different categories of labels to each score in our database. These labels contain the density of the notes in a piece, the mood of a score, the line-up, the composer's style and much more.

Once AIVA has learned the emotional aspect of music and the differences between the different styles, we can create a visual cluster representation of the musical DNA that looks like this:

EnlargeCluster representation of the music DNA

In the picture above, each score is visualized by a point, and its color depends on which composer wrote the piece. You can see that similarly colored dots tend to cluster together, indicating that AIVA is able to figure out information about the style of the scores.

Ultimately, this understanding of music DNA allows AIVA to respond to specific requirements when we are commissioned to write a piece, and so it succeeds in composing the right music for the right project.

In May 2016, AIVA became the first virtual artist whose works were registered with a copyright society (SACEM). This means that all of their works, as well as those of human composers, are copyrighted.

EnlargeAIVA is the first virtual artist whose works are protected by copyright like those of a human composer

In June 2017, AIVA wrote the opening piece for the national holiday in Luxembourg. This piece was performed by the Luxembourg Philharmonic and the choir to illustrate the collaboration between humans and AI.

Since then, AIVA has also been known for other compositions, for example for the world's first AI-composed city anthem "Ode to Dubai", for soundtracks for Nvidia's keynotes and music for video games and short films. The AIVA team worked with pop singer Taryn Southern, who is also a composer, for the single "Lovesick" released on May 1, 2018.

What's next

Storytellers, film producers and the entire entertainment industry rely on music to turn moments into magic. We believe that with AIVA it will be possible to produce very emotional and individual soundtracks for interactive media such as games or VR.

However, our job is not limited to creating personalized music exclusively for the entertainment industry. Just like Samantha, the AI ​​who wrote a piece of music in "Her" to capture certain moments, we believe in a future in which everyone can have a soundtrack for their life based on their own story and personality.

The AIVA team believes that personalizing music will usher in a new era in music and open up a world of new possibilities.



Prof. Dr. Francisco Tigre Moura. AIVA Technology: An Extraordinary AI Music Start-Up (Oct. 17, 2017)

Marc Benioff. Livemint. Artificial Intelligence will be the Defining Tech of 21st Century (Sept. 15, 2016)

Khari Johnson: Venture Beat. AI Weekly: Machine-made Art does not Mean the End of Human Creativity (Jan. 26, 2018)

Chris Ward. D zone. AIVA - The Artificial Intelligence Composer