Shimon is a robot that sings, dances, writes lyrics and even composes melodies. Now, he is going to tour to promote his new album – just like any other musician.
Shimon's new album will feature between eight and ten songs that Shimon wrote with its creator, Gil Weinberg, a professor at the School of Music and founding director of the Center for Music Technology at the Georgia Institute of Technology, where he leads the robotic music group. Shimon's album will be released on Spotify later this spring.
“Shimon was reborn as a singer and songwriter,” said Weinberg, in communication. "Now we collaborate between humans and robots to make music together."
To create music, Weinberg will start with a theme and Shimon will write lyrics around the theme. Weinberg gathers them and composes melodies to adjust them. Shimon also manages to create some tunes for Weinberg to use while putting together a song. Then, with a band of human musicians, Shimon plays the songs and sings.
If the theme is "storm", the bot will generate a batch of associated words, such as "rain", and then put those words together to build a letter. Shimon can also decide which words are most relevant and generate more letters based on that.
"I always wanted to write music, but I can't write lyrics," said Weinberg. "This is the first time that I actually wrote a song, because it had inspiration: I had Shimon writing lyrics for me."
Weinberg and his students trained Shimon with data sets of 50,000 jazz, pop rock and hip-hop lyrics. Shimon uses deep learning technology, a class of learning algorithms, to create his own words.
“The way in which semantic meaning moves through letters is different. In addition, rhyme and rhythm are obviously super important for the lyrics, but that is not present in other text generators. So we use deep learning to create lyrics, but it's also combined with semantic knowledge, ”explained Richard Savery, a doctoral student who works with Shimon.
When Shimon sings songs, he sings even with a unique voice created by collaborators from the Pompeu Fabra University, in Barcelona, who developed his voice and trained it in hundreds of songs.
Along with his new skills, Shimon also has new hardware that changes the way he plays and moves on stage. The robot is standing still, but has a mouth, new eyebrows and new head movements designed to help convey emotions and interact with bandmates. It also has new “hands”, which have totally changed the way it plays marimba – similar to the xylophone.
Shimon manages to count at the beginning of the songs to give the band an input signal and, sometimes, he shakes his hammers to the rhythm of the music.
However, this robot does not come to steal the place from humans. Teaching Shimon new skills is not the same thing as replacing musicians. Partnership with people is essential for Weinberg.
"We are going to need musicians and there will be more musicians capable of making more and new music, because robots will help them, generate ideas, help expand the way they think about music and play music," said Weinberg.
Shimon, Weinberg and the whole band are creating a tour with the aim of bringing their unique mix of songs created by robots and humans to more people. Weinberg said he expects these programs to be more than innovation. "I think we have reached a level where I hope the audience will just appreciate music for the sake of music."
“This is a song that humans, by themselves, would not have written. I want the audience to think: There is something unique about this song and I want to hear it again, even if I don't look at the robot ”.