— I used OpenAI's Jukebox algorithm to improvise over loops in the style of various artists like Cocteau Twins and The Weeknd, any of about 4000 artists it was trained on. I was going for a nostalgic, holiday theme, so I looked for loops and algorithmic models which supported that theme. It turned out that the most striking elements produced by Jukebox were the vocals — they sounded like lost ghosts. I liked the emotional quality even though they were sung by a computer.

Jukebox has the capacity to surprise, which is strange and exciting to say about a non-human system. It can also produce a lot of junk, so a big part of the process was listening to all the renders and picking out the best bits. I was also lucky enough to discover the source-separation algorithm Spleeter right before I started this project. That helped to open up the mix and focus on the vocals. After I had all the computer-generated parts isolated and repaired, it was like making any other track in Ableton.

It took two years to figure out a good way to incorporate AI into my workflow. I started using MIDI AI transformations for a Grimes remix about a year ago and then I tried waveform AI transformations with a lot of different algorithms. After that I took a few months taking apart and rebuilding Magenta's DDSP algorithm, which helped me to understand how neural nets think. I also had to brush up on my linear algebra!

— Knowing a little Python helps to get a better variety of results, but you could probably get by without it if you're good with computers. Some knowledge of Unix commands, digital audio, and remote computing is useful, too.

— The MIDI input is compressed into a higher-level representation of itself, called the latent space, which is thought to correspond to musical characteristics like style and genre but no one really knows for sure. The neural net has learned this latent space by analyzing large numbers of MIDI files, and once it knows how your input MIDI file fits in with what it has already learned, it can alter the MIDI in new ways. For example, it can naturalistically interpolate between two melodies or continue a melody (or drum part) in a way that makes sense musically.

Waveform transformations work on the same principle but they are a lot more difficult because digital audio has so much more information than MIDI. Some algorithms work by training on a big dataset and then rebuilding representative examples of that dataset sample by sample. Other algorithms take more of a synthesizer-based approach, using a smaller dataset and modeling learned DSP elements like sine wave generators, filters, and reverbs. All the algorithms have their distinctive sound, but the results depend mostly on the dataset audio.

— I know exactly what you mean. I get the best melodies when I'm not even thinking about music, or in dreams. Or humming along with a TV show. It can be very difficult to find those magic moments while sitting in front of a computer. So I'm happy for AI to do some of the heavy melodic lifting for me. But I'm usually feeding it some kind of musical seed, like a loop or chord changes, or the dataset. Also, there's something very appealing about working with an entity which has distilled so much human music in a way we can't entirely understand.

The biggest surprises come from Jukebox. It'll often suddenly break into a different genre or tempo, and most of the time this isn't what I want. But every once in a while the algorithm has a moment of coherence, usually just enough to extract an interesting loop. It's kind of like a crazy person who suddenly says something profound.

Usually a track is built around 10-15 of these loops. I used this technique on "December Song" and the Low and Magdalena Bay remixes.

— Ah, I love Burial! Such a unique sound. I think the ghostly effect you're talking about will probably go away with time, but some people will always want to use the older versions of the algorithm, like people want to use an old synth now. This version of Jukebox will always be there if I need it, but I will probably move on to newer versions and other algorithms, as well as non-AI music.

Right now I'm working on a Max for Live device which alters the MIDI in a clip as you perform. It looks a bit like cellular automata. I'll make that available for free when it's done. Also, I can't wait to get a set together and start performing once live music comes back.

— Awesome, thanks! I would have to say "LA Freeway" by Guy Clark because I can't stop playing it on guitar.