Making an Album With Codex
I wanted a Kanye West and Tame Impala song that didn't exist, so I asked Codex to help me make one.
I had no music production experience. I knew what I liked about songs, and I could usually hear when something felt wrong. Explaining it was harder. I'd point to a timestamp and say the verse felt rushed, or the music hadn't settled before the voice came in. I couldn't tell you which operation in a DAW would fix it.
Codex could handle that side. It could research software, write audio-processing code, and eventually operate REAPER. We ended up making five tracks and calling the album deus complex. Getting there involved finding a useful division of work: I could direct the music through what I heard, while Codex worked out how to act on it.
Choosing things that belong together
Our first Kanye–Tame attempt used “POWER” and “New Person, Same Old Mistakes.” Their tempos were close, which made the pairing look promising. But after several revisions, it still felt like two songs stitched together. A better transition would improve one moment without making the whole thing work.
I kept asking Codex to fix the mix. Eventually I had to reject the pairing.
We started comparing particular passages instead of treating compatibility as a property of two entire songs. Where does the verse enter within the bar? Which words get stressed? What chord is underneath a held note? Does the backing have room for that delivery?
Beat This and librosa helped locate beats and inspect rhythm. We tried ChordFormer for harmony and MuScriptor for notes and instrumental content. These gave us ways to narrow the choices before building a full arrangement.
Matching BPM still left a lot unresolved. Two rappers can use the same tempo very differently: one packs words between beats while the other leaves space. We'd slow a verse down to fit and lose what made the delivery good. I could hear that before I had the vocabulary to explain it.
The Kanye–Tame track we kept, “This Must Be Heaven,” paired “Touch the Sky” with “Breathe Deeper.” Choosing better material made the production work much more productive.
Giving Codex a studio
The early mixes were mostly assembled through Python. We used NumPy and SciPy to manipulate audio, Rubber Band R3 for tempo and pitch changes, and Pedalboard for effects. Codex could write a script, place clips, and render the result.
We used BS-RoFormer to extract vocals and HTDemucs FT to separate backing into drums, bass, and other parts. Those separated files, or stems, let us change the balance and arrangement independently. We also tested alternatives such as SCNet XL when a separation wasn't working.
The stems needed inspection. “I Wonder” has sampled singing behind Kanye's lead, and a vocal separator could keep both. That makes sense for its task, but it interfered with our new arrangement. A specialist cleanup model worked on a controlled blend and barely helped with the actual passage. Installing a more specialised model hadn't automatically solved our particular problem.
The bigger change was connecting Codex to REAPER through an existing MCP integration. REAPER holds the tracks, effects, and automation in an editable session. MCP lets the agent operate it through tool calls. We added a JavaScript client so Codex could inspect the project, trim and move clips, change effects, draw parameter curves, and render.
Now my feedback had somewhere more useful to go. If the backing dropped under a verse, Codex could inspect the relevant track and its volume automation. We could make an edit in the session and render it through the same effects, rather than adding another processed file to a chain of fixes.
That ability to build and connect tools was a large part of the experiment. When the setup couldn't do something, Codex could research an option or write the missing code. I could keep directing the music without first learning to implement every edit myself.
What the audio models were useful for
I also wanted Codex to get more information from the audio itself. We tried Gemini and Qwen audio endpoints, MOSS-Music-8B-Thinking, TinyMU, and Audio Flamingo. Some could describe instruments or changes in a passage, which gave us things to investigate.
We had to check whether those descriptions followed the audio. One Gemini response explained a timing difference between two identical clips. In a small MOSS comparison run, a broad prompt answered “same” for every pair, while a focused prompt picked A every time.
We also tried learned music-quality scores and ordinary signal-processing checks. The narrower checks were easier to use: find an unintended delay, check a level change, locate a dropout. They helped Codex verify its edits. My listening decided whether those edits made the song work better.
That was how we used the tools going forward. A model could suggest something to inspect, and a measurement could confirm a specific change. I was still choosing what sounded right for the song.
Fixing what I was actually hearing
“50 Calls,” our Kanye–Juice WRLD interlude, had a vocal passage that sat on the beat grid but still felt wrong. It had been shifted two beats away from its original place within the bar.
In 4/4, each bar has four beats. Move a phrase by two beats and its words can still land on beats, but against different drum hits and chord changes. At our 80 BPM tempo, the correction was 1.5 seconds. We also had to move the backing's release so it continued supporting the relocated line.
That was a useful translation of my complaint. “It isn't synced” had become a specific relationship between the source phrasing and the new backing. Faster Whisper helped us find likely word boundaries, but the cuts also had to preserve pickups and the ends of words.
The Runaway-based track had a different problem: Savage sounded muffled. A cleaner source and separation helped, but the voice could become muddy again when we put it on the backing. The instrumental was competing with it in the same frequency range, so increasing vocal volume wasn't enough.
We needed to inspect the vocal after its processing and compare it with the backing during the same words. Local EQ and balance changes could make room without pushing the entire verse forward. These were mixing decisions; adjusting the final master wouldn't fix a conflict between individual tracks.
The useful pattern was getting from something I could hear to something Codex could inspect and change. We kept the methods that worked in a music-production skill, so later edits could start from what we'd learned.
Making our own contribution
I wanted the tracks to have moments we'd created. We added tools for writing MIDI, including pretty_midi and music21, and instruments we could control from code or REAPER. FluidSynth and sfizz let us render sampled instruments; Surge XT gave us synthesizers. That made original piano parts, fills, and transitions practical to try.
I rejected plenty of them. The notes could agree with the harmony and still feel out of place. A slow piano passage after fast rap could drain the movement. Asking for more complexity sometimes just gave us more notes.
One custom bridge on the Runaway track kept bothering me. The rap stopped and this slow passage arrived. We eventually removed it and used a stronger point from Runaway's own harmony for that handoff. Our opening and closing piano parts had other jobs in the arrangement.
The Eminem rewind worked better. I asked for a meaningful part of his verse to repeat three times, then we shaped the return around it. That repetition changed the momentum of the song and gave us a moment that belonged to our version.
Even there, applying a fade didn't settle the transition. I kept hearing a sudden entrance. We eventually traced part of it to a background vocal swell and a strong attack. Changing the volume curve on one layer hadn't addressed the combination I was reacting to.
We used references from sample-based production and DJ mixes to work through these decisions. Hearing how another producer carried a groove through a change was more useful than repeatedly asking for a smoother transition. It gave Codex a concrete musical operation to study and adapt.
By the end, I was making decisions I couldn't have executed in the software: rejecting a pairing, leaving more space before a verse, keeping a rewind, or removing something we'd composed. Codex could work out the processing and build tools when the existing setup fell short. My taste gave that work a direction.
The five tracks came out of that back-and-forth. I didn't need to become a producer who could perform every operation by hand to have a say in how the music was made.