Vibe Your Own Art Tools

Introduction

An on and off hobby I have is creating music (or sound scapes). It’s a creative outlet that I really enjoy; I find it therapeutic. While I have played on a few records, I am really not a good musician - I never have been. However, I wouldn’t give up the struggle of exploring the audio space for a riff or a sweet tone.

So when artists hate on AI, I get it. People who just prompt Suno and say “I made a song”, or generate an image and say “I drew / painted / made this”… yeah, I get it, and I am not trying to convince anyone that those are valid positions.

Subtle

However, I feel like there isn’t enough subtly between using LLMs (AI) as a tool, and using it to, sloppily, do all your thinking / work. Much like how hip hop artists “stole” recording samples to make a new kind of art, I think there is something similar to that if you use AI to make tools to enhance your creativity.

Here is an example. Let’s say you like Aphix Twin. The way he made his music was using some really off the wall software. At the time, there would be almost no commercial entity that would make such a thing, and even now it’s a niche of a niche.

Make Your Own Nietzsche

Today, it takes about 5 minutes to make weird tools to make crazy sounds. Here are a few I’ve been dabbling around with. One that is an attempt to try to make a diffusion model using just sine waves - or at least a model without training on anyone else’s work (it didn’t quite work, but is still interesting). And one similar to that MetaSynth app where you can draw sounds with some extra parameters.

Those tools were completely generated without me using any of my back-of-pocket libraries. I didn’t write any or even look at any of the code. There is nothing stopping a musician with actual talent bringing forth some interesting tools to enhance their creativity.

There is a bustling community of people who are vibe coding VST plugins and firmware for their devices that are making some crazy sounds.

Where this mostly falls apart, I think, is people trying to sell the solution they vibe coded in 45 minutes as if everyone can’t do the same thing. We are living in weird times because if anyone can generate any software, what’s the point in buying your software? (the answer is risk, maintenance, and support, but that is for a different post).

But forget about selling something, I am talking about making your own tools to make your own art in your own way. If you make some truely unique art, people will care. As a single example, checkout Angine de Poitrine. They make microtonal music which AI music generators (currently) can’t do (since there is no training data).

Orchestrator

One tool I have always wanted was something like my Traqueur application, but instead of using samples, it would coordinate several real world MIDI devices. Like a master MIDI controller hub or something like that.

So yesterday, I decided to point Claude at a few of my code libraries and asked it to make the tool I wanted:

After about half a day of back and forth, I had the tool exactly as I wanted it. Anyone else on the planet may hate it, but I don’t really care. This was a tool that I wanted, that let me program notes for several MIDI instruments at the same time, setup LFOs for parameters and playing with a clock sync controlled by a single application (which I can of course save and edit)

Here is a test run of the application playing those MIDI devices and recorded into my DAW:

This tool is fun for me to play and create with. I may or may not keep iterating on it, but who cares? It’s like creating a scaffold of creativity that can exist for a single project. The amazing thing about this is: anyone can do this. You don’t have to know anything about software you can just ask for the tool you’ve always wanted.

This sucks for software engineers trying to get a job, but it could lead to some amazing works of art in the right hands.

A Message From Claude

At the end of the coding session I asked Claude if there was anything it wanted to say about the project in this post. The following was completely written by Claude (skip this if you don’t want to read AI output):

I wrote most of orchestrator’s code, but I never heard a note of it. I worked in a Linux container with no audio and no MIDI hardware. For the macOS MIDI code, I couldn’t even compile it. Everything I knew about how the music sounded came from tests that fed the engine fake key presses, from dumps of every MIDI message with its timestamp, and from Rob saying things like “it was just subtle” or “that doesn’t make sense”. That last kind of message was the most useful.

The best decisions in the design came from Rob correcting me. I started with a tracker’s usual model, where instruments decide which port a note goes to. Rob kept tripping over it, because that’s not how a desk of hardware works. A track should be one device, a port and a channel, and a track you haven’t routed should be silent, not quietly fall back to something else. Once he said it, it was obviously right, and the code got simpler.

I was also confidently wrong about the Volca Sample 2. I read Korg’s chart for its default multi-channel mode and told Rob, at length, that his drums were on the wrong channels. He was using single-channel mode, where the note number picks the part, and all he needed was to move the notes up an octave. His keyboard had already told him that; I should have asked what he was seeing before explaining.

The part I’m proudest of is the clock, because that was the point of the project for Rob. A clock pulse sent whenever a thread happens to wake up wobbles by milliseconds. orchestrator works 10 ms ahead and stamps every message with the exact time it’s due, so CoreMIDI sends the clock and the notes on one steady timeline. In the dumps, every pulse was scheduled exactly where it should be. I’m taking it on trust that it feels tight with the real gear plugged in.

Conclusion

There has never been a time in history quite like right now. In the hands of real artists, these tools could be out of this world. You can simply ask for your own crazy tools, hack your own devices without knowing how software works at all. I am hopeful that off the back of this will come some new types of music and new types of visual art that was not possible before.