Got an idea for a plug‑in, but no idea how to code? Read on...
If you make music with computers then you surely use plug‑ins. They’re everywhere: sequencers, samplers, instruments, inserts, effects, utilities such as spectrograms and MIDI filters, and more. But have you ever thought about making your own plug‑in? Perhaps you have an idea for a plug‑in that doesn’t yet exist, but you don’t know how to make it yourself?
Until now, your chances of making your own VST, AU or AAX plug‑in were limited unless you could write code, and had the time to invest in building your idea, or you could afford to pay someone else to do it, or you were famous enough to have companies competing to put your name on their plug‑in. Today, there’s another option: a new approach called vibe coding. Vibe coding is a process where you use natural language to describe what software you want to make, and have a large language model write the code for you. You don’t even need to read, or understand, the code.
Large language models, often abbreviated as LLMs, are so‑called ‘artificial intelligences’ that have been trained on a vast corpus of written material across all fields. This allows them to generate different types of material, from essays and forum posts to computer code. If you’ve already used LLMs such as ChatGPT, you’ll find vibe coding is similar, in that it feels like a conversation with someone, except you’re talking about the software you want to make. All you need is the ability to put your ideas into words, and the diligence to see it through, even when things don’t go to plan. These are skills you already use when writing songs. In this article I’ll outline the process you would need to follow to make your own plug‑ins, and I’ll illustrate my points with an example project.
Starting Points
Before you start messing around with AI code generation, you need to know what you want to build. It all starts with one question: what do you want or need? Perhaps you want to work with MIDI data: filter it, generate melodies or invent outlandish new jazz chords. Or you could work directly with audio, and modulate sample playback so that you can have a snare backbeat that constantly evolves over the course of a song. These are possible, as are many other ideas. My only recommendation is that your first attempt is the simplest thing you can imagine. You want to learn the process before attempting to vibe code a complex plug‑in.
You will need to rein in your expectations, too. Some things you try simply won’t work. You can’t expect to create a competitor to plug‑ins that precisely model orchestras, or allow you to have granular control over every parameter of sample playback. Commercial plug‑ins with those features are complex, and need experts to build them. It’s best to think of vibe coding as a means to build something that’s uniquely yours, that may not be available to buy. Don’t underestimate it, though! I’ve frequently been surprised how far I could push my vibe‑coded efforts. The results usually worked better than I expected.
It’s best to think of vibe coding as a means to build something that’s uniquely yours, that may not be available to buy.
It’s a creative process, so I recommend that you approach it with an open mind. Your plug‑in is going to be created during a conversation with a machine, where you give it your ideas and the machine sends new ideas back at you.
In this example, we’ll use Cockos Reaper to host our generated plug‑in, for two reasons: Reaper’s JSFX plug‑in format has its own compiler built in, and Cockos generously offer a month’s free trial to new users. For the code generation we’ll use Anthropic's Claude, as they also offer a generous free tier and it’s consistently given the best code‑generation results in my trials compared to OpenAI’s ChatGPT and Google Gemini. Claude’s free tier does limit the amount of code you can generate in any six‑hour period, so you may have to wait for it to reset, or consider paying for the licence.
Prompt Action
The vibe‑coding process I’ll outline isn’t very risky. As with all software, bugs and crashing are an inherent risk, and there is the possibility that Reaper will fall over if you give it buggy code, but this has never happened to me. Another concern about LLMs is giving them too much personal data, but you’ve got control. The main thing to remember is to never tell the LLM anything that could identify you. Worth noting is that Anthropic allow you to decide if they can use your conversations to train their models in the Privacy section of Claude’s settings menu.
Let’s walk through the process I follow to create my own plug‑ins. While your goals may be different, the process you follow should be the same, and it always begins with considering what the final result should be:
- 1. Define your desired plug‑in.
- 2. Describe it in natural language.
- 3. Use Claude to create code.
- 4. Test the code in Reaper.
- 5. Repeat steps 3 and 4 to optimise the features and fix bugs, if needed.
- 6. Stop when it works (rather than waiting until it’s perfect).
About my own plug‑in idea: I love acid house synths. Occasionally, I use sequencers to create repeatable melodies and rhythmic patterns. I want to vibe code a sequencer tool that’s similar to a Roland TB‑303’s built‑in pattern creator, but with some notable differences. I want it to offer more than 16 steps in a single pattern, and allow the user to select the exact number of steps; I want a visual interface that allows me to see all the notes in the sequence, not just the note assigned to the current step; I want the option to select any octave for each note, rather than just one octave above or below the default; and in place of the 303’s ‘accent’ feature, I’d like to be able to choose from a wider scale of velocities.
Before you begin, you need to have your software ready. First, register for Claude if you don’t use it already. Once you’re registered, it should open a new chat window. Keep that open. Then, start up Reaper, make a new project, add a track to it, and then click on the FX button on the track. The last two steps are to click on the FX menu text in the window that opens, then click on ‘Create new JS FX...’ from the drop‑down menu. A new window will appear. Keep it open, and return to Claude.
Now we’re ready to get vibey. I know what I want to build. Now I need to be able to put that into words. My description will be the ‘prompt’ that I type into Claude’s chat window, giving the LLM the information it needs to build my plug‑in. There’s a bit of an art to creating effective prompts (a skill that is sometimes called ‘prompt engineering’), and it gets easier the more often you work with LLMs. Many Internet forums are dedicated to it, but it’s really just about giving instructions efficiently. Good instructions are specific and clear, include all the vital information, and often break big bits of information down into smaller chunks. Thankfully, LLMs are constantly getting better at understanding what we want, so don’t worry about trying to make a ‘perfect’ prompt.
From The Top
My initial Claude prompt is shown in Screen 1. As you can see, I haven’t given all the details at once. I want to see what Claude creates, and then shape it in rounds of improvements. It’s a bit like writing a song: you try things, move stuff around, add to it and take things away, and over time, something good hopefully starts to emerge! As soon as I hit Enter to send the prompt, Claude starts to generate code straight away. The first time you see this happen, it’s quite startling. Programmers take a long time to think through a coding problem before they begin. AI just dives in!
Incidentally, if you were to give Claude this exact same prompt, you’ll probably get a different answer and a different set of code. That’s because LLMs like Claude generate fresh answers each time they’re asked a question.
The next step is to test the code that Claude has produced. In Claude, there will be a new separate window on the right side showing the generated code. At the top right of that window there is a Copy button. Click on Copy, keep Claude open in your browser, and then head to Reaper.
You should still have the New JS Effect window open in Reaper. You’ll be asked to name your creation, so just give it a short descriptive name for now, such as My 303 Sequencer. Click on OK and you’ll be presented with a new and basic JSFX plug‑in. Clicking Edit will open up the plug‑in’s code window, which will already contain some basic code. You want to select all of that and delete it, so that you start with a blank slate. Now you can paste in your Claude‑generated code and save it.
At this point, one of two things is going to happen. One is that you’ll see a ‘Recompiled OK!’ message at the bottom of the code window, meaning that your plug‑in is ready for testing. Alternatively, you’ll see an error message. Both of these are normal occurrences and are positive steps forward! Sooner or later you’ll hit a roadblock, but don’t worry. They can be easily overcome.
If your plug‑in is ready for testing, go ahead and try it. Just remember to try it ‘in context’, as if you were in a music‑making session. For example, a sequencer won’t make sound by itself, so to test that it’s working properly, you would need to put another plug‑in after it that generates sounds from MIDI data. So play around! Try your first version and see what happens. It might be the start of something incredible! Alternatively, perhaps your code didn’t work; or, although the code compiles, the resulting plug‑in doesn’t behave as you’d hoped. In a moment I’ll show you the process for overcoming these issues.
New Directions
The first version of my sequencer is shown in Screen 4.
The main issue with the initial version was that as soon as I added the plug‑in to a track, it just started running, independent of Reaper’s transport controls. Strangely, it also let me set the tempo independently of Reaper’s tempo, too. This was less a bug than a breakthrough; once again, the code Claude generated gave me an idea I hadn’t had before. I love polyrhythms and strange time signatures. Hearing the sequencer run independently of Reaper’s transport instantly made me think of having more control than the usual subdivisions such as quarter‑note, eighth‑note and 16th‑note, plus triplet options. This made me wonder whether I could have the plug‑in spread any number of steps evenly over a bar of music, such that if I set it to four steps, it would generate one step per crotchet, but if I set it to something odd like 13, it would evenly spread 13 steps across a bar. This is the kind of stuff that thrills me, with unexpected ideas and possibilities emerging in the creative process. I decided to try to get Claude to make this new idea work, as an improvement to the code.
Incidentally, you might find that your plug‑in’s interface is ugly or bland. I recommend that you fine‑tune the functionality before addressing this. This is because every change to the plug‑in’s functionality will have a knock‑on effect on how it looks, so it’s just more efficient to improve how it works before you tweak its appearance.
Power Up
What do you do if your code doesn’t work at all, or doesn’t work how you intended? You get Claude to fix it or improve it! If there was an error when you tried to save the code, copy the exact message that describes it, for pasting into Claude.
Screen 5 shows an error message I received when working on one plug‑in. Annoyingly, Reaper doesn’t allow you to select the text of the error message for copying. I work around this by taking a screenshot of the error message and uploading it to a free website that extracts text from images.
With the error message copied, next write a new message to Claude. Start by saying “I got an error ‑”, and then paste the error message in. Claude will attempt to iron out the bugs in the code for you, and then give you an updated version.
Alternatively, you might want to make improvements to a plug‑in that compiles and runs OK. If that’s the case, you should describe what should be changed in clear and specific language, and let Claude improve the code. The best approach is to prioritise the most important changes first, and do them one at a time. Test each update to the code. You might end up playing ‘whack‑a‑mole’, where a new fix creates a new bug, but that’s just part of the process. When you’ve got updated code, return to your plug‑in’s Edit window, clear all the code that’s there, paste in the new version, and save it. Then test the new version.
This is basically the process you will follow from here onwards until you’re satisfied the plug‑in is done: test the latest version in Reaper, gather feedback, give Claude one piece of feedback after another to improve the code. The great thing about working this way is that you can always go back a version if you add too many ideas and it gets unstable.
With my own plug‑in, I was still extremely excited about having odd timings with rhythmic subdivisions of the bar.
In Screen 6, you’ll see a slightly different prompting approach. I didn’t want to waste time trying to build something that won’t work, so I asked Claude to tell me whether it was possible, without trying to generate any code yet. As you can see, its explanation contains a mistake — the division of seven steps in 4/4 should yield 1/7 of a bar per step — but when I read Claude’s full response, I felt confident it had grasped the idea. It’s a conversation, and sometimes you can ask for an opinion rather than code.
Working fix by fix, I also asked Claude to add the note names up the Y axis, follow Reaper’s main transport for Play and Stop, allow ‘silent’ steps, and add velocity control per step. The last of these didn’t work, even after Claude tried many times, and another issue is that my settings reset every time I hit Play. I’m confident these can be fixed, but Claude hasn’t found the fix yet. This means I have to hit Record and add my notes and settings afterwards. It feels like jamming, but of course I’d prefer to be able to save my work!
The code Claude generated gave me an idea I hadn’t had before.
Good Enough
Screen 7 shows the ‘final’ version of my sequencer plug‑in.
As with anything to do with creativity, ‘final’ is a relative term! It’s often very hard to stop tweaking when you’re vibe coding a plug‑in. If you find you’ve invested too much time, my recommendation is to re‑read your initial description of what you wanted. How close are the results to what you wanted? If they’re way off, perhaps the idea was too ambitious. If they’re close enough to be usable, I recommend you just start using it rather than making any more tweaks. It’s best to finish things, and that frees you up to make your next great plug‑in!
Other Plug‑in Formats
In this article, I’ve used Reaper’s JSFX plug‑in format, because it’s the most straightforward to work with. It’s possible to vibe code other types of plug‑in, too, such as VST and Audio Units, but things will get a bit more complex. That’s because you’ll need to work with an extra development tool alongside Claude to create code, and your DAW to test the plug‑ins.
There are various plug‑in development tools out there. If you’re already comfortable working with code and want to use vibe coding to develop plug‑ins faster, then you probably want to work with JUCE (which has free and paid plans). The great thing about JUCE is that there are published code examples on the web, and many developers discussing it, so you can be sure the information is out there to guide Claude when it’s generating your code! Rather than attempt to summarise the process you’d go through to learn JUCE, let’s just say that you can expect to invest weeks or months to learn it. For most people who want to quickly vibe‑code their own plug‑ins, that perhaps defeats the purpose. On the other hand, if you’re a Pro Tools user, JUCE is your only option for creating AAX plug‑ins, and Avid have strict requirements and approval processes.
Thankfully, for everyone else there’s a tool called Cabbage Audio, which is donationware that is available for free under GNU licence. Don’t let the silly name put you off: Cabbage Audio is capable of building great plug‑ins. You will need to explicitly tell Claude that you’re building for that platform, so the code generated will be Csound rather than JSFX, and you would use their visual designer to create the plug‑in’s interface. You would need to export the plug‑in from Cabbage Audio as a VST or AU for testing in your DAW. You can still expect to spend a few hours or days learning to work with Cabbage Audio (they have tutorials online), but the results should be usable by all DAWs that can accept VST and AU plug‑ins.