The Unreasonable Effectiveness of Claude Code

     

As I mentioned in my last post, I very much enjoy writing code “by hand”. It is the only thing I have consistently dedicated the last 30+ years of my life to. In fact, I have a hard time understanding what software engineering / building software products would even be if one didn’t have a solid grasp of, let alone enjoy, writing code.

However, the reality is that the point of building software isn’t to build software. The point is to make a product or a business. It’s to accelerate some real world process or enable some kind of commerce. Engineers often get hung up on linters, syntax formatting, some made up abstraction, or big O time complexity of some batch process that runs all night. While those things have a place, the end goal of software is to increase shareholder value and make the company money. It’s not my favourite thing to think about, but it’s the truth.

We'll All Vibe Code; It'll Be Anarchy

     

The past few years I’ve been busy learning the fundamentals of machine learning and building my own custom models. Small models of course. SpaCy models, some small CNN models, a few hand rolled ones, and some neat little ML processes using pre built speech-to-text and OpenCV bits and bobs. Most of the things I have been building are specific, proprietary and sensitive so I haven’t really tried “vibe coding”. I’ve used LLMs to summarize some topics, do topic searches and what have you, but I hadn’t really tried letting an LLM write code “for me”.

Better At Research & The ARC Prize

     

Getting Better At Research

Last semester I sat a course on how to properly write a research paper. The course was extremely useful as I believe my academic writing has, in the past, been quite lacking (and I refuse to use LLMs to “fix” my papers). There were many, many insights I took from the course, but one of the best ones came towards the last few weeks: “You should keep a daily journal of your research project.”

AI Hallucinations

     

Introduction

I very much enjoy building machine learning / deep learning / AI systems. I went back to school to understand as much as I could about the subject. And while I am nowhere near Yann LeCun, Geoffrey Hinton, or Andrew Ng level, having built a few systems, I do have a pretty decent high level understanding of how these systems work. Well, in so far as can be taught I suppose - some of the behaviours are quite unexpected.

Emergent Behavior

     

Introduction

I’ve been mulling this over for a few days. I think emergent behaviour in deep neural networks isn’t surprising when you think about it. I also think trying to refine or control those behaviours might not be possible. Allow me to explain what I mean.

Like a Record Baby

I think most people at this point know that deep neural networks (like LLMs: chatgpt, gemini, claude, etc) are pattern matching machines. With simpler machine learning models, we are trying to find much simpler patterns, the most basic being a pattern of a straight line:

On AI Taking Engineering Jobs

     

Introduction

AI has the potential to eventually replace software engineers, but the current iteration of LLMs (large language models like GPT4, Claude, etc) will not be what does it - at least not with how it’s being currently used. I think engineers have an amazing opportunity to ride the wave of a new paradigm of things to build, but it will likely take some effort.

New Kids on The Block

When I was a kid, there was only assembly, basic, and C.

Fight the Exponent

     

brought to you by the 80sMixtape — it's totally hot

Back in the previous ice age, artists would take samples of old records and remix them. They’d take a bunch of samples of, often old Motown, songs and use them as loops to create new songs. Some of my favourite examples of such artists are Grandmaster Flash & The Furious Five, The Sugar Hill Gang, De La Soul, Vanilla Ice (he’s not one of my favourites, but he is notable), The Beastie Boys, Young MC and one of my all time favourites Public Enemy.

Midi GPT custom BPE - Theremin Midi

     

🕶️ Quick jump if you are just looking for the code, or just looking for the weights.

I wouldn’t call myself a musician, but I’ve been playing and recording music since I was a teenager. It’s a hobby of mine that I’ve dabbled with for a long time.

One technique I sometimes use when I am looking for musical ideas is to roll some dice to inspire chord progressions or melodies. Since I’ve been doing more machine learning work over the past few years, I decided it would be a fun project to create a machine learning model to aid in generating musical ideas.

The Simplest Spider

     

In these the days of training models and analysing random things off the internet, sometimes you just need to spider some content off a web site.

If you are going to be doing this for real, some good tools I recommend are:

However, sometimes you just need something quick, dirty and scrappy.

Doing Map Selections Using a Quadtree

     

My methodology to building products is:

I find the faster you can get feedback the faster you can figure out if your hypothesis is correct. The hypothesis being, “my solution solves the problem, and it’s worth building”.

It’s often better not to work on a doomed idea for years and years. Even if the MVP idea doesn’t work out, there will always be some lessons you can learn from them. This post is about one of those cool tricks that worked out well, but the idea surrounding it might take too long to be profitable (for us).