We'll All Vibe Code; It'll Be Anarchy

     

The past few years I’ve been busy learning the fundamentals of machine learning and building my own custom models. Small models of course. SpaCy models, some small CNN models, a few hand rolled ones, and some neat little ML processes using pre built speech-to-text and OpenCV bits and bobs. Most of the things I have been building are specific, proprietary and sensitive so I haven’t really tried “vibe coding”. I’ve used LLMs to summarize some topics, do topic searches and what have you, but I hadn’t really tried letting an LLM write code “for me”.

Better At Research & The ARC Prize

     

Getting Better At Research

Last semester I sat a course on how to properly write a research paper. The course was extremely useful as I believe my academic writing has, in the past, been quite lacking (and I refuse to use LLMs to “fix” my papers). There were many, many insights I took from the course, but one of the best ones came towards the last few weeks: “You should keep a daily journal of your research project.”

AI Hallucinations

     

Introduction

I very much enjoy building machine learning / deep learning / AI systems. I went back to school to understand as much as I could about the subject. And while I am nowhere near Yann LeCun, Geoffrey Hinton, or Andrew Ng level, having built a few systems, I do have a pretty decent high level understanding of how these systems work. Well, in so far as can be taught I suppose - some of the behaviours are quite unexpected.

Emergent Behavior

     

Introduction

I’ve been mulling this over for a few days. I think emergent behaviour in deep neural networks isn’t surprising when you think about it. I also think trying to refine or control those behaviours might not be possible. Allow me to explain what I mean.

Like a Record Baby

I think most people at this point know that deep neural networks (like LLMs: chatgpt, gemini, claude, etc) are pattern matching machines. With simpler machine learning models, we are trying to find much simpler patterns, the most basic being a pattern of a straight line:

On AI Taking Engineering Jobs

     

Introduction

AI has the potential to eventually replace software engineers, but the current iteration of LLMs (large language models like GPT4, Claude, etc) will not be what does it - at least not with how it’s being currently used. I think engineers have an amazing opportunity to ride the wave of a new paradigm of things to build, but it will likely take some effort.

New Kids on The Block

When I was a kid, there was only assembly, basic, and C.

Fight the Exponent

     

brought to you by the 80sMixtape — it's totally hot

Back in the previous ice age, artists would take samples of old records and remix them. They’d take a bunch of samples of, often old Motown, songs and use them as loops to create new songs. Some of my favourite examples of such artists are Grandmaster Flash & The Furious Five, The Sugar Hill Gang, De La Soul, Vanilla Ice (he’s not one of my favourites, but he is notable), The Beastie Boys, Young MC and one of my all time favourites Public Enemy.

Midi GPT custom BPE - Theremin Midi

     

🕶️ Quick jump if you are just looking for the code, or just looking for the weights.

I wouldn’t call myself a musician, but I’ve been playing and recording music since I was a teenager. It’s a hobby of mine that I’ve dabbled with for a long time.

One technique I sometimes use when I am looking for musical ideas is to roll some dice to inspire chord progressions or melodies. Since I’ve been doing more machine learning work over the past few years, I decided it would be a fun project to create a machine learning model to aid in generating musical ideas.

The Simplest Spider

     

In these the days of training models and analysing random things off the internet, sometimes you just need to spider some content off a web site.

If you are going to be doing this for real, some good tools I recommend are:

However, sometimes you just need something quick, dirty and scrappy.

Doing Map Selections Using a Quadtree

     

My methodology to building products is:

I find the faster you can get feedback the faster you can figure out if your hypothesis is correct. The hypothesis being, “my solution solves the problem, and it’s worth building”.

It’s often better not to work on a doomed idea for years and years. Even if the MVP idea doesn’t work out, there will always be some lessons you can learn from them. This post is about one of those cool tricks that worked out well, but the idea surrounding it might take too long to be profitable (for us).

50k Ultramarathon Retrospective

     

I love trail running. I love to hike and tramp; to be in the middle of nowhere with almost no one around. Just me and the planet. Running in the woods is just… it’s just perfect.

I’ve done several smaller trail runs (33k, 24k, and my original love the GutBuster in the Blue Mountains Australia), but I hadn’t broken into the Ultramarathon distance officially until last weekend.

The Taupō Ultramarathon - I’ve been trying to do this race for about 3 years now. In 2021 New Zealand went into lock down, and most everything stopped. The race that year was cancelled - which made sense. The next year I was the lucky recipient of Covid in the middle of my restarted training - so I had to DNS/DNF. But finally… finally the spike proteins aligned, and I was able to show up to the start line.