The Context As a Protocol

     

“If everyone is thinking alike, then no one is thinking. -Benjamin Franklin” -Rob Rohan

For some quick background, I have 3 modes:

  1. I write code “by hand” for work, for school, and for fun.
  2. I use Claude Code for some open source projects, test projects, and to keep up with what’s going on
  3. I have my own lab where I made my own LLM harness, and run a local LLM on a 5080 16gb with 32k context window (i.e. my own personal Anthropic / OpenAI)

I generally do things like number 3 because I am a slow learner. Unless I can take something apart or build it from the ground up with my hands, I have a hard time understanding it.

The Realist Adjusts The Sails

     

I have always lived in the wilderness.

I am the guy who gets called when the co-founder embezzled a bunch of the company’s money, and they need someone to fix the product, but they don’t have the funds. Or the product has already had six “hot shot” ex-fang programmers with “strong opinions” who all half implemented their ideas and now the product’s code goes in different directions because they thought the code wasn’t fashionable for the time. Or there was a rift with the original developer so he ran off with all the DNS and AWS logins. Or the original website developer died so there was no way to get the original source code. Or the product just doesn’t fit the market, and they have about 2 months of runway left and need to pivot to “something”.

Local Agent Vibe Coded Keylogger

     

I’ve been doing a lot of experiments and spikes using my local lab running a local LLM (on an Nvidia 5060Ti with 16gb) using my own AI harness (written by hand in a language other than python or javascript; thank you very much), and I decided to put it through it’s paces and let it try to code something itself.

I am close to releasing an old school digital audio workstation (called a Tracker) for Mac and the Steam Deck. It’s really fun, but if you are not used to trackers you’d probably find it impossible to use. To hopefully mitigate that learning curve, I decided to make some videos showing how to use it.

Ghost Installs via AI Harness

     

This is a feature and a bug. Somewhat scary, but also has the potential for being cool.

People are doing all kinds of interesting things with LLMs, but the original use case (and the thing I find them to be the best at) is translating from one thing to another; doing what is sometimes called a stylistic transfer. In fact, I believe the transformer architecture was created by Google when they were trying to make a better translate.google.com which should be an indication.

Testing a Spatial Memory Index for Strap

     

One of the pieces I still need to build for strap is a long-term memory store. As I’ve written about before, strap compresses context to work within small token budgets. But context compression only handles what’s in the current conversation. The bigger question is how to surface relevant memories from past sessions.

The obvious approach is brute-force cosine similarity over sentence embeddings. It works for a small number of stored memories, and it’s fast enough. But I got curious whether a spatial index could do better, and whether you could make that index inspectable at the same time.

Working Local Coding Agent

     

My use of AI coding agents might not be typical. I mostly find them useful for “rubber ducking” ideas, as a search engine, to build out latex styles, and for generating small POC code or examples. I don’t generally use them to add whole features to a codebase or “vibe code” whole products - although I have been playing with that a bit more lately.

I’ve been trying to build a local setup that will do what I need using as few resources as possible. To try to accomplish this, I’ve been using an old gaming PC, and writing my own harness to try to squeeze every bit out that I can. I think I’ve finally gotten something that works, and it works better than I expected.

Your Own AI on the Cheap

     

Harnesses are all the rage at the moment in AI. A harness is the environment in which you interact with a large language model (LLM). Some examples you may have heard of are Claude Code, Open AI Codex, OpenCode and Claude Cowork but there are many others. Some are focused around developers and writing code, and some (like Claude Cowork) are focused more on office tasks, making documents, posting online, etc. There are some for designers, CAD creating, 3d modeling, etc, etc.

你的代币怎么样？(How's your tokens?)

     

I’ve been having fun playing with coding agents, and I’ve been playing with some different ways, skills calling tools mostly, to enhance them at inference time. Unfortunately, to do my experiments I need to run them within the context of a coding session which gets very costly very quickly.

To try to get to a state where I can iterate, I decided to byte the bullet and try to get a local lab running. However, like most people I am GPU poor. The only thing I have is an old gaming PC with an Nvidia 1660 Ti with 6GB of RAM. That isn’t enough to run just about anything, but I am no stranger to constraints.

Basis - A Lisp Neural Network

     

Claude Code is absolutely insane. For a builder-of-software-and-crazy-ideas, I am unsure of another way to express how incredibly addictive this tool is.

However, it could very well be that I have fallen down one of the oft described rabbit holes. Or maybe I have succumbed to AI psychosis. At this point I am not sure. Like the man who thought he discovered a new form of mathematics because ChatGPT convinced him, perhaps I am falling into that trap as well. Somebody please let me know if I’ve gone off the rails.

The Unreasonable Effectiveness of Claude Code

     

As I mentioned in my last post, I very much enjoy writing code “by hand”. It is the only thing I have consistently dedicated the last 30+ years of my life to. In fact, I have a hard time understanding what software engineering / building software products would even be if one didn’t have a solid grasp of, let alone enjoy, writing code.

However, the reality is that the point of building software isn’t to build software. The point is to make a product or a business. It’s to accelerate some real world process or enable some kind of commerce. Engineers often get hung up on linters, syntax formatting, some made up abstraction, or big O time complexity of some batch process that runs all night. While those things have a place, the end goal of software is to increase shareholder value and make the company money. It’s not my favourite thing to think about, but it’s the truth.