I entered hackathon.dev, bolt.new’s hackathon, planning to build something fun: RiffRoll, an app that generates comedy content automatically. The idea was simple. Feed it a topic, get jokes back, then push the whole thing out as short-form video for social.
To make that work end to end, I built out a real production pipeline. RiffRoll auto-generates captions and renders them into video with Pexels stock footage using Shotstack, then posts the finished clips out to social automatically, no manual editing or uploading required. I also wired up metrics tracking so I could see which jokes and styles were actually landing with an audience instead of just guessing. This is one of my AI comedians…
I spent nearly a month’s worth of my free time (I had a lot, I was laid up with shoulder surgery and dictated most of the prompting to Aqua Voice) refining Riffroll. I built 15 comedian personas and styles, including Madison above, and I added voice controls, laugh tracks, and all sorts of bells and whistles to the platform.
But there was one small problem: the jokes themselves weren’t funny. Even the frontier models could write comedy, but they could not write good comedy. The jokes sucked.
And it makes sense. Comedy is based on misdirection and unexpected improbable twists. LLMs are just predicting the next most probable word based on the context it has, which is literally the opposite of a good punchline. Comedy is entropic and unexpected, LLMs are designed to be probabilistic and predictive.
So I went down a side path. I started having a handful of different LLMs write jokes across different comedy styles, then had a separate set of LLMs judge and rank those jokes against each other. Models writing jokes, other models grading the jokes. It was meant to be a quick internal tool to make RiffRoll’s output better.
Instead, it became its own project: ModelMash.
The Side Project Wins the Hackathon
ModelMash ended up taking 7th place overall, worth $17,500, and also won “Most Likely to Get Funded,” worth another $5,000. Jason Calacanis, one of the judges, picked it out specifically for that award, so that was a bit validating.
I had submitted RiffRoll, ModelMash, and a silly hot potato game for Reddit (as well as a joke site, Punzi). ModelMash had been a complete afterthought, submitted to the hackathon in a weekend. When it won, my wife was puzzled, because I had not mentioned that one. I thought I had been selected for RiffRoll!
I had built a silly app to make jokes, and the byproduct I built to fix that app turned out to be the more valuable thing. RiffRoll was the reason ModelMash existed, but ModelMash was the one that got picked up. Shows how good my instincts are.
From ModelMash to PeerLM
After the hackathon, I kept building on the model-judging-model idea as a weekend side project. What started as “which model tells the funniest joke” grew into something with real utility: a way to compare LLMs head to head on cost, quality, and performance, so teams can make model and routing decisions based on actual evidence instead of guesswork.
That project is now PeerLM, and it’s mostly shipped.
Funny how that works. You set out to build a joke machine, and you walk away with a pre-seed-worthy product because you got annoyed the jokes weren’t landing.