Building An Agent-Driven Test Harness For Omarchy
- YT :: https://www.youtube.com/watch?v=Qwekj_ZtmeY
- Original title :: 🚨🚨 Working on Omarchy Automation Framework 🚨🚨
A live coding stream where Prime works on "Ollamachi" - an automation framework that validates Omarchy releases by having AI agents drive a QEMU VM through the desktop UI and record what happened. The evening covers a failed attempt to wire Linear issue automation to Cursor cloud agents (including an accidental runaway loop), building CLI diagnostics over the session/run/result database, and a long tangent on what remains worth learning as a programmer in 2026.
The pitch: agents walking the UI instead of DHH
The goal is release validation for Omarchy. DHH reportedly sits at the computer for six hours walking through the UI by hand before a release; Prime wants an agent to do that walk instead, storing every intent, action, and log in a database so a later agent (or DHH) can review the run and mark it pass or fail.
The pipeline as built: a Linear ticket tagged agent-test triggers a Linear "loop", which spawns a Cursor cloud agent, which calls Prime's local server through a Cloudflare tunnel. That server has the Omarchy ISO cached locally, spawns a QEMU instance, and can issue commands into it - type a username, take a screen readout, drive the installer, lock and unlock the screen. Everything is instrumented into Sentry so he can see where an agent wasted time. A Sentry trace from earlier showed agent intents that were far too long: "start the Omarchy installer from the rendered boot prompt", then a pile of individual steps, reboot, lock with Super-Escape, unlock with the account password - which crashed it.
The Linear-to-Cursor loop never worked
Most of the first hour is spent failing to get Linear to spawn a Cursor agent. Attempts produce "searched for user cursor / executed delegate issue" runs that don't do what was asked, then "couldn't find a cursor user associated with this session". Toggling the Cursor integration in Linear's coding-and-reviews settings never presents a sign-in flow, and the MCP direction that works is the wrong one - he can drive Cursor from his side, not have Linear reach into Cursor. He gives up on the loop rather than reading the docs top to bottom, concluding it is probably a skill issue or a feature that doesn't exist yet.
Along the way one version of the loop started creating issues on its own - 13 runs, unexplained get stats calls against example.com he never made. The genuine fear: going to bed and waking up to nine thousand issues and an unbounded token bill. His takeaway is that custom agent harnesses make accidental destruction cheap, and that observability across this stack is a nightmare - he had no way to trace which spawned agent did the naughty thing.
One thing he does praise: agents embedded inside products. Asking a product "where is the setting for X" and getting a direct link to it is genuinely good.
CLI diagnostics and the database confusion
He extends the client CLI with a session noun: client session <session-id> returning logs, the test definition, test results, actions, and an all that dumps everything as JSON. Prompting for it with XML-tagged instructions, which he finds works better in general.
Then a long debugging detour into Postgres (via PlanetScale's console) chasing null test results and repeated IDs, including a fight with foreign-key constraints - delete blocked, truncate ... cascade denied by user permissions - before realizing the schema was right all along: a test run holds one or more test definitions, results are tied to runs, and a result has no session ID until the run actually starts. "I've been up too long today."
The real bugs found and filed: image URLs don't load (he later realizes he simply forgot to deploy to Cloudflare), and sessions were not being tied to runs because nothing instructed the agent to associate itself with the run. By the end of the stream the tie-in works and a lock-screen test runs through the VM, picking the America/Denver timezone in the installer.
Model notes scattered through: Luna is noticeably fast and "emotionally painful" to leave; a Gemini session earlier that day explored files for 40 minutes and cost $118 because he wasn't watching; a Soul review of his change caught that test start should also flip status from pending to running.
Was the AI actually faster?
His honest assessment: he could probably have built the automation framework himself in three to five days, and it might have been better. Hard to say. Nothing in the project was novel to him - talk to Linear, get the Sentry stuff, spawn the Cursor agent - so there was no "how should one do this?" moment, which is part of why it isn't very fun.
Advice, tangents, and side projects
To a junior worried that AI can write everything and not wanting to give up writing code by hand: then don't. Not everyone's time is worth the same or equally available - with four kids he has built enough servers to know what building a server takes. But if you're new, get a few server reps in. He still spends time on things he wants to learn for their own sake, like two weeks earlier in the year writing C, driving screens, and using DMA.
On being a good engineer in 2026 he simply doesn't know what "good" means anymore, and notes not all jobs are equal - a website's goals are not to destroy the database or spend a fortune on AWS. Chat gets him to endorse Go and HTMX (Ollamachi's own UI is HTMX, slopped together, and he only cares that it looks right on his phone). He increasingly wants applications that exist only in the terminal and on his phone.
Other threads: his own screenplay/animation tool used to produce video segments (a particle system built on a particle system for wind and leaves; block fades over color fades; an upcoming Apollo 11 video), an Odin game engine arc he dropped once it became game design rather than engine programming, Chris Lattner possibly returning for another standup episode, Computer Enhance recommended for learning optimization, and the day's DNS outage prompting jokes about single points of failure and "software engineering is solved". He also complains nobody caught the joke in his ad - "yesterday's Rust rewrite went marvelous, today we start on the back end" - which implies they started on the front end.