Harness · Day Seven · 2026‑06‑23

An agent that watches
you play —
and tells you how to win.

Scholarship finder on PAI · then we build the real‑time game coach
Harness · A two‑week AI bootcamp
01 / 18
The morning read

First, the Rundown.

Read it, mine the new terms — and watch for anything on vision models or real-time AI. That's the machine we start building this afternoon.

Open today's Rundown  rundown.ai/articles ↗
Classroom
02 / 18
RECAP · WE CHANGED COURSE

Yesterday we went off-script —
and hit a wall on purpose.

We set the sales project aside, looked at AI inside a real company, and met a different kind of harness. The best thing we did all afternoon was pay for a lesson the docs can't teach you.

Classroom · recap
03 / 18
Two things we worked through

AI for yourself ≠ AI inside a company.

  • Office logistics — how the corporate world actually shares work: Office 365, SharePoint, the formats clients expect.
  • PAI vs. the corporate setting — what your personal harness does freely is a different problem inside a running business.
  • Then we met a different kind of harness — and learned what it costs.
Classroom · recap
04 / 18
Hermes · the goal-oriented harness · the hard-fought win

We learned the real cost by paying it.

No Claude sub
Hermes can't use your Claude subscription — it wants its own model + keys
Per-query bleed
Even small queries cost real money — the meter never stops
Tool lock-in
Third-party tools limited to a few options → more API keys, more investment
The lesson
You can't read this in a README. You have to feel the bill.
Classroom · recap
05 / 18
Match the harness to the job — and the budget

Two harnesses, two jobs.

Augments you

PAI

  • Runs on your Claude subscription — no per-query bleed
  • Your skills, your tools, in the loop
Best when you're driving.
Chases a goal alone

Hermes

  • Autonomous — set a goal, it grinds unattended
  • Its own model + per-query cost; tools limited
Best when no human is watching.

Use-case drives the choice — so for the scholarship finder, we choose deliberately.

Classroom · recap
06 / 18
FIRST BUILD · THE SCHOLARSHIP FINDER

Build it on PAI.
Run it on a schedule.

You started this on Hermes. After feeling that bill — put it where it belongs: in PAI, on a routine, on your Claude subscription, working for your own future while you sleep.

Build · scholarship finder
07 / 18
No bleed · your tools · on a timer

A finder that hunts while you sleep.

A search that runs every day forever is exactly the wrong place to pay per query. PAI runs on your Claude, uses your tools, and its scheduler fires it on a cron — the same routine backbone behind the free-games bot and Dexter.

  • Each run: find scholarships that fit you → dedup → surface only the new ones. And this one pays you back.
The shape of it
PAI agent — markdown + tools
+ a scheduled job — runs daily
+ your Claude — no extra meter
Set it once. It works while you don't.
Build · scholarship finder
08 / 18
THE MAIN EVENT · THE STRETCH GOAL, NOW REAL

A coach that watches you play
and talks you through it.

We've been pointing at this since Day 2. An agent that reads your screen every few seconds, understands the game, and speaks advice in your ear — "you've died at the same corner three rounds running; hold back and let them peek first." It's buildable today, for about a quarter per half-hour session.

The game coach
09 / 18
The architecture · one agentic loop, on a cadence

Observe → analyze → decide → speak.

Observegrab a frame / game state
→
Analyzelocal model reads it fast
→
DecideClaude reasons on summaries
→
SpeakTTS into your headset

That loop runs on a timer — fast eyes every couple of seconds, the strategic brain every 30. Same OBSERVE → … → LEARN spine as PAI's inner loop, pointed at a game.

The game coach
10 / 18
All the required components · free or near-free, on Windows

The full stack, one piece at a time.

Capture
DXcam — screen frames, ~5ms, anti-cheat safe · github.com/ra1nty/DXcam
Game state
CS2 GSI (official JSON) · Overwolf (Valorant) · OCR fallback · or your own telemetry
Fast eyes
Local VLM via Ollama: Moondream 2B / Qwen3-VL 8B — reads frames cheaply
Strategic brain
Claude — Haiku every 30s on text summaries, Sonnet/Opus at round breaks
Voice
Kokoro TTS (local, free, <300ms) or ElevenLabs (premium) → headset via VB-Cable
Orchestration
A Python loop — asyncio frame queue + state dict (or LangGraph)
Post-game
OBS + FFmpeg clips & narration · SQLite stats · pynput APM
The game coach
11 / 18
The one decision that makes it work

Claude never sees a frame. It reads notes.

A cheap local model watches every frame. Claude only gets a text summary every 30 seconds — so it reasons deeply without the cost or the lag.

  • Cost: ~$0.25 per 30-min session, hybrid — vs. $1.47–$4.41 pure cloud. (~$18/mo vs. $100–300.)
  • Latency: local eyes answer in under 2s; Claude's slowness is fine on a 30s beat. (Day 4: match the timescale to the agent.)
  • The wall, again: this needs an Anthropic Console API key — Claude Max alone won't do it.
The game coach
12 / 18
Two ways in · we start with the second

Coach a game you watch — or one you own.

Path A · watch any game

Capture + vision

  • DXcam screen capture → local VLM → Claude → voice
  • Works on CS2 / Valorant
  • Harder: 8GB VRAM budget, anti-cheat care, vision is fiddly
Powerful, but a fight.
Path B · a game you own ← start here

Build your own GSI

  • Fork an open-source game (Godot FPS template)
  • Add a WebSocket telemetry node → broadcasts state as JSON
  • Coach reads structured state — no vision needed
Ties straight to your GitHub + Risk work.

Path B is the same idea as CS2's GSI — you just build it yourself, which is exactly the lesson. Fork: github.com/Whimfoome/godot-FirstPersonStarter

The game coach
13 / 18
The structural plan · build it in vertical slices

Each slice works end-to-end before the next.

1Input working — a frame in (DXcam) or JSON in (your telemetry node)observe
2One coaching line — state → Claude → a single piece of advice (text)decide
3Give it a voice — pipe that line to Kokoro / ElevenLabs → your headsetspeak
4The live loop — observe → analyze → decide → speak, on a cadence, hands-freeloop
5Post-game (stretch) — OBS clips of key moments, narrated review, APM statsreview

Reference build: github.com/tejashah88/gaming-ai-coach — screenshots → LLM → TTS, the whole pipeline, already working.

The game coach
14 / 18
How far we get, and where it runs

First slice today. Full loop Thursday–Friday.

This afternoon

Slice 1 → 3

Get input flowing, get one spoken coaching line out. Prove the pipe end to end.

Day 9–10

The live loop

Run it hands-free during real play, then the post-game review.

Where it runs

Your Windows rig

This lives on your RTX 3070+ gaming PC, not the MacBook — which is why Friday we talk Windows.

The game coach
15 / 18
WHY THIS ONE

Every piece you've built
shows up here.

GitHub forks, a game engine you understand, agents with tools, the latency lesson, the cost lesson, scheduling, voice — the coach is where all of it converges into one thing you'd actually use. And it's yours: a self-built AI coach in your own repo is a college essay that writes itself.

The game coach
16 / 18
How today runs

Finder first. Then start the coach.

First build

Scholarship finder

Move it onto PAI, give it tools, put it on a daily schedule.

Then

Stand up the coach

Pick your path, fork the repo, get input → Claude → one spoken line.

Thu–Fri

The full loop

Live coaching during play, then post-game review.

The game coach
17 / 18
GO BUILD

Finder on a schedule.
Then a coach in your ear.

Ship the scholarship finder, then start the build that's been the target all along. Pick your path, fork the repo, and get the first slice working end to end. Log your time in Harvest — this is the fun one.

Harness · Day Seven
18 / 18