I put every CSI button in a CLI

Every hacker movie has the same scene. Somebody leans into a monitor, says enhance, and a smear of pixels resolves into a license plate. Off to the side there's usually a person with a headset and too many screens who can find anyone, anywhere, in about forty seconds. Spider-Man calls that job being the guy in the chair. I've always called it the man in the chair, and I have wanted to be him since I was a kid.

The frustrating part is that the fantasy isn't fake. Audio fingerprinting is real, and has been since 2003. Face clustering is real. Deterministic denoise-and-upscale really does recover characters that OCR was getting wrong. What has never existed is one place where an agent can reach all of it in plain language and remember what it found.

So I built one. Overcast is a video OSINT agent: a CLI you talk to that can watch, listen, read, match, map, and remember across a whole corpus of video. I demoed it at DEF CON 34 Demo Labs on Friday and Saturday. This post is that talk, written down.

The title I submitted was Overcast: Video OSINT Agent. Point it at 100 videos, ask anything. The real title, the one that wouldn't have fit the program, is the one in the headline.

This is the overview pass: the whole system, end to end. I'll go deeper on the individual pieces in later posts.

There is no grep for video #

If you work with coding agents you've noticed the good ones converging on the same Unix pattern: read files, grep for patterns, edit in place, use the shell as the orchestrator. That works because text has a lovely property: text is its own searchable representation. A Python file is simultaneously the program being run and the document being searched.

That assumption is baked into how we design agents: context windows, tool use, file-based memory. It holds for code, logs, configs, and documents. It does not hold for video. An MP4 is a binary container of compressed frames and audio packets. grep "white van" pier9-cam.mp4 returns nothing, and worse, no error either. The concept simply isn't in the bytes. (I wrote about the representation side of this at length in How to grep video.)

A transcript isn't the fix, because a video is several data types wearing a trench coat: speech, pixels, on-screen text, people, audio, and time. Which one matters depends entirely on what you're investigating. A transcript is one channel, not the container.

What a transcript costs you, measured #

To put a number on that, I pointed Overcast at the AI Engineer conference series (566 public talks across five events, median length 18.7 minutes) and analyzed the whole thing from captions and metadata alone. Zero video bytes.

That got surprisingly far. You learn the format (an 18-to-20-minute slot), the habits (82% open with a self-intro, 91% link a call to action in the description), the recurring narratives and product mentions. Genuinely useful.

It was also blind in a specific, measurable way. When I went back and actually looked at the screens: 27,222 of 38,217 distinct on-screen terms (70%) were never spoken aloud. Eval matrices appeared in 50 of 53 analyzed decks. Nobody says "here is my architecture diagram, the boxes are labeled retriever, reranker, and cache." They point at it. You cannot grep a representation you never extracted.

The pixels are the OSINT #

Fine, but does the missing channel matter outside a tidy conference corpus? In late July the Department of War CTO account posted about a task force embedding with the Pacific Fleet at Pearl Harbor: twenty-plus custom AI agents built in four days. That's the story in the post copy.

The photographs attached to that post are a different story. Enlarge the monitors in the background and you get a staffer's name in a greeting line, internal project and app names, and legible chat history, none of which is anywhere in the caption.

That crop is the one that gets a reaction in a room. The prompt reads I NEED HELP BUILDING AN AGENT TO CREATE A WAR. Stop reading there and you have a headline.

So don't stop reading. The attachment sitting directly above that prompt is WAR (13-JUL - 17-JUL).docx: a Word file covering a Monday-to-Friday date range. In government and military usage, WAR is a Weekly Activity Report. And the post's own copy brags about collapsing a three-day operational reporting process into one hour. Read together, that is somebody automating their weekly status report, with last week's report attached as the template.

I can't prove that from a photograph. But it's three pieces of evidence pointing one way and none pointing the other, and that is the entire discipline in a single frame. The pixels gave up a lead the caption never contained, and then the pixels killed the exciting misreading of that lead. An OCR hit is an observation. What it means is a separate claim, and it needs corroboration before you say it out loud.

ENHANCE, for real #

Which brings us to the button. Overcast ships an enhance verb. It works, with rules.

That's an aircraft tail number cropped from a ramp photo and knocked down to surveillance quality: 64 pixels wide, heavy sensor noise. Baseline OCR reads E-RUC, which is wrong. A deterministic pass (denoise plus a 2× upscale, no model, no invented pixels) reads EI-RJC, and the full-resolution source confirms it: a CityJet Avro RJ85.

The generative super-resolution pane is the crispest one to look at and it also reads EI-RJC. It's also the one you can't trust, because on repeat runs it slips the final character: EI-RJO, EI-RJQ. It synthesizes plausible pixels, and plausible is not the same as correct.

That's the boundary the whole tool is organized around, and every verb declares which side of it that verb sits on:

• Read and match: OCR, transcription, fingerprints, metadata. Recovering captured signal. This is evidence.
• Similarity: embeddings, face clusters, semantic search. Ranked guesses that need a human. These are leads.
• Reconstruct: super-resolution, rotation, age progression. Pixels no camera ever captured. These are hypotheses, and in Overcast they're quarantined out of case evidence entirely: reconstruct records carry a caveat and can never be cited by ask or brief.

So what is Overcast, mechanically? #

An agent is a model inside a loop: assemble context, choose an action, run a tool, observe the result, write something durable down, go again. The harness is the thing that wraps that loop into something you can actually run. Overcast is my harness. It's built on pi, so it inherits a real agent loop, a TUI, and per-directory sessions instead of reinventing them.

Two things sit on top of that loop:

• Tools are one typed action each: watch clip.mp4, face --match probe.jpg, audio match sample.wav. Every one is also a plain subcommand on the CLI, so you can pipe them through scripts or call them from your own agent without ever starting mine.
• Skills are reusable playbooks: a SKILL.md that tells an agent when to use a capability, in what order, with which checks and caveats, plus any scripts it needs. Overcast ships 35 of them: the lineup, the crime board, stakeouts, BOLOs, situation rooms, wiretaps.

Four concepts hold the system together: providers bring media in, senses turn it into typed observations, the case remembers, and views read the case back out. That third one is the important one. Every capability writes to the same case. The case is the spine, not another output format.

A case is just a directory with a .overcast/ folder in it. Records are append-only JSONL, one file per verb. A record is small and boring on purpose:

{
  "id": "rec_f51279ddb2ff0ec9",
  "verb": "watch",
  "media": { "ref": "7483218403105246506_4465bdb0.mp4", "at": [52.79, 56.79] },
  "meta": { "provider": "tinycloud", "time": "2026-07-30T16:35:30Z" },
  "payload": { "scene_text": "Based On A True Story Tour / 28 June ..." }
}

An id, the verb that made it, the media it points at, the exact time range, and the payload. That contract is what makes everything downstream possible.

The case: three years of Will Smith #

For a demo target I wanted somebody public, with a lot of first-party video, who nobody would mistake for a real investigation. Generative AI already picked him for me: ever since a Reddit user rendered Will Smith eating spaghetti in 2023, every text-to-video model has been informally benchmarked on the same prompt. He was also the target of my DEF CON 33 Recon Village talk, where the task was to find one clip in a pile.

Last year, finding Will ended the hunt. This year it starts one.

Pointed at his official YouTube and TikTok accounts, Overcast froze a corpus of 438 playable videos, 7.42 hours, spanning 2023-07-30 to 2026-07-30: 257 TikToks, 139 YouTube Shorts, 42 long-form uploads. The ingestion funnel is part of the record: 800 posts discovered all-time, 442 inside the window, 441 downloadable, 440 playable, 438 sensed and searchable. The two that dropped out were an age-gated video and a photo-mode post with no video track.

Coverage gaps are evidence too. X was not collected at all, because the profile timeline isn't enumerable and the subject is inactive there, and that gap is written into the case itself, because the alternative is a corpus that quietly pretends to be complete.

Once it's a corpus rather than a clip, the questions change. Who keeps appearing. Which recordings get reused. Which places keep coming up. What changed, and when. That replay is four accumulators answering those four questions at once, and each one is a separate piece of machinery underneath.

"Will Smith" is a query. It is not an identity. #

Before any of that, though: a real YouTube keyword search for Will Smith 2025 returns thirty results, and they are not thirty results about one man. There's the actor. There's the LA Dodgers catcher who was in the 2025 World Series. There's a three-time World Series-winning pitcher. Routing those thirty gave 5 confirmed target, 19 candidates, 4 quarantined, and 2 a different Will Smith entirely.

So the rule is: an official handle is an identity anchor, a keyword hit is only a candidate, and anything unresolved gets quarantined and never enters memory. A contaminated corpus produces confidently wrong memory, which is worse than an empty one, because it looks like knowledge.

"Run it against the database." Okay, which database? #

The other thing television gets wrong is implying there's one database. There isn't, because "same" is not one operation. What you index falls out of what you intend to ask:

A multimodal model is not automatically Shazam plus a face database plus a temporal index. Overcast ships all of these, and most of them run locally: DeepFace for faces, open_clip for images, LAION CLAP for audio semantics, pyannote for voice prints, OpenCV SIFT/ORB with RANSAC for keypoint matching, and a from-scratch implementation of the Wang 2003 constellation algorithm for audio fingerprints.

Same recording: audio fingerprinting #

Fingerprinting is the trick that feels most like magic and is least like AI: there's no model call anywhere in it. Run an STFT over the audio and keep the local energy peaks that survive re-encoding. Pair nearby peaks into (f1, f2, Δt) hashes, which are robust to volume and codec. Store hash → (recording, offset) in an inverted index so lookup is O(1). Then let candidate hits vote for a single consistent time offset. A real match doesn't just score high, it spikes.

Two roughly 30-second posts produced 12,094 and 14,742 hashes. A TikTok and a YouTube upload aligned at 6,869 votes on an offset of −0.05 seconds, a 21.3× margin over the runner-up, while every other recording in the corpus scored 1,309 or less. That isn't similarity. That's the same recording, re-encoded per platform.

Across the corpus, confirmed at votes ≥ 200, ratio ≥ 0.02, and margin ≥ 3.0: 614 confirmed pairs forming 76 reuse families. The largest family is 24 different videos sharing one recording, which turned out to be Will Smith and Russ promoting a song called "Work of Art," a conclusion the platform's own track metadata reached independently.

The honest caveat travels with the result: the method isn't robust to pitch or speed changes. A sped-up repost scores near the noise floor and gets rejected. A miss is not evidence of no reuse.

Same person: face search and clustering #

Detection, embedding, clustering, and identity are four separate operations, and conflating them is exactly how you get a confidently wrong lineup. Sample a bounded set of frames per video. Detect faces, then throw away the tiny, blurred, extreme-profile ones before they poison a cluster. Embed the survivors into a space where distance approximates identity difference. Nearest-neighbor search, then cluster. Then verify with a probe image the cluster was never built from.

Every step has a threshold, and the threshold is the result. A smoke test on a single 32-second post found 13 faces across 12 sampled frames and produced 8 "people" at the default setting: one person shattered across lighting and pose. Go too loose and you glue different people together instead.

At corpus scale: 4,684 quality-gated faces (minimum 80 pixels, sharpness ≥ 60, brightness ≥ 40, contrast ≥ 28), two-stage clustering at a loose 0.55, then remove the subject, then a tight 0.72. The subject shows up in 167 posts and 694 faces. Two recurring people got named, Big Sean and Martin Lawrence, and only because two independent channels agreed: the account's own caption and a Google Lens reverse-image search. A third distinct, consistent person stayed "Person 149," because reverse search returned nothing, and single-source identity isn't identity. A confident false label is worse than no label.

The failure mode is on that figure on purpose. Four clusters that looked like recurring collaborators were the subject himself: in profile, under colored stage lighting, or in archival footage. And while 77 clusters had three or more posts, that's 3 people, not 77: one 335-face cluster was a single video's frames. Rank by unique posts, not raw count.

Same moment: searching time #

"When did X happen?" and "when does X appear?" are different algorithms, and picking the wrong one gets you a confident wrong answer.

For a one-way state change (a light goes on, a poster comes down) you can bisect: order the candidates, probe the midpoint, halve the interval, and localize in log2(n) probes instead of n. Roughly a dozen visual questions to find a one-way change inside an hour of footage. But that's only valid if the state is monotonic. A recurring collaborator or a repeated graphic is not monotonic, and needs a presence-window scan instead.

For "where in this clip is the thing," there's a cheaper trick: tile sampled frames into one labeled contact sheet and ask a vision model a single question about the whole clip.

$ overcast grid <post> --count 12 --cols 6
  → 12 cells · 2.49s → 57.28s

$ overcast see grid_tour.png \
    --prompt "which numbered cell best shows a list of tour dates and cities?"
  → "Cell 12, the only panel containing a captioned graphic
     listing dates paired with cities"

The returned cell number resolves back to an exact timestamp, 57.278 seconds, through the record's payload.cells map, and from there you can zoom in frame-precisely. One VLM call triages a whole clip; no per-frame sweep. Across the corpus that gave 9,141 shot boundaries over 438 posts, on-screen text in 401 of them, and 6,718 distinct on-screen-text lines to track motifs against.

Saving the file is not remembering it #

Here's the part I care about most, and the part hacker movies never show. Finding a moment once is a search problem. Being able to use it three weeks later, in a different session, without re-watching anything, is a memory problem.

That chain is one real post: a 59-second vertical TikTok. watch --segment shots lifts on-screen text with exact spans. A single OCR span at 52.79–54.79 seconds contains a tour-date graphic listing 25 dated European stops and a ticket link. Nobody asked a model "where is he touring": the text was read off the screen and time-stamped.

That becomes a typed record carrying an id, the canonical URL, the published timestamp, the exact evidence range, the verb that created it, an evidence class (announced/scheduled plus text observed in frame), and a confidence note, including that one of the dates was independently corroborated by a second post. The analyst's judgment lives in a separate finding record that cites the observation, so raw evidence and interpretation never get conflated.

Three retrieval layers sit over the same 438 records: local grep for keywords, a local semantic index (409 documents, 7,020 vectors), and a remote collection for cross-device search. Months later:

$ overcast ask "what tour dates appear on screen in this case?" --verb watch

It answers by citing record ids and their exact evidence spans. The media file is never reopened. That's the difference between a saved file and a memory.

Named is not visited #

Places accumulate the same way. 67 places with at least two independent pieces of text or speech evidence, over 940 days: Philadelphia (12 on-screen text, 15 spoken), Miami (15 text, 7 spoken), France, London, Spain, Gran Canaria. Canada, Monaco, Nyon, Marrakech, Scotland and Sicily were excluded as ambiguous: single weak hits, brand tokens, short tokens, or a region rather than a point. A search for "orange" produced 114 false hits from the color before the French city survived.

There is deliberately no "observed" or "verified" layer on this map, and the markers are never connected into a route. Named, spoken, announced, observed and visited are five different claims. TikTok and YouTube don't carry GPS. This is a publication clock, not a travel log. The map is a source-selection tool: it tells you where to go looking next, not where anybody was.

Camera walls #

Which leads to the most hacker-movie view in the whole thing. Philadelphia is the corpus's most-referenced place (his TikTok bio literally says "Same kid from West Philly") and it's the only city where speech outweighs on-screen text. Overcast can pull 192 public cameras within 18 kilometers of the city center onto one live wall, from keyless public sources: Windy Webcams plus OpenStreetMap surveillance nodes, rendered offline from a 7 MB Protomaps extract.

And the badge on that wall reads STATED, NOT OBSERVED. 0 observed sightings. Same for Miami (189 cameras), London (782), and Las Vegas (185, where the honest label is "no subject evidence; this is simply where we are"). London is the only city with a real announced date, read off a tour poster and confirmed in the pixels.

This is scale without resolution. Public traffic cameras are not identity-capable and were never going to be. The wall is useful for canvassing which sources could have overlooked a scene. Mapped does not mean present, and present does not mean useful field of view.

It's also where I'll admit that an earlier run of this case produced a North American itinerary that turned out to be fabricated. It got caught and quarantined, and the camera-wall counts now regenerate from the quarantined data, which is why every wall shows zero observed. Keeping that failure on the record was deliberate.

Same case, many cockpits #

The case is built. Now pick how you want to drive it. Because everything writes to the same records, the surface is a preference rather than an architecture:

• The CLI: the interactive agent, or overcast -p "brief me on the case" headless for scripting, or any single verb standalone with no agent at all.
• VS Code: right-click a video in the explorer, pick a sense, run it locally, keep the record. Media you encounter mid-development becomes case evidence without changing work surfaces.
• Your own agent: Overcast installs as a Claude Code plugin or as plain agent skills, so Claude, Codex or Cursor can drive the same case.
• The situation room: a live page over the case that watches your configured feeds and updates its panels as new observations land.
• Man in chair: a server and a QR code that hand control of the running agent to your phone, so you can keep watching the case from somewhere else.

The part where the hologram is a hypothesis #

The rest of the toolkit is the fun stuff, and it's organized by trope on purpose: a crime lab (zoom and enhance, photo lineup, voiceprint, camera ballistics, sun-and-shadow time checks), surveillance (BOLO board, stakeout, scanner chatter, camera canvass, geofence), a war room (red-string graph, monitor wall, situation room, man in chair, captain's brief), and a records pull (reverse image search, follow the money, provenance, event timeline).

The last tier crosses a line I want to be loud about. overcast reconstruct car.jpg --ops model turns a single flat photograph into an orbitable 3D mesh, a view no camera ever captured. It's genuinely useful for thinking. It is not evidence.

Notice what the figure refuses to let you forget. There is exactly one real view of that car in the case, and it sits in a box labeled OBSERVED SOURCE. The other 359 degrees are a model's guess at the sides of a car it never saw. The mesh is a good hypothesis about where to point a real sensor next, and it is worth exactly nothing as proof of what the far side of that car looks like.

And then there's the trope that fails hardest, which is exactly why I keep showing it:

Age progression. Nine runs, four licensed source photographs, six different deltas, repeated seeds. The +15-year run produced a different man (skin lightened, hair straightened, nose and jaw morphology altered) and both random seeds agreed on that wrong face. Bigger deltas drift further. The most plausible of the nine still lightens skin tone. Independent seeds converging on the same wrong output means this is systematic drift, not sampling noise, and re-rolling doesn't rescue it.

So generated likenesses in Overcast are never scored by face, cluster or similar. They're quarantined from ask and brief, they can't trigger findings, and every record carries a caveat. This is not a prediction of anyone's future appearance and it is not a claim of biometric continuity. Publishing the negative result next to the working ones is what keeps the spectacle honest.

Authorized use only #

One thing I won't soften. Overcast is dual-use. Use it only for lawful purposes, against targets you're authorized to investigate. Public does not mean harmless. Biometric and identity sources ship opt-in and off by default, each with its own legal constraints. A match score is not certainty. Treat every result as a lead, not proof.

Two more things, both inherited honestly. Scraped captions and media are untrusted input and a live prompt-injection vector. And Overcast inherits pi's trust model: no sandbox, no permission system. The agent runs as you, over material you pulled off the internet. That's a deliberate single-operator design rather than an oversight, but it means you should run investigations in an environment you're willing to expose to what you collect. The repo ships a responsible-use doc with the per-source constraints spelled out.

Try it #

It's on npm, Apache-2.0, and it is unapologetically a hobby project.

$ npm i -g @kdrrr/overcast
$ overcast doctor            # preflights pi, ffmpeg, providers, credentials

$ overcast                            # interactive agent
$ overcast -p "brief me on the case"  # one headless task
$ overcast watch ./clip.mp4           # one sense, no agent at all

You'll want Node 22+, ffmpeg and ffprobe on your PATH, yt-dlp for pulling public video, and a Python/uv toolchain if you want the local model databases. Perception defaults to tinycloud with a Cloudglue key.

The brain is bring-your-own. Overcast rides on pi, which speaks 38 LLM providers (Claude, OpenAI, Gemini, xAI, Bedrock, OpenRouter) or any OpenAI-compatible local endpoint like Ollama or vLLM. Perception is just as reboundable: every sense runs over one provider contract, so you can swap in ElevenLabs, fal or Hugging Face on keys, or drop to the local stack. overcast provider install ./yours, no fork required.

The surface is about 40 verbs, grouped four ways in the docs. Senses turn media into records (watch, listen, see, enhance, face, voice, exif, verify). Inspect looks at the evidence (wall, map, graph, grid, situation). OSINT reaches out (scan, capture, monitor, index). Read synthesizes (ask, brief, case). Every one is a standalone command that emits a portable JSON record, so you can script against all of it without ever starting the agent.

Prefer your own agent? The whole skill lineup installs into one:

# Claude Code plugin
/plugin marketplace add kdr/overcast
/plugin install overcast@overcast

# or plain agent skills for Claude, Codex, Cursor, friends
$ overcast skills install
$ npx skills add kdr/overcast

That lineup is organized by trope on purpose, because it is genuinely how I think about the work. Here is the whole bill, straight out of the README:

Plus overcast itself as the broad driver, overcast-init for one-time setup, and overcast-skill-creator if you want to author your own.

And the VS Code extension is on the marketplace. Search for Overcast, or ext install kdrrr.overcast. It drives the same CLI underneath.

Everyone in the chair #

That's the whole thing: senses, search, memory and reach, wired to a case that remembers. The code is on GitHub, the runnable case files are at overcast.video, the visual version of this walkthrough is at defcon34.overcast.video, and I'd genuinely love for you to take it apart.

I built it because I've spent twenty years on the algorithms that make the fantasy possible, and it bothered me that they were scattered across papers, services and specialist tools nobody outside the field ever gets to touch. This was never meant to be a surveillance product. The point is that with an agent harness, decent representations, and a case that remembers, the man in the chair can be anybody, and the honest version of that dream includes knowing exactly which of your buttons produce evidence, which produce leads, and which produce beautiful, confident nonsense.

If you want the hardened production version of the perception layer, that's Cloudglue and Tinycloud, which my team builds. And if you're building something in this space, come find me: @kdrwins.