How to make Claude watch a video, frame by frame
Claude cannot see a video. Hand it a link and it reads the words, then guesses at the rest. Two small skills fix that: one pulls the file down, the other cuts it into stills Claude can actually look at.
Paste a reel link into Claude and the reply will sound confident. Ask what the person was pointing at in the last three seconds and the confidence disappears, because those three seconds never reached it. The screen recording, the caption burnt into the corner, the product sitting on the table: none of it arrives.
What does arrive is text. That is the whole problem, and also the whole solution, because Claude reads images perfectly well when you hand it images. So you stop asking it to watch, and start giving it something to look at.
01 / Setup
The three things you install once
01
Claude Code
The version of Claude that runs in your terminal, in your editor, or in the desktop app. It can run commands and open files on your computer, which a browser tab cannot.
why you need itSkills only work where Claude can actually run things, and everything here is a command that has to touch your hard disk.
02
yt-dlp
A free command line downloader for video sites. On a Mac, brew install yt-dlp. On Windows, winget install yt-dlp.yt-dlp. Anywhere Python is installed, python3 -m pip install -U yt-dlp.
why you need itIt handles the messy part: finding the actual video file behind a page and saving it with a name you can point at later.
03
ffmpeg
The tool that does the cutting. Use brew install ffmpeg, winget install Gyan.FFmpeg, or sudo apt install ffmpeg. Check it landed with ffmpeg -version.
why you need itEvery frame Claude eventually looks at comes out of ffmpeg, and yt-dlp quietly uses it too, to join separate video and audio streams back together.
In plain English
A skill is a folder with a text file in it. The file tells Claude how to do one job and when that job applies. There is nothing to compile and nothing to install. Claude Code watches the folder while it runs, so a skill you save now works in the session you already have open.
02 / First skill
The skill that fetches the video
04
Make the folder
Run mkdir -p ~/.claude/skills/fetch-video. Anything inside ~/.claude/skills/ is available in every project on your machine. The folder name becomes the command, so this one answers to /fetch-video.
why this pathA skill in the wrong folder is simply invisible. If you want it to travel with one project instead, use .claude/skills/ inside that project.
05
Save this as SKILL.md inside it
The block between the two --- lines is the label Claude reads to decide whether this skill is worth opening. Everything under it is the instruction it follows once it does.
why it reads this wayThe description is doing the matching, so it is written the way you would actually ask. Vague descriptions are the reason a skill sits there unused.
--- name: fetch-video description: Downloads a video from a URL to a local folder with yt-dlp so it can be looked at frame by frame. Use when someone shares a video link and wants to know what is on screen, not only what is said. allowed-tools: Bash(mkdir *) Bash(yt-dlp *) Bash(ffprobe *) Bash(ls *) --- # Fetch video Download the video at $ARGUMENTS so a later skill can turn it into frames. ## Steps 1. Make a working folder in the current directory: mkdir -p video-work 2. Download at 720p or lower and merge it into one MP4: yt-dlp -f "bv*[height<=720]+ba/b[height<=720]" \ --merge-output-format mp4 \ -o "video-work/source.%(ext)s" \ "$ARGUMENTS" 3. Read the duration, so the next skill can pick a frame rate: ffprobe -v error -show_entries format=duration \ -of default=noprint_wrappers=1:nokey=1 video-work/source.mp4 4. Report the file path and the duration in seconds. Stop there. ## Rules - 720p is a ceiling, not a target. Bigger frames get scaled down before they are looked at anyway. - If the download fails because the video is private, age gated or login gated, say so and stop. Do not try to work around it. - If the site needs a session, tell the user to rerun with --cookies-from-browser firefox, or whichever browser they use. Never ask for a password. - Download only what the user owns or has permission to use.
Before you download anything
yt-dlp will happily pull down almost any video you point it at, which is not the same thing as being allowed to. Stick to your own footage, your client’s footage, and clips you have written permission to use. Plenty of platforms forbid downloading in their terms, and a skill that runs quietly in the background is exactly the sort of thing that turns a one off into a habit.
03 / Second skill
The skill that turns video into frames
06
Make the second folder
Run mkdir -p ~/.claude/skills/video-frames, which gives you /video-frames. Two folders, two skills, because half the time the video is already sitting on your disk and there is nothing to fetch.
why split themKept separate, this one works on a screen recording, an export from your editor, or a file a client sent over, not only on things with a URL.
07
Save this as SKILL.md inside it
This one checks the length first, picks a rate to match, extracts, and only then starts reading. For long recordings it tiles sixteen moments into a single image before looking at anything.
why the rate rules matterChoosing how often to take a still is the entire job. Too sparse and you miss the cut, too dense and you drown the answer in near identical pictures.
--- name: video-frames description: Cuts a local video file into still images with ffmpeg and reads them in order, so what happens on screen can be described shot by shot. Use after fetch-video, or when a video file is already on disk. allowed-tools: Bash(mkdir *) Bash(ffmpeg *) Bash(ffprobe *) Bash(ls *) Read --- # Video frames Turn the video file at $ARGUMENTS into stills, then read them in order. ## 1. Pick a rate from the duration ffprobe -v error -show_entries format=duration \ -of default=noprint_wrappers=1:nokey=1 "$ARGUMENTS" - Up to 90 seconds: fps=1, one frame a second. - 90 seconds to 10 minutes: fps=1/5, one frame every five seconds. - Over 10 minutes: skip to step 3 and tile instead. ## 2. Extract mkdir -p frames ffmpeg -i "$ARGUMENTS" -vf "fps=1,scale=1280:-2" -q:v 3 frames/f_%04d.jpg Swap fps=1 for the rate chosen above. ## 3. For anything over ten minutes, tile first ffmpeg -i "$ARGUMENTS" -vf "fps=1/15,scale=480:-2,tile=4x4" \ -q:v 3 frames/sheet_%03d.jpg Each sheet holds sixteen moments in one image. Read the sheets, find the stretch that matters, then rerun step 2 on that stretch alone. ## 4. Read them ls frames/ Report the count first, and wait for a go ahead if it is above 20. Then read the frames in filename order, no more than 20 at a time. ## 5. Report - Describe what changes between frames rather than each frame on its own. - Turn filenames into timestamps. At fps=1 the number is roughly the second it came from, so f_0042.jpg is around the 42 second mark. - Say when on-screen text is too small to read instead of guessing at it. - Frames carry no sound. Never describe anything as said or heard.
What a frame costs to look at
Claude reads an image as a grid of 28 by 28 pixel patches, so a 1280 by 720 still works out at 46 across and 26 down, a little under 1,200 visual tokens. Sixty of those is roughly 72,000 tokens spent before you have asked your question. That arithmetic is the reason the rate matters, and the reason one tiled sheet beats sixteen separate stills by about seven to one.
04 / In use
What it looks like when you make Claude watch a video
08
Ask for it in plain English
Open Claude Code in a folder you do not mind filling with files, paste the link, and say what you want to know. Something like: fetch this video and tell me what is on screen in the first ten seconds. You can also force it with /fetch-video followed by the URL.
why it might not fireIf Claude ignores the skill, the description is too vague, not the setup. Rewrite it using the words you would say out loud.
09
Check the count before it starts reading
Ask for the frame count once extraction finishes. A ninety second reel at one frame a second gives you ninety images, which is more than Claude should take in one pass and almost certainly more than you need.
why to look firstFeeding it everything fills the working memory and gets you vaguer answers, not sharper ones. Twenty well chosen frames beat ninety indifferent ones.
10
Give it the words as well as the pictures
Frames carry no sound. Pull the audio out with ffmpeg -i video-work/source.mp4 -vn -q:a 2 audio.mp3, run it through whatever transcription you already use, and paste the text into the same conversation.
why botherThe pictures show what was demonstrated and the words say what was claimed. Most of what you are actually looking for sits in the gap between the two.
Watch out
Common mistakes when making Claude watch a video in 2026
The setup takes ten minutes. Using it badly is where the time goes, and four traps account for most of it.
- Leaving the frame rate to ffmpeg. Without a
fps=setting it keeps every frame, so a thirty second clip lands as nine hundred files. - Pointing it at a folder that does not exist. ffmpeg will not create the output directory for you. Run
mkdir -p framesfirst, every single time. - Leaving the working folders inside a repository. Video and stills are heavy. Add
video-workandframesto.gitignorebefore the first run, not after. - Asking it to watch, then expecting an answer. A vague prompt gets you a description of each frame. A specific question, such as which seconds show the price on screen, gets you something you can use.
Questions
FAQ
Does this cost anything extra to run?
No, in the sense that yt-dlp and ffmpeg are free and do their work on your own computer, so there is no processing bill for the download or the cutting. Yes, in the sense that every frame Claude looks at counts against the usage of the plan you are already on. The cost sits in how many frames you hand over, not in the tools.
Can I do this in the Claude app instead of Claude Code?
Partly. The app cannot run commands on your machine, so it cannot fetch or cut anything for you. You can run the two commands yourself and upload the stills by hand, keeping in mind that claude.ai accepts twenty images per message.
How long a video can this handle?
Length is not really the limit, frame count is. A thirty minute recording sampled every fifteen seconds comes to a hundred and twenty stills, and tiled sixteen to a sheet that is eight images. Well within reach.
Will Claude tell me who is in the video?
No. Claude will not identify people from images and will tell you so. It will describe what someone is doing, wearing or holding up to the camera, which is usually the part you needed anyway.
What if the download fails?
Usually the video sits behind a login or an age gate, and the fix is to pass your browser session with --cookies-from-browser. If it fails on every link, check that ffmpeg is installed, since yt-dlp needs it to join high quality video and audio into one file.
Quick summary · key points
- ✓The three things you install once
- ✓Claude Code
- ✓yt-dlp
- ✓ffmpeg
- ✓The skill that fetches the video
- ✓Make the folder