Auto-clip a long video into shorts

Turn one long-form video (podcast clip, webinar, vlog, talking-head) into several ready-to-publish shorts. Kuhukoo transcribes the video, finds the best moments, and produces vertical clips with captions — you review, tweak, and publish. As of E18, kuhukoo can also track the speaker's face across each clip so the 9:16 crop keeps them in frame.

When you see the button

Composer → attach exactly one video (90 s – 20 min, ≤ 500 MB) → ✂️✨ Auto-clip into shorts.

If the video is shorter than 90 s, longer than 20 min, or over 500 MB the button doesn't appear. A small note explains why for the middle case.

Tier requirement

Auto-clip is a Growth+ feature (same tier as AI alt-text and analyze-media). Free and Starter tenants can attach + preview videos in the composer but the button won't fire. Upgrade in Settings → Billing.

Step 1 · Settings

• How many clips — 3 / 5 / 8. Non-overlapping clips are picked, so asking for 8 on a short source may return fewer.
• Length preset — 15–30 s (TikTok), 20–60 s (Reels, default), 30–90 s (Shorts).
• Platform — Reels / TikTok / Shorts. Informational for now.
• Caption style — Netflix / TikTok / Instagram / Cinema.
• Framing (E18) — 🎯 Smart (track speaker) [default] or Centered. Smart runs MediaPipe FaceDetector on each kept clip and pans the 9:16 crop to follow the face. Centered is the older behaviour (fixed centered 9:16 slice) — a good pick when the source is a wide landscape scene without a clear subject.
• Auto-generate captions — ON by default. Uses the transcript we've already made, so it's essentially free.

Low-power devices (< 4 CPU cores or < 4 GB RAM) auto-default to Centered so tracking doesn't lag the render. You can still flip to Smart manually.

Step 2 · Progress

Runs locally in your browser (nothing but the transcript snippets is sent to a server):

1. Transcribing — Whisper.js (~40 MB, cached forever after first download).
2. Finding moments — scene changes + audio energy peaks.
3. Scoring with AI — one Gemini call per run rates each candidate window and drafts a title + caption + hashtags.
4. Building clips — assembles the top N non-overlapping windows.

Step 3 · Results

Each card shows the suggested title, duration, score, and a transcript snippet. Cards using Smart framing carry a 🎯 badge. Per-clip toggle switches THIS clip between 🎯 Smart and → Center without touching the session default.

• Preview plays the source video constrained to that clip's range.
• Discard removes it from the batch (you can restore).
• Save all kicks off the batch render.

Smart framing — how it works

Post-selection (so we only pay for the tracking cost on kept clips):

1. A detached <video> element decodes the source once and is reused across all clip tracks — the browser never re-decodes.
2. Per kept clip, MediaPipe FaceDetector (Apache-2.0, ~230 KB model from Google's mediapipe-models bucket) samples 2 fps and returns per-frame face boxes.
3. pickDominantFace clusters detections across frames to pick the primary subject (weighted mean of presence + face area + optional active-speaker peak alignment when audio-energy peaks help disambiguate).
4. smoothToKeyframes produces a jitter-free pan track: dead-zone (crop only moves when the face leaves the inner 30% zone), exponential smoothing, max pan speed clamp.
5. If detection drops for > 2 s the crop holds its last known center, then drifts back to geometric center over 1.5 s.
6. The keyframes bake into a single ffmpeg crop filter with piecewise-linear x/y expressions — one filter call per clip, no concat overhead.

When Smart framing falls back to Centered

Honest limits:

• No face detected in any sample → we use a centered crop and the 🎯 badge disappears.
• Strong profile shots (> 60°) — MediaPipe's short-range detector loses the face intermittently; the hold-then-drift fallback keeps the crop sensible rather than snapping.
• Multi-face podcast (two hosts) — the active-speaker heuristic prefers the face aligned with audio-energy peaks. When both hosts speak in the same clip we tend to lock onto the visually-larger one, which is often the correct answer but not always. Manual toggle to Centered per-clip is the escape hatch.
• Glasses / occluded faces — MediaPipe handles glasses well; heavy sunglasses or hands-in-front-of-face degrade detection.

What "Save all" does

For each kept clip:

1. Trims the source to the chosen window.
2. Applies either the tracked pan (Smart) or a centered 9:16 slice (Centered).
3. Burns in captions styled per your Step 1 pick.
4. Renders to mp4 in-browser (ffmpeg.wasm).
5. Creates a draft Post with the caption + hashtags prefilled. No auto-publish — you review and hit Publish yourself.

Edge cases

• No speech in the source — the modal shows "No speech detected in this video."
• Non-English speech — Whisper English-only model may transcribe imperfectly; captions still appear but you may need to edit before publishing.
• Gemini unavailable — clips still get picked by local signals only, titled "Clip 1 … N", captions + hashtags left empty for you to fill.

Feedback

Text-based edit-by-transcript and full Descript-style workflows are queued for a future round — report the moments where Smart framing missed the speaker so we can tune the heuristic: /help/report-bug.