Whole-video re-identification
The person is found again after scene cuts, camera changes and re-entries, in every scene of the searched range.
CatalogLatency Async; roughly 2-3x the length of the searched range Async true Rate Limit Per API key; jobs run one at a time per GPU Max Input Searched range up to 600 seconds, file up to 500 MB
cn8/person-tracking
Draw a box around a person's face or body on one frame of your video. The service finds the
Learn more about Person Tracking Videoasync (job)Instant previewvideotrackingperson
Input
string
Publicly reachable video URL (e.g. upload-media public_url). MP4 (H.264) recommended.
number
Second where the box is drawn. The person's face should be clearly visible on that frame.
object
Box around the person's face or body in percent of the frame: {x, y, w, h} (top-left corner + size, 0-100).
Instant sample preview — no cost, no account. Sign up free to run with your own input.
Result
Completed Instant previewJson
{
"status": "success",
"data": {
"status": "completed",
"warnings": [],
"video": {
"duration": 175.1,
"fps": 30,
"width": 1280,
"height": 640
},
"range": {
"start": 0,
"end": 175.1
},
"person": {
"reference": "body",
"faceVisible": true
},
"appearances": [
{
"start": 0,
"end": 9.7,
"frames": 292
},
{
"start": 12.067,
"end": 13.967,
"frames": 58
}
],
"frames": [
{
"frame": 4,
"time": 0.133,
"box": {
"x": 35.285,
"y": 4.573,
"w": 41.237,
"h": 95.427
},
"confidence": 1,
"polygons": [
[
[
47.59,
12.66
],
[
47.03,
30.26
],
[
49.64,
43.65
],
[
48.13,
48
]
]
],
"face": {
"box": {
"x": 50.141,
"y": 14.894,
"w": 12.612,
"h": 36.872
},
"score": 0.709
}
}
],
"stats": {
"scenes": 57,
"scenesWithObject": 31,
"framesWithObject": 2810,
"framesWithFace": 2294,
"framesPredicted": 0,
"processingSeconds": 318.1
}
},
"usage": {
"units": 175.1
}
}Example response from this endpoint's API reference. Sign in to run it on your own input.
This request costs 0.4 credits per second.
Overview
Draw a box around a person's face or body on one frame of your video. The service finds the same person in every scene where they appear, not just until the next cut, and tracks them frame by frame. Identity is decided by face, so people in the same clothes or the same team kit are not mixed up; where no face shows (from behind, far away) body and clothing decide. You get, for every frame where the person is visible, their box or exact outline as polygon points and their face box, in percent of the frame, so overlays drawn from them fit any screen size or player. Face data is used only while the job runs; face embeddings are never returned or stored. This is an async job: submit it, poll until it's ready, then read the frames.
Key capabilities
Told apart by face
Only the marked person is followed; others in the same clothes are rejected by face. Where no face shows, body and clothing decide.
Face box in every frame
Each frame carries the person's face box with its similarity to the marked face, ready for face blurring, vertical reframing or actor info cards.
Box or exact outline
outputType box returns a rectangle per frame; polygon adds the person's outline as points.
Resolution-independent
All coordinates are percent of the frame (0-100), so overlays scale to any player size.
Public manifest for your player
Publish a finished job with /v1/interactive-videos to get a public JSON manifest a web player can load without your API key.
When to use it
Actor and speaker info cards
Show who is on screen and link to their profile whenever they appear (X-Ray style).
Privacy and face blurring
Blur one person through the whole video, or keep only them visible and blur everyone else.
Highlights and vertical reframing
Cut every moment a player or presenter is on screen, or crop 9:16 around their face.
Input & output
input
Video URL, the second where the box is drawn, the box on the face or body in percent, optional range and output type
JSON body
output
Appearances (time ranges) and per-frame box / polygon points and face box in percent (async job result)
JSON
Guides & tips
How it works
- The person inside your box is segmented on the reference frame; their face (if visible) and their body look are remembered.
- The searched range is split into scenes; in each scene, people are found and compared with the marked person, first by face, then by body where no face shows.
- In scenes where they match, the person is tracked forward and backward frame by frame; a track whose faces mostly belong to someone else is dropped.
Getting good results
- Mark a frame where the person's face is clearly visible and fairly large. The box may cover only the face or the whole body.
- If no face shows on the marked frame, identity falls back to body and clothing (person.faceVisible is false and a warning says so); people dressed alike may then be confused.
- Search only the part you need with startTime/endTime: it is faster and cheaper.
- Very small faces in wide crowd shots cannot be recognised; those frames rely on body and clothing.
Reading the frames
- All values are percent of the frame: left = x% of the player width, top = y% of its height.
- frames lists only the frames where the person is visible. Draw frames[i] from frames[i].time until the next frame; if the next entry is more than one frame later (1/fps), the person is off screen in between.
- face is the face box in that frame with score (similarity to the marked face), or null when the face is not visible (from behind, far away). score is null and predicted is true when the face box was filled in a short gap between two detected faces.
- confidence (0-1) shows the evidence: 0.75-1.0 when the face is matched in that frame, 0.75 inside a verified track without a face; lower on estimated frames. It does not flag wrong frames on its own.
- predicted: true marks a short gap (up to 1.5 s, same shot) where the person was momentarily lost and the position was estimated from motion. Draw these frames faded. Gaps across a scene cut are never filled.
Privacy
- Face embeddings exist only in memory while the job runs; they are never returned, logged or stored.
- The result holds boxes and similarity scores only, no biometric template.
Errors
- The submit is always accepted and returns a job id; the request is checked when the job starts.
- A wrong request (no person in the box, time outside the range, range over 600 s, video too large or not reachable) ends as status failed, with the reason in the job's error field.
- Failed jobs are not billed; the reserved credits are released.
Specs
Schema
Request body
videoUrlrequiredstringPublicly reachable video URL (e.g. upload-media public_url). MP4 (H.264) recommended.
timerequirednumberSecond where the box is drawn. The person's face should be clearly visible on that frame.
boxrequiredobjectBox around the person's face or body in percent of the frame: {x, y, w, h} (top-left corner + size, 0-100).
startTimenumberStart of the searched range in seconds. Default 0.
endTimenumberEnd of the searched range in seconds. Default: end of the video. The range may be at most 600 s and must contain time, so for videos longer than 10 minutes always send startTime/endTime.
outputTypestringbox (default) or polygon. polygon adds the person's outline points to every frame.
Pricing
0.4 credits per second of the searched range, minimum 10 seconds. A 3-minute episode searched end to end costs about 70 credits. Only the searched range is billed, not processing time. Failed jobs are not billed.
| Service | Unit | Price |
|---|---|---|
| Video Person Tracking | second | 0.4 credits/second |
Box and polygon output cost the same.
Publishing the manifest (/v1/interactive-videos) is free.