Objects
Track up to five things you name through a video and see when each is on screen.
objects follows the things you name through a video and reports when each was on screen, with a box and an outline
on every frame. Name up to 5 in objects, in your own words: "water bottle", "red running shoe". An empty objects
is refused with INVALID_CAPABILITIES (422). Video length and fps are capped per plan: see Limits.
Only what you name is found: there is no "detect everything" mode. And there is no identity across cuts: two
appearances of "a bottle" may be different bottles, so total_appearances counts stretches of screen time, not
objects.
Request fields
| Prop | Type | Description |
|---|---|---|
| objects | string[] | Required. 1 to 5 things, up to 100 characters each. Trimmed, lowercased, repeats dropped. |
| fps | integer (1-4) | Frames looked at per second. Default 1, clamped to your plan's max_fps. |
Something on screen for less than one frame interval may be missed: raise fps to catch brief appearances.
curl -X POST https://api.netraflow.com/v1/jobs \
-H "Content-Type: application/json" \
-H "X-Api-Key: sk_live_your_key_here" \
-d '{
"url": "https://www.youtube.com/watch?v=dQw4w9WgXcQ",
"capabilities": ["objects"],
"objects": ["water bottle", "running shoe"]
}'{
"data": {
"job_id": "job_abcdefghij123456",
"status": "completed",
"objects": ["water bottle", "running shoe"],
"results": {
"objects": [
{
"name": "running shoe",
"confidence": 0.91,
"first_seen": 4.2,
"last_seen": 38.6,
"total_visible_duration": 21.4,
"total_appearances": 3,
"appearances": [
{
"timestamp_start": 4.2,
"timestamp_end": 12.8,
"visible_duration": 8.6,
"confidence": 0.91,
"bounding_box": { "x": 0.41, "y": 0.62, "width": 0.18, "height": 0.14 },
"frames": [
{
"timestamp": 4.2,
"score": 0.89,
"box": { "x": 0.41, "y": 0.62, "width": 0.18, "height": 0.14 },
"mask": { "size": [144, 256], "counts": [1820, 41, 214, 46] }
}
]
}
]
}
]
}
}
}Response fields
results.objects has one entry per thing found. A thing never seen is absent.
| Prop | Type | Description |
|---|---|---|
| name | string | The phrase you asked for, trimmed and lowercased. |
| confidence | number | How well the region matches your words, 0-1. |
| first_seen | number | Seconds into the video when it first appears. |
| last_seen | number | Seconds into the video when it was last seen. |
| total_visible_duration | number | Seconds on screen across every appearance. |
| total_appearances | number | How many separate stretches of screen time it had. |
| appearances | ObjectAppearance[] | Each unbroken stretch of screen time. |
Appearance fields
| Prop | Type | Description |
|---|---|---|
| timestamp_start | number | Seconds into the video where the stretch begins. |
| timestamp_end | number | Seconds into the video where it ends. |
| visible_duration | number | Length of the stretch in seconds. |
| confidence | number | The score on the representative frame, 0-1. |
| bounding_box | { x, y, width, height } | The representative frame's box, as 0-1 fractions. Good for a thumbnail or a jump-to. |
| frames | ObjectFrame[] | Every frame it was found on, with a timestamp, score, box and run-length outline, so an overlay can follow it. Two in one frame are two entries. Absent on older results. |
Visual evidence
With return_proof on, an objects job adds a storyboard, so the boxes have frames to sit on. See
Frames.
When detection could not run
When detection is unavailable, the job fails with OBJECT_DETECTION_UNAVAILABLE, free, or completes without
objects: see When one capability cannot run. When it ran
but couldn't finish, results.meta.object_detection_degraded is true. Check both before reading an empty list as
"not in the video".
See also: Object detection and tracking, with a real result.