Objects
Track up to five things you name through a video and see when each is on screen.
objects follows the things you name through a video and reports when each one was on screen, with a box and an outline on every frame it was found in.
Name them in objects, in your own words — "water bottle", "red running shoe", "laptop". An objects job with an empty objects array is rejected with INVALID_CAPABILITIES (422).
Two limits are worth knowing before you build on this. Nothing is found that you did not name — there is no
"detect everything" mode, so anything absent from objects is absent from the result. And there is no identity
across cuts: two appearances of "a bottle" are not known to be the same bottle, so total_appearances counts
stretches of screen time, not distinct objects.
Objects analyzes frames, so it uses your plan's standard video ceiling — 1 minute on Free, 3 on Starter, 10 on
Pro, 20 on Scale. Longer videos fail with VIDEO_TOO_LONG. At most 5 things per job; more is rejected at validation.
Detection runs over the frames the job already extracted, so fps shapes how finely a short appearance can be caught. A thing on screen for less than one frame interval may be missed entirely.
curl -X POST https://api.netraflow.com/v1/jobs \
-H "Content-Type: application/json" \
-H "X-Api-Key: sk_live_your_key_here" \
-d '{
"url": "https://www.youtube.com/watch?v=dQw4w9WgXcQ",
"capabilities": ["objects"],
"objects": ["water bottle", "running shoe"]
}'{
"data": {
"job_id": "job_abc123",
"status": "completed",
"objects": ["water bottle", "running shoe"],
"results": {
"objects": [
{
"name": "running shoe",
"confidence": 0.91,
"first_seen": 4.2,
"last_seen": 38.6,
"total_visible_duration": 21.4,
"total_appearances": 3,
"appearances": [
{
"timestamp_start": 4.2,
"timestamp_end": 12.8,
"visible_duration": 8.6,
"confidence": 0.91,
"bounding_box": { "x": 0.41, "y": 0.62, "width": 0.18, "height": 0.14 },
"frames": [
{
"timestamp": 4.2,
"score": 0.89,
"box": { "x": 0.41, "y": 0.62, "width": 0.18, "height": 0.14 },
"mask": { "size": [144, 256], "counts": [1820, 41, 214, 46] }
}
]
}
]
}
]
}
}
}Response fields
results.objects carries one entry per thing you named that was found. A thing that never appeared is simply absent.
| Prop | Type | Description |
|---|---|---|
| name | string | The phrase you asked for, trimmed and lowercased. |
| confidence | number | How sure the detector is that this region is the thing you named, 0-1. It is a match against your words, not a recognition of a known object. |
| first_seen | number | Seconds into the video when it first appears. |
| last_seen | number | Seconds into the video when it was last seen. |
| total_visible_duration | number | Seconds on screen across every appearance. |
| total_appearances | number | How many separate stretches of screen time it had. |
| appearances | ObjectAppearance[] | Each unbroken stretch of screen time. |
ObjectAppearance
| Prop | Type | Description |
|---|---|---|
| timestamp_start | number | Seconds into the video where the stretch begins. |
| timestamp_end | number | Seconds into the video where it ends. |
| visible_duration | number | Length of the stretch in seconds. |
| confidence | number | The detector's score on the representative frame, 0-1. |
| bounding_box | { x, y, width, height } | The representative frame's box, as fractions of frame width and height — a stable anchor for a thumbnail or a jump-to. |
| frames | ObjectFrame[] | Per-frame detections across the stretch, each with a timestamp, a score, a box and a run-length outline, so an overlay can follow the thing rather than draw one static box. Absent on results stored before this shape existed. |
Visual evidence
An objects job with return_proof left on builds a storyboard, the same filmstrip a brands job gets, so the boxes and outlines have frames to sit on. Its sheets are fetched through GET /v1/frames/:key.
When detection could not run
If the detector was unavailable, the job still completes and results.meta.object_detection_degraded is true with objects short or empty. Check that flag before reading an empty list as "not in the video".