[{"data":1,"prerenderedAt":184},["ShallowReactive",2],{"reference-detail-en-understand_media_api_v1_media_understand_post":3,"developer-code-9xxlc7":182,"developer-code-optk0u":183},{"tag":4,"sourceTag":4,"tagSlug":5,"method":6,"path":7,"slug":8,"summary":9,"description":10,"operationId":11,"groupOrder":12,"descriptionHtml":13,"parameters":14,"requestBody":22,"responses":80,"security":169,"codeSamples":172},"Media Understanding","media-understanding","POST","\u002Fapi\u002Fv1\u002Fmedia\u002Funderstand","understand-media","Understand media (structured)","Label or extract structured information from images, audio, and video with a frontier multimodal model.\nYou supply the instructions (`prompt`), the media (`inputs`), and a JSON `schema`; the response `output`\nconforms to that schema — closed label sets, scores, timestamps, transcribed fields, whatever you define.\nSynchronous. Billing follows the upstream model's token usage (input media and prompt, plus output\nincluding reasoning) at 1 credit per $0.10, rounded up per call, charged only on success. Per-key\nconcurrency is limited during beta; requests beyond the limit are rejected, not queued.","understand_media_api_v1_media_understand_post",999,"\u003Cp>Label or extract structured information from images, audio, and video with a frontier multimodal model. You supply the instructions (\u003Ccode>prompt\u003C\u002Fcode>), the media (\u003Ccode>inputs\u003C\u002Fcode>), and a JSON \u003Ccode>schema\u003C\u002Fcode>; the response \u003Ccode>output\u003C\u002Fcode> conforms to that schema — closed label sets, scores, timestamps, transcribed fields, whatever you define. Synchronous. Billing follows the upstream model's token usage (input media and prompt, plus output including reasoning) at 1 credit per $0.10, rounded up per call, charged only on success. Per-key concurrency is limited during beta; requests beyond the limit are rejected, not queued.\u003C\u002Fp>",[15],{"name":16,"in":17,"required":18,"description":19,"type":20,"example":21},"X-API-Key","header",true,"Your VisionStory API key (`sk-vs-...`), kept server-side. Create or manage keys in [API keys](\u002Fapi-keys) (Pro plan and up).","string","sk-vs-your-api-key",{"required":18,"contentType":23,"schema":24,"example":69},"application\u002Fjson",{"type":25,"description":26,"nullable":27,"properties":28},"MediaUnderstandRequest","",false,[29,60,64],{"name":30,"required":18,"type":31,"description":32,"nullable":27,"properties":33,"items":34},"inputs","MediaRef[]","1-8 media items (image \u002F audio \u002F video), each as `asset_id`, public `url`, or `inline_data`. Mixed kinds are fine. Not stored beyond the request.",[],{"type":35,"description":36,"nullable":27,"properties":37},"MediaRef","A media reference — provide exactly one of: `asset_id` (from the asset library, reusable across\nrequests), `url` (one-off public URL), or `inline_data` (one-off base64, not stored).",[38,43,56],{"name":39,"required":27,"type":40,"description":41,"nullable":18,"properties":42},"asset_id","string | null","Asset ID from POST \u002Fapi\u002Fv1\u002Fasset; use for materials reused across requests.",[],{"name":44,"required":27,"type":45,"description":46,"nullable":18,"properties":47},"inline_data","InlineDataModel | null","Inline base64 media data for one-off use; not added to your asset library. Images: image\u002Fjpeg, image\u002Fjpg, image\u002Fpng, image\u002Fwebp, image\u002Fbmp, image\u002Ftiff, image\u002Fgif; audio: audio\u002Fwav, audio\u002Fx-wav, audio\u002Fwave, audio\u002Fmpeg, audio\u002Fmp3; video: video\u002Fmp4, video\u002Fquicktime, video\u002Fmov.",[48,52],{"name":49,"required":18,"type":20,"description":50,"nullable":27,"properties":51},"data","The file's raw bytes encoded as a base64 string (no `data:` URI prefix).",[],{"name":53,"required":18,"type":20,"description":54,"nullable":27,"properties":55},"mime_type","MIME type of the inline data; the gateway uses it to tell image \u002F audio \u002F video apart. The accepted set depends on the endpoint — see the field carrying this object. Talking video accepts audio ['audio\u002Favi', 'audio\u002Fmpeg', 'audio\u002Fmp3', 'audio\u002Fmp4', 'audio\u002Fm4a', 'audio\u002Fwav'] and images ['image\u002Fjpeg', 'image\u002Fjpg', 'image\u002Fpng', 'image\u002Fwebp', 'image\u002Fheic'].",[],{"name":57,"required":27,"type":40,"description":58,"nullable":18,"properties":59},"url","Publicly accessible media URL for one-off use; not added to your asset library.",[],{"name":61,"required":18,"type":20,"description":62,"nullable":27,"properties":63},"prompt","What to extract or label. Describe the task and, if helpful, the meaning of each schema field; the model sees the media in `inputs` in the given order.",[],{"name":65,"required":18,"type":66,"description":67,"nullable":27,"properties":68},"schema","object","JSON Schema (draft 2020-12 subset) the output must conform to. Top level must be an object. Keep it flat and explicit — enums for closed label sets, `description` on each field.",[],{"inputs":70,"prompt":73,"schema":74},[71],{"url":72},"https:\u002F\u002Fexample.com\u002Fcat.jpg","Identify the main subject in the image.",{"additionalProperties":27,"properties":75,"required":78,"type":66},{"subject":76},{"description":77,"type":20},"Main subject in the image.",[79],"subject",[81,141],{"status":82,"description":83,"contentType":23,"schema":84,"example":123,"examples":132,"headers":140},"200","Successful Response",{"type":85,"description":26,"nullable":27,"properties":86},"BaseSuccessResponse_MediaUnderstandResponse_",[87,113,118],{"name":49,"required":18,"type":88,"description":89,"nullable":18,"properties":90},"MediaUnderstandResponse | null","The endpoint-specific payload. See this endpoint's response schema for its fields; null only when the operation returns no payload.",[91,96,100],{"name":92,"required":18,"type":93,"description":94,"nullable":27,"properties":95},"cost_credit","integer","Credits charged for this call (usage priced at upstream rates, rounded up to a whole credit, minimum 1).",[],{"name":97,"required":18,"type":66,"description":98,"nullable":27,"properties":99},"output","The structured result, conforming to the request `schema`.",[],{"name":101,"required":18,"type":102,"description":26,"nullable":27,"properties":103},"usage","MediaUnderstandUsage",[104,109],{"name":105,"required":27,"type":93,"description":106,"nullable":27,"default":107,"properties":108},"input_tokens","Tokens consumed by the prompt and media inputs.",0,[],{"name":110,"required":27,"type":93,"description":111,"nullable":27,"default":107,"properties":112},"output_tokens","Tokens generated (including the model's reasoning).",[],{"name":114,"required":27,"type":20,"description":115,"nullable":27,"default":116,"properties":117},"message","Human-readable status message; `\"success\"` on a successful call.","success",[],{"name":119,"required":18,"type":120,"description":121,"nullable":27,"properties":122},"server_time","string · date-time","Server-side timestamp when the response was produced, in ISO 8601 format (UTC).",[],{"data":124,"message":116,"server_time":131},{"cost_credit":125,"output":126,"usage":128},1,{"subject":127},"cat",{"input_tokens":129,"output_tokens":130},120,12,"2026-09-04T00:00:00Z",[133],{"name":116,"description":134,"summary":135,"value":136},"Examples illustrate the response structure, not live account data or a pricing quote. IDs and URLs are placeholders. Query model discovery and cost endpoints for current availability and prices.","Successful response",{"data":137,"message":116,"server_time":131},{"cost_credit":125,"output":138,"usage":139},{"subject":127},{"input_tokens":129,"output_tokens":130},[],{"status":142,"description":143,"contentType":23,"schema":144,"example":166,"headers":168},"default","Error response. All failures share one envelope: an `error` object with a numeric `code`, a human-readable `message`, an optional `details` string, and an optional `hint` giving an actionable next step (useful for AI agents).",{"type":145,"description":26,"nullable":27,"properties":146},"ErrorResponse",[147],{"name":148,"required":18,"type":149,"description":26,"nullable":27,"properties":150},"error","ErrorDetail",[151,155,159,163],{"name":152,"required":18,"type":93,"description":153,"nullable":27,"properties":154},"code","Machine-readable error code. Mirrors the HTTP status for transport-level failures (e.g. 401, 404, 422, 500) and may carry a business-specific code otherwise.",[],{"name":156,"required":27,"type":40,"description":157,"nullable":18,"properties":158},"details","Optional structured detail about the failure, e.g. a JSON string of per-field validation errors on a 422. Absent when there is nothing extra to report.",[],{"name":160,"required":27,"type":40,"description":161,"nullable":18,"properties":162},"hint","Actionable next step for resolving the error, written for both humans and AI agents (e.g. how to fix the request, or where to obtain an API key). May be absent.",[],{"name":114,"required":18,"type":20,"description":164,"nullable":27,"properties":165},"Human-readable explanation of what went wrong. Safe to log or surface to end users; not localized.",[],{"error":167},{"code":125,"message":20,"details":20,"hint":20},[],[170],{"APIKeyHeader":171},[],[173,176,179],{"lang":174,"source":175},"SDK","from visionstory import VisionStoryClient\n\nclient = VisionStoryClient.from_env()\nresult = client.understand_media(\n    prompt=\"Identify the main subject in the image.\",\n    inputs=[{\"url\": \"https:\u002F\u002Fexample.com\u002Fmedia\u002Fphoto.jpg\"}],\n    schema={\"type\": \"object\", \"properties\": {\"subject\": {\"type\": \"string\"}},\n            \"required\": [\"subject\"], \"additionalProperties\": False},\n)\nprint(result)",{"lang":177,"source":178},"REST","import os\nimport requests\n\nresponse = requests.post(\n    \"https:\u002F\u002Fopenapi.visionstory.ai\u002Fapi\u002Fv1\u002Fmedia\u002Funderstand\",\n    headers={\"X-API-Key\": os.environ[\"VISIONSTORY_API_KEY\"]},\n    json={\n      \"inputs\": [\n        {\n          \"url\": \"https:\u002F\u002Fexample.com\u002Fcat.jpg\"\n        }\n      ],\n      \"prompt\": \"Identify the main subject in the image.\",\n      \"schema\": {\n        \"additionalProperties\": False,\n        \"properties\": {\n          \"subject\": {\n            \"description\": \"Main subject in the image.\",\n            \"type\": \"string\"\n          }\n        },\n        \"required\": [\n          \"subject\"\n        ],\n        \"type\": \"object\"\n      }\n    },\n)\nresponse.raise_for_status()\nprint(response.json())",{"lang":180,"source":181},"cURL","curl --request POST 'https:\u002F\u002Fopenapi.visionstory.ai\u002Fapi\u002Fv1\u002Fmedia\u002Funderstand' \\\n  --header 'X-API-Key: $VISIONSTORY_API_KEY' \\\n  --header 'Content-Type: application\u002Fjson' \\\n  --data '{\n  \"inputs\": [\n    {\n      \"url\": \"https:\u002F\u002Fexample.com\u002Fcat.jpg\"\n    }\n  ],\n  \"prompt\": \"Identify the main subject in the image.\",\n  \"schema\": {\n    \"additionalProperties\": false,\n    \"properties\": {\n      \"subject\": {\n        \"description\": \"Main subject in the image.\",\n        \"type\": \"string\"\n      }\n    },\n    \"required\": [\n      \"subject\"\n    ],\n    \"type\": \"object\"\n  }\n}'","\u003Cpre class=\"shiki github-dark-default\" style=\"background-color:#0d1117;color:#e6edf3\" tabindex=\"0\">\u003Ccode>\u003Cspan class=\"line\">\u003Cspan style=\"color:#FF7B72\">from\u003C\u002Fspan>\u003Cspan style=\"color:#E6EDF3\"> visionstory \u003C\u002Fspan>\u003Cspan style=\"color:#FF7B72\">import\u003C\u002Fspan>\u003Cspan style=\"color:#E6EDF3\"> VisionStoryClient\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E6EDF3\">client \u003C\u002Fspan>\u003Cspan style=\"color:#FF7B72\">=\u003C\u002Fspan>\u003Cspan style=\"color:#E6EDF3\"> VisionStoryClient.from_env()\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E6EDF3\">result \u003C\u002Fspan>\u003Cspan style=\"color:#FF7B72\">=\u003C\u002Fspan>\u003Cspan style=\"color:#E6EDF3\"> client.understand_media(\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#FFA657\">    prompt\u003C\u002Fspan>\u003Cspan style=\"color:#FF7B72\">=\u003C\u002Fspan>\u003Cspan style=\"color:#A5D6FF\">\"Identify the main subject in the image.\"\u003C\u002Fspan>\u003Cspan style=\"color:#E6EDF3\">,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#FFA657\">    inputs\u003C\u002Fspan>\u003Cspan style=\"color:#FF7B72\">=\u003C\u002Fspan>\u003Cspan style=\"color:#E6EDF3\">[{\u003C\u002Fspan>\u003Cspan style=\"color:#A5D6FF\">\"url\"\u003C\u002Fspan>\u003Cspan style=\"color:#E6EDF3\">: \u003C\u002Fspan>\u003Cspan style=\"color:#A5D6FF\">\"https:\u002F\u002Fexample.com\u002Fmedia\u002Fphoto.jpg\"\u003C\u002Fspan>\u003Cspan style=\"color:#E6EDF3\">}],\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#FFA657\">    schema\u003C\u002Fspan>\u003Cspan style=\"color:#FF7B72\">=\u003C\u002Fspan>\u003Cspan style=\"color:#E6EDF3\">{\u003C\u002Fspan>\u003Cspan style=\"color:#A5D6FF\">\"type\"\u003C\u002Fspan>\u003Cspan style=\"color:#E6EDF3\">: \u003C\u002Fspan>\u003Cspan style=\"color:#A5D6FF\">\"object\"\u003C\u002Fspan>\u003Cspan style=\"color:#E6EDF3\">, \u003C\u002Fspan>\u003Cspan style=\"color:#A5D6FF\">\"properties\"\u003C\u002Fspan>\u003Cspan style=\"color:#E6EDF3\">: {\u003C\u002Fspan>\u003Cspan style=\"color:#A5D6FF\">\"subject\"\u003C\u002Fspan>\u003Cspan style=\"color:#E6EDF3\">: {\u003C\u002Fspan>\u003Cspan style=\"color:#A5D6FF\">\"type\"\u003C\u002Fspan>\u003Cspan style=\"color:#E6EDF3\">: \u003C\u002Fspan>\u003Cspan style=\"color:#A5D6FF\">\"string\"\u003C\u002Fspan>\u003Cspan style=\"color:#E6EDF3\">}},\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#A5D6FF\">            \"required\"\u003C\u002Fspan>\u003Cspan style=\"color:#E6EDF3\">: [\u003C\u002Fspan>\u003Cspan style=\"color:#A5D6FF\">\"subject\"\u003C\u002Fspan>\u003Cspan style=\"color:#E6EDF3\">], \u003C\u002Fspan>\u003Cspan style=\"color:#A5D6FF\">\"additionalProperties\"\u003C\u002Fspan>\u003Cspan style=\"color:#E6EDF3\">: \u003C\u002Fspan>\u003Cspan style=\"color:#79C0FF\">False\u003C\u002Fspan>\u003Cspan style=\"color:#E6EDF3\">},\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E6EDF3\">)\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#79C0FF\">print\u003C\u002Fspan>\u003Cspan style=\"color:#E6EDF3\">(result)\u003C\u002Fspan>\u003C\u002Fspan>\u003C\u002Fcode>\u003C\u002Fpre>","\u003Cpre class=\"shiki github-dark-default\" style=\"background-color:#0d1117;color:#e6edf3\" tabindex=\"0\">\u003Ccode>\u003Cspan class=\"line\">\u003Cspan style=\"color:#E6EDF3\">{\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#7EE787\">  \"data\"\u003C\u002Fspan>\u003Cspan style=\"color:#E6EDF3\">: {\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#7EE787\">    \"cost_credit\"\u003C\u002Fspan>\u003Cspan style=\"color:#E6EDF3\">: \u003C\u002Fspan>\u003Cspan style=\"color:#79C0FF\">1\u003C\u002Fspan>\u003Cspan style=\"color:#E6EDF3\">,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#7EE787\">    \"output\"\u003C\u002Fspan>\u003Cspan style=\"color:#E6EDF3\">: {\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#7EE787\">      \"subject\"\u003C\u002Fspan>\u003Cspan style=\"color:#E6EDF3\">: \u003C\u002Fspan>\u003Cspan style=\"color:#A5D6FF\">\"cat\"\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E6EDF3\">    },\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#7EE787\">    \"usage\"\u003C\u002Fspan>\u003Cspan style=\"color:#E6EDF3\">: {\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#7EE787\">      \"input_tokens\"\u003C\u002Fspan>\u003Cspan style=\"color:#E6EDF3\">: \u003C\u002Fspan>\u003Cspan style=\"color:#79C0FF\">120\u003C\u002Fspan>\u003Cspan style=\"color:#E6EDF3\">,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#7EE787\">      \"output_tokens\"\u003C\u002Fspan>\u003Cspan style=\"color:#E6EDF3\">: \u003C\u002Fspan>\u003Cspan style=\"color:#79C0FF\">12\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E6EDF3\">    }\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E6EDF3\">  },\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#7EE787\">  \"message\"\u003C\u002Fspan>\u003Cspan style=\"color:#E6EDF3\">: \u003C\u002Fspan>\u003Cspan style=\"color:#A5D6FF\">\"success\"\u003C\u002Fspan>\u003Cspan style=\"color:#E6EDF3\">,\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#7EE787\">  \"server_time\"\u003C\u002Fspan>\u003Cspan style=\"color:#E6EDF3\">: \u003C\u002Fspan>\u003Cspan style=\"color:#A5D6FF\">\"2026-09-04T00:00:00Z\"\u003C\u002Fspan>\u003C\u002Fspan>\n\u003Cspan class=\"line\">\u003Cspan style=\"color:#E6EDF3\">}\u003C\u002Fspan>\u003C\u002Fspan>\u003C\u002Fcode>\u003C\u002Fpre>",1788547144653]