Documentation

Chat API Reference

Agentic chat sessions with image and document context, streaming responses

Agentic Chat

Create chat sessions to have AI-powered conversations about your images and documents. The system uses specialized agents for search, analysis, document search, analytics, cross-referencing, and folder organization.

Tip: Save Tokens with Standalone Search

If you only need to search images or documents without multi-turn conversation, use the standalone search agents directly. They consume significantly fewer tokens by skipping session overhead and conversational context.

Session Management

POST/api/v2/chat/sessions

Create a new chat session

Request

json
{
"title": "Equipment Analysis", // optional, max 255 chars
"initial_image_ids": ["550e8400-e29b-41d4-a716-446655440000", "660f9500-f39c-52e5-b827-557766550111"], // optional, up to 1000 if use_all_images is false
"use_all_images": true, // default: true
"domain": "civil_engineering", // optional, default: "general"
"project_id": "770a0600-e29b-41d4-a716-446655440000" // optional
}
// domain: tailors agent search and analysis behavior
// values: general, civil_engineering, maps, mining, construction, facility_assessment
// project_id: scope file queries to a specific project workspace

Response

json
{
"session_id": "550e8400-e29b-41d4-a716-446655440000",
"title": "Equipment Analysis",
"domain": "civil_engineering",
"project_id": "770a0600-e29b-41d4-a716-446655440000",
"created_at": "2025-01-15T10:30:00Z"
}
GET/api/v2/chat/sessions

List user's chat sessions with optional filtering

Request

json
// Query parameters:
?search=equipment // Search in session titles and message content (max 200 chars)
&is_active=true // Filter by active status (true/false)
&min_messages=1 // Filter sessions with at least N messages
&limit=20 // 1-100, default 20
&offset=0 // default 0

Response

json
{
"items": [
{
"id": "550e8400-e29b-41d4-a716-446655440000",
"title": "Equipment Analysis",
"total_messages": 12,
"total_tokens": 5400,
"remaining_tokens": 94600,
"remaining_messages": 88,
"is_active": true,
"use_all_images": true,
"selected_image_count": 0,
"domain": "civil_engineering",
"project_id": null,
"last_message_at": "2025-01-15T11:00:00Z",
"last_message_preview": "Based on the analysis...",
"last_user_message": "What damage do you see?",
"created_at": "2025-01-15T10:30:00Z",
"updated_at": "2025-01-15T11:00:00Z"
}
],
"total_count": 5,
"limit": 20,
"offset": 0,
"has_more": false
}
GET/api/v2/chat/sessions/{session_id}

Get session details with message history

Request

json
// Query parameters:
?include_messages=true // Include messages in response (default: true)
&message_limit=50 // Limit number of messages returned (1-1000, optional)

Response

json
{
"session": {
"id": "550e8400-e29b-41d4-a716-446655440000",
"title": "Equipment Analysis",
"total_messages": 4,
"total_tokens": 1200,
"remaining_tokens": 94000,
"remaining_messages": 96,
"is_active": true,
"use_all_images": false,
"selected_image_count": 2,
"last_message_at": "2025-01-15T10:31:02Z",
"last_message_preview": "I can see corrosion on...",
"last_user_message": "What damage do you see?",
"created_at": "2025-01-15T10:30:00Z",
"updated_at": "2025-01-15T10:31:02Z"
},
"messages": [
{
"id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
"role": "user",
"content": "What damage do you see?",
"token_count": 12,
"processing_time_ms": null,
"created_at": "2025-01-15T10:31:00Z",
"image_context": null,
"metadata": null
},
{
"id": "b2c3d4e5-f6a7-8901-bcde-f12345678901",
"role": "assistant",
"content": "I can see corrosion on...",
"token_count": 450,
"processing_time_ms": 1250,
"created_at": "2025-01-15T10:31:02Z",
"image_context": [
{
"image_id": "550e8400-e29b-41d4-a716-446655440000",
"thumbnail_url": "https://...",
"description": "Steel beam with rust",
"confidence": 0.95,
"title": "Beam photo",
"stored_url": "https://...",
"filename": "beam_001.jpg"
}
],
"metadata": {
"vision_used": true,
"offers_detailed_analysis": false
}
}
],
"selected_image_ids": ["550e8400-e29b-41d4-a716-446655440000", "660f9500-f39c-52e5-b827-557766550111"],
"selected_video_ids": [],
"context_summary": "2 images selected"
}
PATCH/api/v2/chat/sessions/{session_id}/title

Rename a chat session

Request

json
{
"title": "Updated Session Title" // required, 1-255 chars, trimmed
}

Response

json
{
"session_id": "550e8400-e29b-41d4-a716-446655440000",
"title": "Updated Session Title",
"updated_at": "2025-01-15T11:00:00Z"
}
DELETE/api/v2/chat/sessions/{session_id}

Close an active chat session and release resources

Response

json
{
"session_id": "550e8400-e29b-41d4-a716-446655440000",
"closed_at": "2025-01-15T11:00:00Z",
"total_messages": 12,
"total_tokens": 5400
}

Messaging

POST/api/v2/chat/sessions/{session_id}/messages

Send message and get complete response (non-streaming)

Request

json
{
"message": "What safety issues do you see in these images?", // required, 1-10000 chars
"force_detailed_analysis": false, // optional, default false - force high-resolution image analysis
"expected_image_ids": ["img_001", "img_002"] // optional - validate/sync image selection
}

Response

json
{
"message_id": "550e8400-e29b-41d4-a716-446655440000",
"content": "Based on my analysis of the images...",
"token_count": 450,
"processing_time_ms": 2500,
"remaining_tokens": 94550,
"remaining_messages": 97
}
POST/api/v2/chat/sessions/{session_id}/messages/stream

Send message with streaming response using Server-Sent Events (SSE)

Request

json
{
"message": "What are the key safety requirements?", // required, 1-10000 chars
"force_detailed_analysis": false, // optional, default false
"expected_image_ids": ["img_001", "img_002"] // optional - validate/sync image selection
}

Response

json
// Server-Sent Events stream:
// connection - Initial connection confirmation
event: connection
data: {"type": "connection", "message_id": "msg_001", "status": "connected"}
// token - Individual response tokens
event: token
id: 1
data: {"type": "token", "content": "The", "accumulated": "The", "token_index": 0}
// status - Real-time processing status updates
event: status
data: {"type": "status", "phase": "searching", "message": "Searching documents...", "agent_name": "SearchAgent"}
// thinking - AI reasoning phase indicator
event: thinking
data: {"type": "thinking", "status": "processing"}
// thinking_step - Detailed agent reasoning with content
event: thinking_step
data: {"type": "thinking_step", "phase": "analysis", "agent_name": "AnalysisAgent", "display_name": "Analysis", "message": "Examining images...", "reasoning": "Examining image for..."}
// tool_invocation - Agent tool execution notification
event: tool_invocation
data: {"type": "tool_invocation", "tool_name": "search_images", "agent_name": "SearchAgent"}
// tool_result - Agent tool execution result
event: tool_result
data: {"type": "tool_result", "tool_name": "search_images", "agent_name": "SearchAgent", "result_count": 5}
// image_results - Images found during response
event: image_results
data: {"type": "image_results", "images": [...], "count": 5}
// document_results - Documents found during response
event: document_results
data: {"type": "document_results", "documents": [...], "count": 3}
// complete - Response finished with full content and metadata
event: complete
data: {"type": "complete", "content": "...", "resolved_content": "...", "message_id": "550e8400-...", "session_id": "660f9500-...", "agents_used": ["SearchAgent", "AnalysisAgent"], "has_results": true, "result_count": 5, "image_context": [...], "tool_document_data": [...], "tool_chunk_data": [...], "tool_link_data": [...], "result_refs": {...}, "suppress_gallery": false}
// Optional fields in complete event:
// "reasoning": "..." - only included when debug_reasoning=true query param is set
// "folder_modified": true - present when a folder was modified (for cache invalidation)
// "pending_actions": [...] - present when the response requires user confirmation
// auth_error - Authorization failed during stream
event: auth_error
data: {"type": "auth_error", "error": "Session not found", "message": "Authorization failed during stream initialization"}
// error - Error occurred during processing
// Generic errors have only "type" and "message":
event: error
data: {"type": "error", "message": "An error occurred while processing your request."}
// Usage limit errors additionally include "code" and "context":
event: error
data: {"type": "error", "message": "You've reached your monthly chat message limit (200/200).", "code": "USAGE_LIMIT_EXCEEDED", "context": {"limit_type": "monthly_chat_messages", "current_usage": 200, "monthly_limit": 200, "reset_date": "2025-02-01", "upgrade_url": "/settings/billing"}}
// ping - Heartbeat (keep-alive during long operations)
event: ping
data: {"type": "ping"}
// close - Stream terminated
event: close
data: {"type": "close", "message_id": "msg_001"}

resolved_content in Complete Event

The complete event includes a resolved_content field — the response text with all markup patterns ([[ref:KEY]], [[img:ID|filename]], [[doc:ID|filename|p:N]], [[link:ID|title]]) replaced with human-readable text. Use this for CLI output or any context where you don't need interactive reference rendering. The original content field with markup is always included alongside for custom UI rendering.

auth_error Event

The auth_error event is emitted as the very first event if session authorization fails during stream initialization (e.g., session deleted or ownership changed between the pre-stream check and the actual stream start). When received, the stream terminates immediately. Your client should handle this by showing an error and redirecting the user to create a new session.

Context Management

PATCH/api/v2/chat/sessions/{session_id}/mode

Toggle between all images and selected images mode

Request

json
{
"use_all_images": true // required
}

Response

json
{
"session_id": "550e8400-e29b-41d4-a716-446655440000",
"use_all_images": true,
"selected_image_count": 0,
"updated_at": "2025-01-15T11:00:00Z"
}
PUT/api/v2/chat/sessions/{session_id}/images

Update which images are available in chat context (max 1000)

Request

json
{
"image_ids": [
"550e8400-e29b-41d4-a716-446655440000",
"660f9500-f39c-52e5-b827-557766550111"
]
}

Response

json
{
"session_id": "550e8400-e29b-41d4-a716-446655440000",
"selected_image_ids": [
"550e8400-e29b-41d4-a716-446655440000",
"660f9500-f39c-52e5-b827-557766550111"
],
"selected_video_ids": [],
"context_summary": "2 images selected",
"estimated_tokens": 1500
}
PUT/api/v2/chat/sessions/{session_id}/documents

Update which documents are available in chat context (max 100)

Request

json
{
"document_ids": [
"550e8400-e29b-41d4-a716-446655440000",
"660f9500-f39c-52e5-b827-557766550111"
]
}

Response

json
{
"session_id": "550e8400-e29b-41d4-a716-446655440000",
"selected_document_ids": [
"550e8400-e29b-41d4-a716-446655440000",
"660f9500-f39c-52e5-b827-557766550111"
],
"document_count": 2
}
PUT/api/v2/chat/sessions/{session_id}/videos

Update which videos are available in chat context (max 100)

Request

json
{
"video_ids": [
"550e8400-e29b-41d4-a716-446655440020",
"660f9500-f39c-52e5-b827-557766550121"
]
}

Response

json
{
"session_id": "550e8400-e29b-41d4-a716-446655440000",
"selected_video_ids": [
"550e8400-e29b-41d4-a716-446655440020",
"660f9500-f39c-52e5-b827-557766550121"
],
"video_count": 2
}
PATCH/api/v2/chat/sessions/{session_id}/domain

Update the domain context of a chat session. Domain context tailors agent search and analysis behavior.

Request

json
{
"domain": "civil_engineering"
}
// domain: required, one of:
// "general" — default, general-purpose
// "civil_engineering" — civil engineering context
// "maps" — geographic/cartographic context
// "mining" — mining industry context
// "construction" — construction industry context
// "facility_assessment" — facility assessment context

Response

json
{
"session_id": "550e8400-e29b-41d4-a716-446655440000",
"domain": "civil_engineering",
"updated_at": "2025-01-15T11:00:00Z"
}
// Returns 400 if domain value is invalid
// Returns 404 if session not found
GET/api/v2/chat/images/all

Get all images available for chat context

Request

json
// Query parameters:
?limit=5000 // 1-5000, default 5000
&offset=0 // default 0

Response

json
{
"items": [
"550e8400-e29b-41d4-a716-446655440000",
"660f9500-f39c-52e5-b827-557766550111",
"770a0600-a40d-63f6-c938-668877660222"
],
"total_count": 150,
"limit": 5000,
"offset": 0,
"has_more": false
}
POST/api/v2/chat/images/search

Search for images using natural language queries

Request

json
{
"query": "red cars from last week", // required, 1-500 chars
"limit": 50, // optional, 1-100, default 50
"offset": 0 // optional, default 0
}

Response

json
{
"items": [
{
"image_id": "550e8400-e29b-41d4-a716-446655440000",
"description": "Red sedan in parking lot",
"relevance_score": 0.87,
"thumbnail_url": "https://...",
"medium_url": "https://...",
"full_url": "https://...",
"created_at": "2025-01-10T08:00:00Z"
}
],
"total_count": 15,
"limit": 50,
"offset": 0,
"has_more": false,
"query_interpretation": {
"original_query": "red cars from last week",
"filters_applied": ["color: red", "date: last 7 days"],
"search_type": "semantic"
}
}

Image Search

The search engine understands natural language including context, synonyms, and filters. Example queries: "red cars from last week", "images with dogs and cats", "photos of people outdoors".

Export

GET/api/v2/chat/sessions/{session_id}/export

Export chat session in various formats

Request

json
// Query parameters:
?format=markdown // json | markdown | text (default: markdown)
&include_metadata=false // Include metadata in export (default: false)

Response

json
// Returns file download with Content-Disposition header
// markdown format:
# Chat Session: Equipment Analysis
## User
What are the safety requirements?
## Assistant
Based on the documents, the key safety requirements are...
// json format:
{
"session_id": "550e8400-e29b-41d4-a716-446655440000",
"title": "Equipment Analysis",
"messages": [...],
"metadata": {...}
}