Skip to main content
Serve Model

Body

application/json
model
string
required
Minimum string length: 1
Pattern: ^[\w.\-/]+(:[\w.\-]+)?$
gpus
integer
default:0
Required range: x >= 0
name
string | null
Pattern: ^[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?$
backend
string | null
max_model_len
integer | null
Required range: 256 <= x <= 262144
gpu_memory_utilization
number | null
Required range: 0.1 <= x <= 0.95

Response

HTTP 201 response

id
string
required
name
string
required
status
enum<string>
required
Available options:
pending,
scheduled,
running,
paused,
completed,
failed,
cancelled,
retrying,
building,
degraded,
restarting,
stopped
created_at
string<date-time>
required
model
string | null
backend
string | null
desired_state
string | null
model_pulled
boolean
default:false
gpu_count
integer
default:0
gpu_ids
string[]
gpu_utilization
Gpu Utilization · object
endpoint
string | null
max_model_len
integer | null
gpu_memory_utilization
number | null