Inference

The Inference surface connects to every registered model in the Model Library and gives you a form to test a single image or text query.

Predictors and execution

• Lazily loaded models — predictors load on demand, so memory stays free until a prediction is actually requested.
• Task-aware settings — inference parameters adapt to the selected model. Classification models hide threshold sliders; detection and segmentation tasks expose IoU and confidence thresholds. YOLO segmentation defaults to a confidence of 0.65 and an inference IoU of 0.7.
• Inline results — normalized bounding boxes, polygons, and labels come back immediately, with the overlay drawn over the input.

One normalized response

Inference output is normalized across model families. A YOLO segmentation result and a Keras U-Net result arrive in the same shape, so the review surface keeps its layout when the model underneath changes.

History and logging

Every prediction is logged to the project's history.

• Historical payloads are stored alongside execution metrics — duration_ms and per-step timings.
• History can be filtered by project, or cleared to purge the database rows and the related overlay artifacts together.

Interactive LLM inference

Vision and NLP tasks run here as single-shot predictions. Language models work differently: pick the llm_finetune task to serve a fine-tuned model and open an interactive chat instead of a one-off form. See LLM Fine-tuning & Chat.