What kinds of uncertainty matter?
Data uncertainty comes from ambiguous or noisy inputs. Model uncertainty comes from limited knowledge or weak coverage. The useful distinction is whether more data, a better model, or human judgment can reduce the risk. Methods include ensembles, repeated sampling, predictive distributions, and task-specific confidence signals.
How should uncertainty affect an application?
Connect the signal to confidence gating and human in the loop paths. Validate uncertainty through model calibration before assigning thresholds. High uncertainty may trigger retrieval, clarification, another model, or abstention. Choose the response by the cost of being wrong: a cheap, reversible task can simply retry, while an irreversible action should stop and route to a person with the model’s evidence attached. Include the resulting decisions in AI risk management, because an unused uncertainty score changes nothing about production behavior.
How do we turn uncertainty into a decision at Soba Labs?
With deterministic gates, not model self-reports. The systems we ship score each output, and plain code compares that score to a threshold and decides: proceed, or stop and route to a person. The gate is ordinary deterministic software, so the same input always gets the same decision, which makes the abstention behavior testable and auditable; the reasoning is in our piece on deterministic confidence gates. We set each threshold from observed errors on real cases rather than from the model’s stated confidence, and in domains where a wrong answer costs most, such as the health assistant in our Shen.AI case study, the gate errs toward handing the case to a person.