Rejected. That is where counterfactual explanations usually enter the story. A loan applicant is declined by an automated system. A hiring candidate is filtered out. An insurance customer is priced into an unfavorable category. The counterfactual explanation is supposed to answer a practical question: what would need to change for the model to give me the desired outcome? ...
A credit model approves the familiar applicant. A diagnostic model reads the common scan. A pricing model values the house in a neighbourhood it has seen a thousand times before. Everyone relaxes. The model is “confident”. Then a strange case arrives. The applicant has an unusual income pattern. The scan comes from an underrepresented patient group. The house sits outside the areas covered by historic transactions. The model still produces an answer, because that is what models are trained to do. Press button, receive number. Very efficient. Occasionally ridiculous. ...
A customer asks your AI assistant to “find me a better phone contract.” The agent browses comparison sites, selects a cheaper plan, authorizes the switch, cancels the old plan, and arranges payment of the cancellation fee from the user’s bank account. Lovely, in the way a self-driving forklift is lovely: impressive until it nudges the wrong shelf. ...
TL;DR for operators A model that says “I don’t know” is not automatically trustworthy. It may be cautious. It may be badly calibrated. It may be uncertain for the wrong reasons. It may also be using uncertainty as a very elegant trapdoor. Polite refusal, unfortunately, is still refusal. Stephan Rabanser’s thesis, Uncertainty-Driven Reliability: Selective Prediction and Trustworthy Deployment in Modern Machine Learning, is useful because it treats uncertainty not as a philosophical mood, but as an operational control layer.1 The key question is not whether a model can emit a confidence score. Most models can emit something confidence-shaped. The harder question is whether that score can decide which cases should be automated, deferred, reviewed, rejected, routed to a larger model, or audited. ...
TL;DR for operators An AI investment assistant does not enter every portfolio discussion as a blank analyst. The paper behind this article shows that large language models can carry latent investment preferences: for certain sectors, for larger companies, and for contrarian rather than momentum arguments.1 The important mechanism is simple and uncomfortable. When buy and sell evidence are balanced, the model’s internal prior can break the tie. When counter-evidence later becomes stronger, that prior does not necessarily disappear. In mixed-evidence settings, the model may latch onto the fragment of evidence that supports its original inclination and discount the stronger opposing side. Splendid. Your “neutral” analyst has discovered confirmation bias and brought it to the investment committee. ...