The Cost of Following the Herd

TL;DR for operators

A recourse recommendation must do more than identify a profile that an automated decision system would approve. It should describe changes a person can realistically make. In Follow the Herd: A Data-Driven Approach for Actionable Recourse, Nishtha Madaan and Srikanta Bedathur propose Data-Driven Action Recourse (DDAR), which learns likely changes from historical trajectories instead of requiring manually specified action costs during recommendation generation.¹

DDAR substantially outperforms four counterfactual-generation baselines on three synthetic or semi-synthetic benchmarks, including independently defined evaluation costs. Longer action sequences and multiple sampled candidates improve success further.

The operational opportunity is to generate more plausible sequences of recommended actions without pricing every possible change in advance. The unresolved issue is what historical frequency actually measures. A common transition may reflect unequal access or institutional constraints rather than low effort. Organizations considering this approach still need independent evidence of affordability, permissible actions, subgroup fairness, and real-world outcomes.

An approved profile is not necessarily an achievable one

A rejected loan applicant asks what they could change to qualify. An automated explanation might identify a different income, occupation, education level, or account balance that would receive approval. But the applicant needs to know which changes are possible, how long they would take, and what they would cost.

Conventional recourse systems commonly optimize the distance between the existing and desired profiles, sometimes using expert-defined costs for changing particular attributes. Those specifications require domain knowledge and can miss relationships between attributes. A small numerical change need not correspond to a practical action.

Historical applicant records offer another source of information. They show how people have moved between circumstances, rather than merely which hypothetical profiles receive favorable decisions.

DDAR formalizes that idea by treating the probability of a transition between two user states as a proxy for its cost. The more frequently a transition occurs among historically similar individuals, the cheaper the method assumes it is.

That substitution removes one modeling burden while introducing a consequential assumption: observed behavior must provide a reasonable ordering of how difficult different actions are.

The cost function is replaced by a learned probability

In conventional actionable recourse, the objective is to find a future state that achieves the desired decision while minimizing the cost of moving there.

DDAR proposes:

Here, $x_{\mathrm{prev}}$ represents the person’s current circumstances, and $x$ is a proposed future state. The conditional probability estimates how likely the transition is under learned historical behavior.

Taking its negative logarithm assigns lower surrogate costs to more probable transitions. DDAR consequently seeks a candidate state with high conditional likelihood while satisfying the desired classifier outcome, $f(x)=y^\ast$.

The critical distinction is between statistical accessibility and personal affordability.

A transition might be uncommon because a population faces unequal educational opportunities, regulatory barriers, or restricted access to credit. Another transition might be rare in historical records yet affordable for an individual with different resources.

The learned probability describes historical regularities. It does not directly measure money, time, effort, preferences, or the causal consequences of taking an action.

Guided diffusion turns likely transitions into recommendations

DDAR implements the probability model with conditional denoising diffusion.

The system encodes categorical and quantized numerical attributes, then trains a transformer-based denoiser to reconstruct future states conditioned on current states. During generation, gradients from the decision classifier steer candidate states toward the favorable label.

The resulting process combines two objectives: generate a state consistent with learned transition dynamics and make that state attractive to the decision model.

The implementation uses a 12-layer transformer with 12 attention heads and 2,000 diffusion steps. These are reported implementation choices, not independently demonstrated sources of the performance improvement.

DDAR can also generate successive future states, producing a multi-step trajectory rather than a single recommended endpoint. For a loan applicant, that makes it possible to represent changes across periods instead of demanding an immediate jump to an approved profile.

However, classifier guidance optimizes a predicted decision. It does not establish that following the recommended trajectory would cause the corresponding institution to grant approval.

Benchmark gains are substantial, but their costs are defined externally

The study compares DDAR with Wachter, DiCE, FACE, and Structured Counterfactual Diffusion (SCD) across three environments: synthetic loan trajectories (LoanSynth), synthetic graduate-admissions trajectories (GradSynth), and AdultIncome++, which adds simulated transitions to the UCI Adult Income data.

The principal evaluation asks how often recommendations achieve a favorable classifier outcome while staying within a specified maximum action cost. Its RR-AUC metric summarizes recourse success across cost allowances. Higher RR-AUC and lower average evaluation cost are preferable.

Source: Experiments, Table 4. Baseline winners are identified separately for RR-AUC and cost; they need not be the same method. Comparisons are within each benchmark, not across datasets.

The strongest baseline RR-AUC scores come from SCD for LoanSynth and AdultIncome++, and DiCE for GradSynth. DDAR exceeds each by a substantial margin. It also produces lower average costs than every comparison method under the benchmarks’ expert-defined evaluation functions.

This evaluation design is especially revealing. DDAR does not receive the expert cost functions during generation, but those functions are still used to judge its recommendations. The study therefore tests whether learned transition probabilities can serve as a substitute for manually supplied costs, rather than eliminating the need for independent cost judgments.

A second main evaluation tests statistical plausibility. Table 5 reports DDAR conditional and unconditional negative log-likelihood pairs of 29.26/21.27 on LoanSynth, 16.42/17.56 on GradSynth, and 40.17/19.50 on AdultIncome++. All are lower than the corresponding baseline results.

Conditional likelihood evaluates whether a proposed transition fits its starting state; unconditional likelihood evaluates whether the generated state is plausible more generally. Neither measures whether a person can afford to make the change.

A qualitative LoanSynth example adds another observation: DDAR preserves race and sex in the displayed recommendation, while SCD changes race and FACE changes both. This illustrates a difference in generated examples, not a universal guarantee that immutable attributes will remain unchanged.

More steps and more candidates improve the measured success rate

The paper separately investigates two extensions to the basic generation procedure.

The first varies the number of recourse steps from one to seven. At seven steps, DDAR reports a recourse rate of 0.90 on LoanSynth under allowable cost 8, and 0.80 on GradSynth under allowable cost 100. Longer trajectories can expose favorable paths that a single transition misses.

The second extension samples multiple trajectories and selects favorable candidates. In a reported LoanSynth best-of-N experiment, DDAR reaches a recourse rate of 1.0 at allowable cost 2.0, compared with approximately 0.12 for the next-best baseline.

These are planning-horizon and candidate-selection comparisons, not comprehensive architectural ablations. They demonstrate how additional search can improve benchmark coverage without proving that the learned transition probabilities become more accurate.

For a production recourse service, a longer recommended trajectory also introduces elapsed time, intermediate dependencies, and the possibility that institutional rules change before the final state is reached. More samples additionally require generation and evaluation resources. The paper does not provide a comparative inference-cost analysis sufficient to price that tradeoff.

Deployment requires a separate test of what people can actually do

For a lending platform with representative longitudinal applicant records, DDAR suggests an architecture for producing candidate improvement paths without manually defining the cost of every attribute change. An educational admissions service could explore analogous recommendations across successive applications.

That is a Cognaptus operational inference, not a demonstrated application outcome.

Before presenting such recommendations to customers, an organization would need to validate several distinct propositions: that suggested changes are permissible, that individuals can complete them, that their actual costs are acceptable, and that completing them improves the real-world decision.

A useful evaluation would track predicted approval, action completion, observed effort and expense, and eventual institutional outcomes separately. It would also examine whether recommendation availability or cost differs across population subgroups.

These checks are particularly necessary because learning historical transitions can reproduce historical restrictions. A model may faithfully estimate what people typically achieve while underestimating what less-represented individuals could achieve with appropriate opportunities.

The research does not test real completed recourse interventions, recommendation acceptability, or systematic subgroup fairness. It also lacks comprehensive robustness analysis under shifting populations and institutional policies.

Statistical assessment has further boundaries: dataset sample counts and detailed splits are insufficiently reported, and the main benchmark tables do not provide uncertainty estimates. Some numerical statements in the paper’s results prose also conflict with Table 4; the figures reported here follow the explicit table values.

These gaps do not erase the benchmark improvements. They limit how confidently an organization can translate those improvements into deployment decisions.

A promising generator, not a validated measure of effort

DDAR makes a meaningful methodological contribution by replacing hand-crafted action-cost optimization with learned conditional transition dynamics. Its results show that this approach can generate favorable, statistically plausible recommendations at substantially lower expert-defined benchmark costs than the tested alternatives.

Its central assumption remains open to independent validation. Historical transition probability may help rank candidate actions, but frequency alone cannot determine whether an action is affordable, permissible, or causally effective for a particular person.

For organizations developing recourse systems, the next practical step is to evaluate learned transition likelihood against observed action costs and completed outcomes. Until those relationships are established, DDAR is best understood as a candidate-generation method that can reduce manual specification, not a complete authority on which actions people should take.

Cognaptus: Automate the Present, Incubate the Future.