Back office teams rarely suffer from a shortage of opinions.
They suffer from a shortage of completed cases.
A bank wants to know whether a loan application will require costly rework. A hospital wants to know whether an emergency-department case will need laboratory processing. An operations manager wants to know how long a running case will take before it becomes tomorrow’s apology email. Predictive Process Monitoring, or PPM, is supposed to help with exactly this kind of question. It looks at event logs and predicts what will happen next: total completion time, future activities, process outcomes, delays, exceptions.
The awkward part is that classical PPM usually wants a decent data buffet. Thousands of traces. Stable processes. Clean attributes. Enough examples for rare but expensive events. In other words, precisely the conditions that many real organizations do not have when they most need prediction.
That is the useful discomfort behind Padella, de Leoni, and Dumas’s paper, “Exploring LLM Features in Predictive Process Monitoring for Small-Scale Event-Logs.”[^paper] The paper does not merely ask whether an LLM can be bolted onto process mining as a friendly chatbot. We have enough dashboards with conversational frosting, thank you. It asks a sharper question: when only 100 completed traces are available, can an LLM use those traces, the sequence of events, and the meaning of activity names to make competitive predictions?
The answer is yes, but the interesting part is not the yes.
The interesting part is why.
The paper’s practical lesson is not “LLMs beat process models.” That is too lazy, and also too broad. The better lesson is that LLMs can combine three mechanisms that classical sparse-data models struggle to combine at the same time:
- They read event traces as ordered, time-aware sequences rather than only as flattened vectors.
- They exploit the semantic meaning of activity and attribute names.
- They appear to mix multiple simple reasoning strategies without collapsing into any one of them.
That combination matters because business processes are not just numbers wearing column names. They are sequences of meaningful actions. Sometimes the column name is doing more work than the spreadsheet wants to admit.
The paper studies cold-start prediction, not chatbot decoration
Predictive Process Monitoring starts from event logs. A trace represents one process instance: a loan application, an account closure, a hospital visit, a support ticket, a compliance review. Each trace is a sequence of events, and each event typically contains an activity name, timestamps, and attributes.
The paper evaluates two prediction targets:
| Prediction target | Type of task | Business meaning |
|---|---|---|
| Total Time | Regression | How long the ongoing case will take to complete |
| Activity Occurrence | Classification | Whether a costly or important activity will occur later |
The authors test three real-life event logs:
| Use case | Process context | Completed traces | Activities | Global attributes | Target activity for occurrence prediction |
|---|---|---|---|---|---|
| Bpi12 | Loan application process | 8,616 | 6 | 1 | W_Nabellen incomplete dossiers |
| Bac | Bank account closure process | 32,429 | 15 | 2 | Service closure request with back-office responsibility |
| Hospital | Emergency department process | 37,945 | 46 | 3 | LABORATORIO |
The experimental setup is intentionally harsh. The LLM receives only 100 traces as examples. Relative to the available training logs, that is tiny: 1.45% for Bpi12, 0.38% for Bac, and 0.32% for Hospital. In normal enterprise analytics language, this is the point where someone says “let’s wait six months for more data” and everyone pretends that was a strategy.
The model used in the experiments is Gemini 2.5 Flash Thinking. The authors are careful to use “training” in a broad sense: the LLM is not fine-tuned on these traces. It receives traces in the prompt as context. The benchmark models, by contrast, are trained conventionally.
That distinction matters. The paper is not showing that 100 traces are magically enough to train a universal process predictor from scratch. It is showing that a pretrained LLM can use a small event-log sample as context and combine it with prior knowledge.
So the question becomes: what exactly does the LLM see?
Mechanism 1: traces become readable sequences, not flattened counters
A conventional model needs an encoding function. It cannot consume a business process directly; it needs a representation. Some encodings count which activities appeared. Some retain prefixes. Some aggregate history. Some preserve sequence. Every encoding chooses what the model is allowed to notice.
The paper’s LLM-oriented encoding turns each trace into a textual structure containing three main elements:
| Encoding component | Example role | Why it matters |
|---|---|---|
| Global attributes | Requested amount, closure reason, triage color, patient age | Gives the model case-level context |
| Activity-time sequence | Ordered activity and duration pairs | Preserves the path already taken by the running case |
| KPI value for completed examples | Total time or activity occurrence label | Lets the model infer task-specific patterns from examples |
For a running case, the prompt includes the partial trace and marks it as still ongoing. The model is asked to predict the KPI and to produce reasoning in a structured output format.
This is not just “put the CSV into a prompt.” The design is doing several things at once. It keeps the sequence legible. It keeps selected global attributes. It excludes local attributes, partly because long prompts dilute attention and partly because prior PPM work suggests global attributes often carry stronger predictive signal than local ones. The authors also include domain-specific background information where available.
That makes the prompt a miniature process-analysis environment: examples, task definition, encoding description, running trace, and expected output.
The business translation is simple. A classical model asks, “What numerical representation can I learn from?” The LLM prompt asks, “What process story can I reason over?” That sounds softer, but in sparse-data settings it can be harder-edged than it looks. When data is limited, preserving meaning and order may be more valuable than compressing everything into an elegant but hungry representation.
Elegant models are lovely. Starving models are less lovely.
The cold-start results are strongest when compared against 100-trace baselines
The main evidence is the 100-trace comparison. For Total Time prediction, the authors report Mean Absolute Error, in minutes. Lower is better.
| Use case | CatBoost, 100 traces | PGTNet, 100 traces | LLM, 100 traces, non-hashed | Interpretation |
|---|---|---|---|---|
| Bpi12 | 9,394 ± 1,275 | 8,856 ± 576 | 6,508 ± 235 | LLM is substantially lower-error in the scarce-data setting |
| Bac | 6,393 ± 365 | 4,753 ± 472 | 2,265 ± 1,072 | LLM is much stronger, though with high variance |
| Hospital | 259 ± 52 | 132 ± 32 | 115 ± 34 | LLM narrowly improves on the strongest 100-trace benchmark |
For Activity Occurrence prediction, the authors report F1-score. Higher is better.
| Use case | CatBoost, all training data | CatBoost, 100 traces | LLM, 100 traces, non-hashed | Interpretation |
|---|---|---|---|---|
| Bpi12 | 0.80 | 0.72 ± 0.07 | 0.77 ± 0.06 | LLM beats the 100-trace classifier but not the full-data classifier |
| Bac | 0.95 | 0.78 ± 0.08 | 0.98 ± 0.04 | LLM beats both the 100-trace and full-data CatBoost results |
| Hospital | 0.90 | 0.90 ± 0.01 | 0.90 ± 0.08 | LLM matches the benchmark |
This is where interpretation needs discipline. The paper’s strongest claim is about low-data prediction. With only 100 traces, the LLM is competitive or superior across both KPIs and all three logs. That is operationally important.
It does not mean data no longer matters. For Total Time prediction, the full-data PGTNet result remains better than the 100-trace LLM in all three use cases: Bpi12, Bac, and Hospital. The more precise conclusion is therefore not “LLMs defeat trained process models.” The more precise conclusion is:
LLMs can dramatically lower the minimum data needed to get useful PPM predictions, especially before specialized models have enough examples to behave properly.
That is a colder claim. It is also a more useful one.
In enterprise settings, the cold-start period is often when prediction has the highest marginal value. New workflows, newly digitized processes, exceptional cases, changing regulations, newly merged systems, new product operations—these are exactly the places where a full historical dataset may not yet exist. A method that works before the data warehouse grows up is not a replacement for mature analytics. It is a bridge over the ugly early stage.
Mechanism 2: semantic hashing shows the model is reading the labels, not just the rows
The paper’s cleanest mechanism test is semantic hashing.
The authors anonymize context-sensitive strings—activity names, categorical values, and global attribute names—into deterministic four-character identifiers. This preserves correlations. The same activity still maps to the same anonymous code. The sequence remains. The distribution remains. The time information remains.
What disappears is meaning.
LABORATORIO no longer tells the model “laboratory.” Triage_Color no longer suggests emergency-care priority. A loan follow-up activity no longer sounds like rework. The process still has structure, but the room has been stripped of signs.
This is an ablation test. Its purpose is not to create a deployable anonymization method. Its purpose is to separate two possible explanations:
| Possible explanation | What hashing tests |
|---|---|
| The LLM is only using correlations among example traces | Hashing should not hurt too much |
| The LLM is using semantic prior knowledge from labels and attributes | Hashing should degrade performance |
The degradation is clear.
For Total Time prediction:
| Use case | LLM, non-hashed | LLM, hashed | Effect |
|---|---|---|---|
| Bpi12 | 6,508 ± 235 | 9,246 ± 873 | Error rises by about 42% |
| Bac | 2,265 ± 1,072 | 3,880 ± 3,254 | Error rises substantially |
| Hospital | 115 ± 34 | 2,077 ± 232 | Error explodes by about 1700% |
For Activity Occurrence prediction, the drops are smaller but still consistent:
| Use case | LLM, non-hashed F1 | LLM, hashed F1 | Effect |
|---|---|---|---|
| Bpi12 | 0.77 ± 0.06 | 0.74 ± 0.06 | Moderate decline |
| Bac | 0.98 ± 0.04 | 0.96 ± 0.04 | Small decline |
| Hospital | 0.90 ± 0.08 | 0.84 ± 0.06 | Larger decline |
The authors apply Nemenyi post-hoc tests and report significant differences between hashed and non-hashed prompts across the tested tasks and use cases. The practical reading is not subtle: semantics are doing real predictive work.
This is the paper’s most business-relevant technical result.
Organizations often treat naming conventions as clerical hygiene. In this setup, names become part of the predictive substrate. A label like LABORATORIO, a field like Triage_Color, or an activity meaning “follow up incomplete dossier” carries domain priors that an LLM can use when few examples exist.
That does not mean firms should write poetic activity names. Please do not rename Invoice Review to The Sacred Accounting Rite. It does mean that process taxonomies, activity labels, and attribute definitions become part of model performance. In an LLM-based PPM pipeline, documentation quality is not decoration. It is signal.
Mechanism 3: the LLM seems to combine simple predictors without being reducible to them
The third research question asks whether the LLM is secretly using an ensemble of simpler prediction models. The authors approach this by inspecting the model’s reasoning explanations.
They manually analyze 50 LLM reasoning traces per use case and KPI, giving 150 reasoning traces per KPI. From these explanations, they derive recurring patterns and formalize them as $\beta$-learners: reproducible predictors that imitate the strategies the LLM appears to describe.
For Total Time, the extracted $\beta$-learners include:
| $\beta$-learner family | What it imitates |
|---|---|
knn act with mean, median, or mode |
Nearest-neighbor reasoning over activity-based representations |
knn att with mean, median, or mode |
Nearest-neighbor reasoning over attribute-based representations |
time seq with mean, median, or mode |
Temporal-sequence aggregation |
path pred with mean, median, or mode |
Predicting possible future paths and aggregating outcomes |
For Activity Occurrence, the extracted patterns include:
| $\beta$-learner family | What it imitates |
|---|---|
| Activity-Based | Reasoning over the activity sequence |
| State-Based | Reasoning from the latest state/event |
| Attribute-Based | Reasoning from trace attribute values |
| Positive Evidence | Checking whether the target activity has already appeared |
This part of the paper is not just an explainability ornament. It is a mechanism probe. If one of these $\beta$-learners could match the LLM, the LLM’s impressive result might reduce to a simple heuristic wrapped in fluent prose. That would still be useful, but less interesting.
The results go the other way. Across Total Time experiments, the LLM consistently outperforms the reimplemented $\beta$-learners. For Activity Occurrence, the same general pattern holds. In Bpi12, some differences are less dramatic, but the LLM remains above the individual learner families.
The key implication is that the LLM is not merely choosing one reusable rule, such as nearest neighbor over activity sequence. It appears to adapt among multiple reasoning templates and combine evidence differently depending on the case.
That matters because business processes do not fail in one style. Sometimes the latest state matters most. Sometimes the path matters. Sometimes the attribute matters. Sometimes the fact that a costly activity has already appeared is more informative than the activity count. A rigid predictor must choose its representation in advance. The LLM can, at least in these experiments, behave more like an analyst switching lenses.
Of course, “appears to” matters here. The $\beta$-learners are derived from the model’s explanations, and explanations are not guaranteed to be faithful internal traces. The paper partially addresses this by reimplementing those explanation patterns and comparing performance. That is a useful test. It does not open the model’s head. It shows that the visible heuristics are insufficient to explain the whole result.
In consulting language: the intern did not simply copy one spreadsheet formula. Annoying, but informative.
The appendix-style checks support the mechanism, not a second thesis
Several parts of the experimental design are best read as support tests rather than as separate headline claims.
| Test or component | Likely purpose | What it supports | What it does not prove |
|---|---|---|---|
| 100-trace comparison against CatBoost and PGTNet | Main evidence for RQ1 | LLMs can be competitive in scarce-data PPM | LLMs always beat full-data specialized models |
| Semantic hashing | Ablation for RQ2 | Activity and attribute meanings materially support prediction | Hashing is a recommended production privacy method |
| $\beta$-learner reimplementation | Mechanism test for RQ3 | LLM reasoning is not reducible to the extracted simple heuristics | LLM explanations are perfectly faithful |
| Wilcoxon tests for $\beta$-learners | Statistical comparison | Individual $\beta$-learners generally underperform the LLM | The exact internal algorithm of the LLM is known |
| Good-Turing smoothing for novel $\beta$-learners | Coverage validation | The sampled explanations likely captured the major visible reasoning families | No unseen reasoning pattern can ever appear |
| KPI convergence figure | Sampling sanity check | Small samples can approximate relevant KPI distributions in the experiment | 50 or 100 traces are always enough in every organization |
The Good-Turing result is especially easy to overread. The paper reports very low expected discovery rates for new $\beta$-learners: for 100 new prediction instances, 0.003 for Total Time and 0.014 for Activity Occurrence. This supports the idea that the manual analysis captured the main visible reasoning families. It does not mean the LLM has a tiny, fully known internal reasoning space. It means that, under this experimental setup and explanation-analysis method, the observed pattern catalog was probably not missing many recurring visible strategies.
That is still useful. It turns “the LLM reasoned somehow” into a more auditable claim: it used recognizable families of process reasoning, and those families alone did not reproduce the full model performance.