The LLM Is Not the Converter

TL;DR for operators

An engineering team migrating behavior logic between control architectures has two separate problems: understanding what an existing specification means, and reproducing that behavior without losing a branch, loop, transition, or termination condition. Automating the first problem is relatively easy to imagine. Trusting automation with the second is harder because a plausible-looking conversion can still change executable behavior.

The paper’s result is striking because the underlying language model is not sufficient. Zhang et al.’s LLMDUCF framework¹ reaches 90.65%-98.35% first-pass accuracy across its tested LLMs and scenarios, while corresponding models without the structured conversion procedure remain around 29%-39%. Yet deterministic rule-based converters still achieve 100% conversion accuracy in the benchmark.

For operators, that changes the purchase decision. LLMDUCF is not an argument for replacing a stable deterministic converter with an LLM. It is an argument for wrapping an LLM in explicit representations, decomposition rules, and verification when incoming behavior specifications vary enough that maintaining deterministic conversion logic becomes expensive.

Behavior migration is a semantic problem, not a syntax problem

A finite state machine organizes behavior as states connected by event-driven transitions. A behavior tree organizes actions and decisions hierarchically through control nodes. Both can represent autonomous-system logic, but moving between them requires more than renaming elements.

A loop in a state machine, for example, has execution meaning. So does a selector or sequence in a behavior tree. A converter therefore has to preserve what the system can reach, what it can terminate, and what it does under each supported transition—not merely reproduce similar-looking structure.

LLMDUCF treats that task as constrained semantic reasoning. The LLM handles flexible interpretation of the source representation, but its possible transformations are narrowed by explicit conversion procedures and domain-specific structural mappings.

That distinction explains most of the benchmark.

The procedure contributes more than the underlying LLM

The main experiments cover 1,200 source behavioral models across symbolic, robotic-manipulation, and game-AI scenarios. Three LLM families are tested in both conversion directions.

For behavior-tree-to-FSM conversion, LLMDUCF records first-pass accuracy between 90.65% and 97.80% across reported model-scenario combinations. The corresponding unconstrained LLM variants sit between 28.70% and 32.80%.

For FSM-to-behavior-tree conversion, LLMDUCF reaches 94.05%-98.35%, while the unconstrained versions range from 31.20% to 38.70%.

The relevant comparison is therefore not “LLM versus no LLM.” It is the same broad model capability placed inside two different operating systems for conversion.

First-pass accuracy is also a demanding metric here: a conversion counts only when the initial generated model passes both structural checking and formal behavioral verification, before regeneration.

Two structural mechanisms do most of the work

The ablations make it difficult to attribute the result to more elaborate prompting alone.

In behavior-tree-to-FSM conversion, the paper first compresses redundant nesting of same-type control nodes. It then divides the normalized tree into hierarchical conversion units and applies differentiated conversion rules to them.

Depth compression by itself moves ChatGPT accuracy only modestly, from roughly 29%-32% to about 33%-37% across the three scenarios. The hierarchical conversion-rule module alone raises it to 83.40%-86.85%. Combining both produces 93.65%-96.95%.

That pattern suggests that decomposition is doing substantive representational work. Compression helps, but the larger gain comes from deciding which parts of the tree should become distinct conversion units and how their local execution semantics map into the resulting FSM.

The opposite conversion direction is even more interaction-dependent. LLMDUCF first recognizes entry, loop, chain, and branch structures in the FSM, then maps them into a proposed loop execution behavior tree, or LEBT. Either component alone leaves the reported ChatGPT accuracy below 50%. Together they raise it to 95.50%-98.35%.

The system succeeds because recognition and representation are coupled. Identifying a loop is insufficient unless the target representation has an explicit way to preserve what that loop does.

Verification is part of the converter

Higher generation accuracy does not remove the need for checking. LLMDUCF first uses an LLM-based structural verifier and then applies NuSMV formal verification.

Formal verification here means machine-checking required behavioral properties. The reported checks include reachability, completeness, absence of additional behavior, deadlock freedom, and termination consistency.

In the demonstrated robotic BT-to-FSM case and predator-prey FSM-to-BT case, LLMDUCF and the deterministic converters satisfy all evaluated criteria. The unconstrained LLM outputs fail several of them.

This matters operationally because the paper also documents hallucination-related failure modes, including duplicated structures and incorrect identification of conversion units. Failed first attempts require an average of 1.56 to 2.28 retries across the reported configurations. Feedback-guided regeneration corrects the evaluated failures to eventual 100% accuracy, but that figure should not be confused with the lower first-pass result.

The verification loop is therefore not cleanup added after generation. It is one of the mechanisms that makes an LLM usable for this kind of engineering transformation.

Deterministic converters still win the raw-accuracy contest

The benchmark does not show LLMDUCF outperforming traditional converters on raw accuracy. BSD and CHDS-BT reach 100% in the tested scenarios.

Their weakness appears elsewhere. The reported input-compatibility comparison shows deterministic methods directly supporting the predefined text representation, while the LLM-based approaches handle text, images, tables, unseen encodings, and new conventions in the evaluation.

The paper also reports lower graphical complexity for LLMDUCF-generated structures in several comparisons, along with smaller edit distances for some subsequent behavior additions. Ten behavioral-modeling engineers report average cognitive-load scores of 1.8 ± 0.3 for LLMDUCF-generated models versus 4.3 ± 0.6 for traditional methods, with comprehension time reduced by 42%.

Those results support a different value proposition: flexibility and maintainability rather than maximum deterministic accuracy.

The business case depends on how often the specification changes

What the paper directly shows: within its tested model classes, a constrained LLM conversion system can approach deterministic conversion reliability while accepting a broader range of input representations and often producing structurally simpler outputs.

Cognaptus inference: the strongest adoption case is not a high-volume conversion pipeline with a fixed schema. If the source format, conventions, and behavioral semantics rarely change, a deterministic converter’s 100% benchmark accuracy and predictable execution remain attractive.

The economics shift when conversion is infrequent, specifications come from heterogeneous legacy sources, or conventions change faster than teams want to maintain dedicated conversion code. In that setting, the LLM’s representation flexibility can reduce preprocessing and rule-engineering work, while formal verification supplies a machine-checkable gate before the converted logic is accepted.

The formal scope is narrower than arbitrary behavior translation

The framework assumes source models that are complete and semantically coherent. Ambiguous or logically inconsistent inputs must be resolved before formal conversion.

Its FSM-to-BT equivalence argument applies to fully defined deterministic finite-state systems with unique successors for supported state-event pairs. The BT-to-FSM treatment is scoped to normalized trees containing sequence, selector, and leaf nodes. Parallel nodes, decorators, more complex cross-tick memory, hierarchical FSMs, and other advanced variants require additional mappings or templates.

There is also a deployment boundary. The main results come from three designed evaluation scenarios rather than broad production use. A local Qwen-35B deployment on an RTX 3060 with 16 GB RAM reaches only 80% FSM-to-LEBT and 85% BT-to-FSM accuracy on 20 representative models, materially below the cloud-model benchmark.

Choose the conversion architecture before choosing the model

The most transferable lesson from LLMDUCF is not that language models have become reliable behavioral-model compilers.

It is that a language model can become one component of a reliable converter when the surrounding system determines what structures to recognize, how to decompose them, what target representations are legal, and how behavioral equivalence will be checked.

For teams with stable formats, deterministic conversion remains difficult to displace. For teams whose migration problem is dominated by heterogeneous specifications, changing conventions, and one-off engineering work, a constrained LLM plus formal verification offers a different trade-off: slightly lower first-pass certainty in exchange for substantially greater representational flexibility.

The model supplies interpretation. The converter architecture supplies discipline.

Cognaptus: Automate the Present, Incubate the Future.