When Buffers Bite Back: Teaching AI to Respect Pallets in Flexible Job Shops

Factories rarely fail because a machine cannot work. They fail because the machine, the operator, the part, the fixture, the pallet, and the next free square meter of floor space refuse to arrive in the same universe at the same time.

That is why a scheduling paper about pallets is more interesting than it sounds.

The paper Learning Flexible Job Shop Scheduling under Limited Buffers and Material Kitting Constraints studies a variant of the flexible job-shop scheduling problem where jobs are steel plates, some operations produce parts that must be sorted, and those parts must temporarily sit on a limited number of pallets under material-kitting rules.¹ A pallet is not an infinite waiting room. It can hold only parts of one category. If a new job produces part categories that do not match the categories already sitting in the buffer, and there are not enough empty pallets, a pallet has to be moved out and replaced. That replacement takes time. Multiply the irritation across a long sequence of production decisions and the schedule begins to bleed.

The obvious scheduling problem is: which operation should run on which machine, and when?

The less obvious problem is: which operation should not run now, because it will create the wrong kind of waiting material in the wrong kind of buffer?

That second question is where this paper lives.

The scheduling problem changes when the buffer becomes part of the machine

Classical flexible job-shop scheduling already has enough pain to keep optimization researchers employed. A job consists of ordered operations. Each operation can be processed by one of several compatible machines. The scheduler must decide both the operation sequence and the machine assignment, usually minimizing makespan, the completion time of the last operation.

In the paper’s notation, the usual objective is:

where $C_{ij}$ is the completion time of operation $O_{ij}$.

That formulation treats waiting mostly as a timing consequence. A job waits because a machine is busy, or because its predecessor operation is not finished. The buffer is usually either simplified, assumed available, or treated as a secondary constraint.

The paper changes the object of attention. In the Flexible Job-Shop Scheduling Problem with Limited Buffers and Material Kitting, or FJSP-LB-MK, the buffer has a categorical state. A pallet can hold only one part category. Parts from different steel plates can share a pallet if they belong to the same category, but different categories cannot be mixed.

That makes the buffer not merely “capacity.” It is structured capacity.

When a newly sorted steel plate produces categories already represented in the buffer, the parts can be assigned to the corresponding pallets. When it produces new categories, empty pallets are needed. When the number of new categories exceeds the number of empty pallets, existing pallets must be moved to the warehouse and replaced. The paper formalizes the replacement delay as:

where $N_{\text{excess}}$ is the number of new categories exceeding available empty pallets, and $t_{\text{switch}}$ is the time required for one pallet replacement.

This small formula is the point. It says a schedule can look efficient from the machine perspective and still be foolish from the material-flow perspective. The scheduler may keep machines busy while quietly filling the shop floor with avoidable pallet changes. Very modern. Very “optimized.” Very expensive.

The real bottleneck is category compatibility, not just machine availability

A reader familiar with job-shop scheduling may assume the main difficulty is the size of the search space. That is true, but incomplete. The paper’s more useful correction is that some production bottlenecks are created by shared resources whose consequences appear later.

Suppose two operations are both ready. One has a slightly better machine completion time. The other produces parts that match categories already sitting in the buffer. A myopic scheduler may choose the first operation because the machine assignment looks locally better. But if that operation introduces new part categories and forces pallet replacement, the “fast” decision has just purchased a delay elsewhere.

This is the practical misconception the paper attacks: factory scheduling is not only about sequencing operations and assigning machines. In high-mix production, the buffer can become a memory of previous decisions. It remembers what categories are already on pallets. It punishes the scheduler for ignoring that memory.

The authors’ example is steel plate processing, but the pattern is broader. Any environment with limited intermediate storage, batching compatibility, kit preparation, fixture families, tote constraints, or category-specific handling can exhibit the same logic. The operational object is not just the job. It is the job plus the temporary state it imposes on shared infrastructure.

That is why simply adding another dispatching rule is unlikely to solve the deeper problem. The scheduler has to see the buffer state as part of the global state.

The paper gives the buffer a node and gives pallet changes a price tag

The proposed method uses deep reinforcement learning trained with Proximal Policy Optimization. At each decision step, the agent selects an operation-machine pair from eligible candidates. That is familiar enough.

The important part is not PPO itself. PPO is the engine. The steering system is the state representation.

The paper represents the scheduling state as a heterogeneous graph containing machines, operations, and a buffer node. Operation features include the scheduling features used by the DRL baseline the authors build on, plus three additions:

The buffer also has features: part-category information and occupancy rate. This means the model does not merely know that “there are six pallets.” It knows what is currently occupying the pallet system and how full it is.

Then comes the key architectural choice: the buffer node is connected only to part-sorting operations, not to every operation. This is selective connectivity.

That detail matters. A graph neural network aggregates messages from neighboring nodes. If the buffer broadcasts information to every operation, it can add noise to operations that do not care about pallet state. The model is then asked to learn relevance after the representation has already blurred relevance. The paper’s design is more disciplined: let the buffer talk only to operations that can actually trigger buffer consequences.

The second design choice is cost-sensitive propagation. The edge from the buffer to a part-sorting operation is weighted according to the estimated number of pallet changes for that operation:

with $\alpha = 0.3$ in the authors’ setting. The operation-buffer feature is then computed as:

where $B$ is the buffer feature. Operations with higher anticipated switch cost receive stronger buffer messages.

This is the mechanism-first core of the paper: the model is not merely “using a graph.” It is using graph structure to make a physical constraint visible at the exact decisions where that constraint bites.

The reward teaches the model to care about both completion time and pallet switching

The learning objective also reflects the dual nature of the problem. The reward includes both change in makespan and change in pallet changes:

Here, $P_{\max}$ represents total pallet changes, and $\lambda$ links the switch objective to the makespan objective.

This matters because the paper is not optimizing a pure logistics metric. It is not saying “minimize pallet switches at any cost.” It is trying to learn a schedule that remains compact while avoiding unnecessary buffer disruption.

That distinction becomes important in the real production-line results. The method usually improves makespan, but it does not always produce the lowest switch count on every real line. That is not necessarily a failure. In some lines, a slightly higher number of switches may accompany a shorter overall schedule because machine-operation structure, part categories, and pallet availability interact differently.

A factory manager will recognize the pattern. The KPI you optimize in isolation becomes a trap. The better question is which trade-off actually improves flow.

What the experiments are really testing

The experiments have three layers, and they should not be read as one big scoreboard.

The synthetic datasets range from 10 jobs × 5 machines to 40 jobs × 10 machines. The real datasets come from four steel-plate production lines. For each real line, the authors sampled 10,000 training instances, 100 validation instances, and 100 test instances, with each instance representing a production segment of 20 jobs. The real-line data also varies in machine-operation mappings, part-category counts, pallet replacement times, and available pallets.

The baselines are also worth reading carefully. The paper compares against priority dispatching rules, an adapted DRL method from prior FJSP work, and Google OR-Tools CP-SAT. For OR-Tools, the authors use part-sorting sequences obtained via MCTS as static input and impose a 1,800-second time limit per instance. In the tables, OR-Tools acts as a strong reference point, but not as a magical omniscient factory god descending from the cloud with infinite time.

The synthetic results show the advantage widens with scale

On synthetic instances, the proposed method beats priority dispatching rules and the adapted DRL baseline across all six problem sizes in makespan. It also consistently reduces pallet changes compared with the DRL baseline and the dispatching rules.

The most telling pattern is scale. In the 10 × 5 synthetic setting, the sampling version of the proposed method achieves a makespan of 158.25 versus 171.48 for the adapted DRL sampling baseline, with switches reduced from 12.01 to 9.65. In the largest 40 × 10 setting, the proposed sampling method reaches 564.89 versus 715.23 for the adapted DRL sampling baseline, with switches reduced from 62.73 to 31.47.

The exact numbers matter less than the direction of the widening gap. As the problem grows, a representation that ignores buffer dependency becomes increasingly expensive. That is exactly what we would expect if pallet state creates long-range consequences: the longer the decision chain, the more room there is for small local mistakes to accumulate into congestion.

The paper’s Figure 6 makes this visually intuitive. In a representative 10 × 5 test case, the adapted DRL baseline has 16 pallet changes and a makespan of 192, while the proposed method has 10 pallet changes and a makespan of 167. The white rectangles in the Gantt chart represent delay caused by pallet changes. In plain English: the baseline keeps tripping over the buffer. The proposed method trips less.

Progress.

The real-line results are stronger for makespan than for pure switch minimization

The real production-line results are more nuanced, and therefore more useful.

Across production lines A, B, C, and D, the proposed method achieves the best makespan among the learning and dispatching baselines. Under the sampling strategy, its makespans are:

Line C is especially interesting. The proposed sampling method slightly beats the OR-Tools reference on makespan, with 40359.80 versus 40377.03, while requiring 4.94 seconds rather than an OR-Tools run under the 1,800-second limit. This should not be misread as “DRL defeats exact optimization.” The OR-Tools setup uses static part-sorting sequences and a time budget; the paper itself treats OR-Tools as a reference, not a universal optimum certificate. Still, the result is operationally meaningful: learned scheduling can deliver high-quality solutions quickly in constrained instances.

The switch counts tell a subtler story. On real line C, the proposed greedy method has 25.11 switches, lower than the adapted DRL greedy baseline at 31.62 and lower than most dispatching rules. But on line A, the proposed sampling method has 16.42 switches, slightly more than the adapted DRL sampling baseline at 15.86 and MWR at 15.28, while still delivering better makespan than those methods. On line B, switch counts are all low and tightly clustered. On line D, the proposed sampling method has 19.42 switches, close to the adapted DRL sampling baseline at 19.27, while improving makespan.

This is not a contradiction. It is the factory reminding us that “fewer switches” and “shorter schedule” are related, not identical. The authors attribute this to heterogeneity across lines: total part categories, part categories per job, and available pallets vary substantially. Different buffer congestion regimes create different trade-offs.

For business readers, this is the part to keep. The model is not a pallet-switch minimizer with a scheduling costume. It is a scheduler that has learned to price buffer disruption as part of the scheduling decision.

The ablations show that the model needs the right constraint, not just more graph

Ablation studies are often where papers quietly reveal whether the proposed mechanism is doing real work or merely decorating a neural network with fashionable nouns.

Here, the ablations are useful.

The first ablation removes combinations of key state features on the 10 × 5 synthetic scale. The full model uses PS + Type + SwEst. Removing Type or SwEst hurts both makespan and switches substantially. The PS + SwEst variant has makespan 183.94 and switches 15.41, compared with 169.85 and 12.10 for the full model. The PS + Type variant is even worse on makespan at 187.73 and has 15.42 switches.

The Type + SwEst variant is much closer to the full model: makespan 172.41 and switches 12.08. That suggests the expensive information is not merely whether an operation is a part-sorting operation. The expensive information is what type of parts it produces and how many pallet switches it is likely to trigger.

The second ablation tests graph connectivity and cost-sensitive propagation:

This is a small table, but it carries a useful lesson. Connecting the buffer to all operations performs worse than connecting it selectively. Broadcasting operational context everywhere is not intelligence. It is noise with a GPU budget.

The weighted version performs best because it gives larger buffer messages to decisions with higher estimated switch cost. The inverse-weight version, which emphasizes lower immediate switch cost, is weaker. The authors interpret this as evidence that chasing low immediate switch cost can become short-sighted, while highlighting high-cost decisions helps the model avoid downstream bottlenecks.

That interpretation is plausible and business-relevant. In operations, the expensive decision is often the one that looks merely “available” right now.

What this means for manufacturing AI systems

The business relevance of the paper is not that every factory should immediately train a PPO scheduler. That would be the kind of advice usually produced by people whose closest encounter with manufacturing is unboxing a monitor.

The more useful takeaway is architectural: if a constraint creates delayed shared-resource consequences, the AI system must represent that constraint explicitly.

For a high-mix production environment, a deployment path would look roughly like this:

The paper directly shows that a learned scheduler with buffer-aware heterogeneous graph representation can improve makespan and reduce switch-related overhead on the tested synthetic and steel-plate datasets.

Cognaptus’ business inference is narrower but important: many failed “AI scheduling” projects probably fail not because the model is too weak, but because the state representation is too polite. It includes machine availability and job order, then quietly ignores the ugly physical details that actually cause delays: pallets, totes, racks, fixtures, staging lanes, operator zones, curing rooms, cold-chain windows, inspection queues, and all the other little kingdoms of constraint.

The uncertain part is implementation cost. To use this in a real plant, the company must have reliable data on part categories, operation-machine eligibility, buffer occupancy, and switch times. If that information lives in spreadsheets, operator memory, and the tragic archaeology of ERP comments, the first ROI comes from building the data layer, not from training the model.

Where the paper stops, and where factories begin

The authors are explicit that their real-line dataset idealizes the production environment by not accounting for machine failures or dynamic order insertions. That is a meaningful boundary. A factory schedule is not a chessboard that politely waits for the AI to finish thinking. Machines fail. Rush orders appear. Operators get reassigned. Quality checks block release. Forklifts become mythical creatures last seen near receiving.

The paper’s method is therefore best read as a strong offline or semi-online scheduling foundation under structured constraints, not as a complete autonomous factory control system.

There are three boundaries to keep in mind.

First, the real-line validation is valuable but domain-specific. Steel plate processing with material kitting is not the same as electronics assembly, pharmaceutical packaging, or warehouse wave planning. The mechanism may transfer, but the feature design and constraint model will need translation.

Second, the method relies on fine-tuning for real production lines. The authors train first on small synthetic instances, transfer to larger synthetic tests, then fine-tune on real-line datasets with KL regularization to avoid unstable policy updates. This is sensible. It also means deployment is not a one-click model download. Local adaptation matters.

Third, minimizing switches is not always the winning operational objective. The real-line results show that the proposed method can improve makespan without always having the lowest switch count. That is not a problem if the business objective is total flow efficiency. It is a problem only if the buyer was sold a simplistic “AI will reduce every KPI simultaneously” story. Those stories are convenient because they fit on a slide. Reality, inconsiderately, prefers trade-offs.

The real lesson: make the hidden constraint visible before asking AI to optimize it

This paper’s strongest contribution is not that it adds another DRL result to the scheduling literature. It formalizes a problem that many factories already know in their bones: the schedule is shaped by resources that standard models treat as background scenery.

Limited pallets are not scenery. Material-kitting rules are not scenery. Buffer occupancy is not scenery. They are state.

Once the buffer is represented as state, the rest of the design follows: connect it to the operations that matter, weight its influence by estimated switch cost, and train the policy to care about both completion time and pallet disruption. The empirical results then become more than a leaderboard. They become evidence for a design principle.

For manufacturing AI, that principle is simple:

Do not ask a model to respect a constraint it cannot see.

And do not be surprised when a schedule that looks elegant on a screen gets bitten by a pallet on the floor.

Cognaptus: Automate the Present, Incubate the Future.