Smart Moves: How SmartPilot is Revolutionizing Manufacturing with a Multiagent CoPilot

TL;DR for operators

SmartPilot is not best understood as “ChatGPT for the factory floor.” That would be the lazy reading, and factories already have enough lazy dashboards with heroic colour palettes and no operational courage.

The paper proposes a compact, neurosymbolic, multiagent manufacturing copilot that joins three practical functions: anomaly prediction, production forecasting, and domain-specific question answering.¹ Its strongest idea is architectural. PredictX watches for anomalies using time-series and image data. ForeSight predicts near-term production using sequence models plus process-specific features. InfoGuide answers operator questions using manuals, retrieval, and real-time data. The system is then connected to live manufacturing infrastructure through OPC-UA and to domain knowledge through manufacturing ontologies.

The reported results are promising. PredictX reaches 93% accuracy and 93% weighted F1-score on the rocket-assembly anomaly task. ForeSight is reported to improve forecasting by an average of 21.51% over an LSTM baseline, although the improvement is uneven across product subtypes. InfoGuide scores 92.1% relevance, 88.6% factual accuracy, 4.7 out of 5 in a 10-person user-satisfaction study, and a 2.3-second average response time.

The business interpretation is straightforward but not magical. SmartPilot points toward a factory copilot that reduces the cost of diagnosis, improves short-horizon planning, and gives operators faster access to contextual knowledge. It does not prove that one model can run any plant. Its value depends on curated datasets, plant-specific manuals, reliable integration, and ontology work that someone still has to do. Annoying, yes. Also exactly where the serious value lives.

The factory problem is not missing AI; it is disconnected AI

Most factories do not suffer from a complete absence of analytics. They suffer from analytics that arrive in pieces.

One system detects anomalies. Another forecasts output. A third stores manuals. A fourth dashboard shows live sensor values. The operator is then expected to stitch together a coherent answer under time pressure, often while a machine is drifting, a batch is slipping, or a component has quietly failed to appear where physics and procurement promised it would.

That fragmentation is the real target of SmartPilot. The paper’s authors are not claiming that manufacturing has never seen predictive models, forecasting models, or Q&A assistants. They are arguing that these capabilities are usually too specialised, too disconnected, or too hard to interpret when deployed in real operational settings.

SmartPilot’s answer is a compact multiagent system. Each agent has a defined operational role, but the system is designed so that prediction, forecasting, and explanation can talk to each other. This is where the paper becomes more interesting than another “AI copilot” brochure. A chatbot bolted onto a dashboard can explain yesterday’s manual. A manufacturing copilot has to reason across live signals, historical production behaviour, machine context, and what the operator is actually trying to fix.

That distinction matters. In manufacturing, “helpful answer” is not a literary category. It is a workflow category.

SmartPilot works because the agents divide the factory problem cleanly

The system has three named agents: PredictX, ForeSight, and InfoGuide. Each one maps to a different operational question.

The design choice is sensible. Anomaly prediction, production forecasting, and operator Q&A are related, but they are not the same task. Treating them as one giant model would be elegant in a conference slide and unpleasant in a plant. SmartPilot instead separates the jobs while preserving inter-agent connectivity.

PredictX can feed anomaly-related insight into ForeSight. ForeSight can adapt forecasts to evolving production states. InfoGuide can draw on the outputs of PredictX and ForeSight when answering questions about the current line. The paper uses a fine-tuned DistilBERT model with LoRA to help connect InfoGuide to prediction and forecasting outputs.

That is the mechanism-first lesson: the copilot is not a single “brain.” It is a small federation of specialised models joined around the operator’s workflow. Less romantic, more deployable.

PredictX shows the strongest evidence for knowledge-infused learning

PredictX is the anomaly-prediction agent used in the rocket-assembly case. The task is to detect anomalies caused by missing components during assembly. The data comes from the Future Factories rocket assembly setup and includes time-series measurements plus synchronised images from two cameras.

The architecture combines several ingredients:

• a time-series autoencoder;
• a fine-tuned EfficientNet-B0 model for images;
• decision-level fusion;
• transfer learning;
• a custom loss function using sensor-range knowledge from a process ontology.

The paper’s ablation table is important because it is not just decorative benchmarking. It shows the likely contribution of each design element.

The key result is not merely “93% accuracy.” The stronger reading is that the best model combines three things factories often keep apart: sensor traces, visual evidence, and explicit process knowledge. The ontology is not used as a nice semantic label pasted on after the prediction. It contributes during training through a custom loss function that penalises predictions inconsistent with expected sensor ranges.

This is one reason the paper’s “neurosymbolic” label is not empty branding. The symbolic part is not pretending to replace the neural model. It constrains and explains it. Very civilised behaviour, for once.

The explanation layer matters too. For a predicted anomaly, the system can identify which variable is responsible, what the robots were doing in that state, and the expected values of that variable. That is closer to operator-grade explanation than model-grade explanation. The difference is not cosmetic. Operators do not need a lecture on latent representations. They need to know what is wrong, where to look, and whether the model’s claim makes sense in the process state.

ForeSight is useful, but the gains need a careful reading

ForeSight tackles production forecasting. In the rocket use case, it forecasts production around assembly schedules. In the Vegemite use case, it forecasts production for an industrial yeast evaporation process using variables such as raw yeast input quantities, evaporation ratios, flow rates, temperature, pressure settings, and product-quality measures.

Technically, ForeSight uses an LSTM-based architecture with two LSTM layers. The model then concatenates the temporal representation with structured process features before prediction. In plain English: it does not ask the sequence model to discover every relevant production relationship from time alone. It feeds the model contextual process features that manufacturing engineers already know matter.

The paper reports low MAE and RMSE values across the use cases:

The Yeast - FMX case has the largest errors. The authors attribute this likely to greater variability or noise in the underlying production data. That is exactly the kind of unevenness a business reader should notice. A model can be valuable while still being less reliable for noisier product streams. Real production data does not become obedient because someone wrote “Industry 4.0” near it.

The paper also says ForeSight shows an average 21.51% improvement over LSTM. That headline is useful, but the table behind it is more instructive than the average. The improvement is not uniform across all product subtypes. Toy rocket and Yeast - BRD show strong gains. Yeast - FMX also improves. Yeast - BRN is reported with a negative improvement in the ablation table.

For operators, that means ForeSight should be read as a planning-assist mechanism, not an oracle. The business value is in narrowing uncertainty, surfacing likely production deviations earlier, and supporting decisions around materials, labour, and scheduling. It is not a promise that every production subtype receives the same uplift.

That distinction is dull only if one has never had to explain a production miss to finance.

InfoGuide turns manuals into operational answers, but the guardrail is the important part

InfoGuide is the domain-specific Q&A agent. It processes manufacturing manuals and related documents, cleans and chunks the text, identifies operationally meaningful terms such as safety, maintenance, operation, installation, inspection, warning, danger, and caution, then uses summarisation and retrieval to support answers.

The retrieval design combines neural and symbolic methods. Neural retrieval uses BERT embeddings and cosine similarity. Symbolic retrieval uses keyword tokens and Jaccard similarity. If the similarity score is high enough, the retrieved context and an agent-specific prompt template are passed to Mixtral to generate an answer. If the score is below threshold, the system refrains from answering.

That refusal path is more important than it looks. In factory settings, a hallucinated answer is not merely embarrassing. It can be operationally expensive or unsafe. A useful manufacturing Q&A system must know when the manual, live context, or retrieved evidence is insufficient. “I do not have enough support to answer that” is not a weakness. It is one of the few sentences an industrial AI system should learn before it learns to sound clever.

The reported InfoGuide results are encouraging:

The small user-study size matters. Ten professionals can tell us whether the interface and answers are plausible in the demonstrated setting. They cannot settle broad usability across factories, shifts, languages, training levels, and safety regimes.

Still, InfoGuide addresses a real operational pain. Manufacturing knowledge is often scattered across manuals, senior technicians, undocumented habits, and the one person who somehow knows which valve behaves badly on humid afternoons. A retrieval-based assistant will not replace that tacit knowledge. But it can reduce the search cost for formal knowledge and connect that knowledge to live process states.

That is a defensible business value. Not glamorous. Useful.

OPC-UA integration makes this more than a lab notebook

A manufacturing copilot that cannot touch live systems is a report generator wearing a hard hat.

SmartPilot’s deployment design connects trained models to an OPC-UA server for sensor-data retrieval. Cameras provide image data. Manuals provide textual data. User interfaces built with HTML, CSS, JavaScript, and Streamlit expose the three agents through dashboards and chat-style interaction.

This integration layer is easy to underplay because it is not as fashionable as the model names. It is also where many AI pilots quietly die. A model trained on a clean dataset is one thing. A model that can ingest live sensor data, interact with operator interfaces, and return usable predictions in the rhythm of production is another.

SmartPilot is demonstrated in two environments: a rocket assembly facility in a full-function academic setup and a commercial Vegemite production line. That dual setting strengthens the paper’s practical story. The rocket assembly case gives a controlled environment for multimodal anomaly detection and ontology-based explanations. The Vegemite case gives a process-industry forecasting setting with real production variability.

The two cases also define the limits. This is not a proof of universal manufacturing generalisation. It is a proof that the architecture can be instantiated across two distinct production contexts when the necessary data, manuals, process knowledge, and integration paths exist.

The business value is modular decision support, not factory autopilot

SmartPilot’s business relevance is clearest when separated by decision type.

The practical promise is not that SmartPilot removes humans from manufacturing decisions. It gives humans a better operating surface. That is a more mature claim.

In many factories, the problem is not that people lack judgement. It is that judgement is forced to operate through scattered screens, stale documents, and delayed signals. A copilot that links prediction, forecast, and explanation can compress the time between “something feels off” and “here is the likely issue, expected range, relevant manual context, and production implication.”

That compression is the business case.

The ROI pathway would likely come through four channels:

1. Reduced diagnostic time. Operators and engineers spend less time identifying anomaly causes and relevant process context.
2. Better short-horizon planning. Forecasts support staffing, input-material preparation, and schedule adjustment.
3. Lower knowledge-access cost. Manuals become queryable in context rather than searchable as static PDFs.
4. Improved trust through explanation. Ontology-based explanations make model outputs easier to challenge, accept, or override.

None of those requires the system to be perfect. They require the system to be reliably better than the current bundle of dashboards, spreadsheet checks, manual searches, and hallway expertise. A modest bar in theory. A surprisingly high bar in practice.

The ontology is not a luxury feature; it is the price of seriousness

The paper’s most important design bet is knowledge infusion. SmartPilot uses process ontologies both to improve model training and to generate user-level explanations. In PredictX, sensor-range knowledge from the ontology is infused through a custom loss function. In the rocket assembly case, the Dynamic Process Ontology captures knowledge about sensors, cycle states, robots, and machinery.

This is not free. Ontologies require domain modelling. They require maintenance. They require plant-specific understanding. They are exactly the kind of work that gets cut when organisations want a fast AI demo and then rediscovered when the demo cannot explain why it is wrong.

The business reader should take the hint. The advantage of SmartPilot does not come from ignoring domain specificity. It comes from formalising enough domain specificity that the models can use it.

That has consequences for adoption. A company considering this kind of architecture should not start by asking, “Which LLM do we use?” It should start with more annoying questions:

• Which production states matter?
• Which sensor ranges are meaningful in each state?
• Which anomalies are operationally distinct?
• Which manuals are current, reliable, and machine-readable?
• Which live data streams can be connected through standard interfaces?
• Which explanations would operators actually trust?

The model is the visible part. The knowledge substrate is the serious part. Naturally, the serious part is less fun to put in a vendor deck.

The limitations are specific, not fatal

SmartPilot is credible as an architecture pattern, but the paper’s evidence has boundaries.

First, the evaluation uses two use cases. They are useful and distinct, but they do not establish generalisation across automotive, electronics, pharmaceuticals, metals, logistics-heavy assembly, or highly regulated batch manufacturing.

Second, the reported results depend on curated datasets and available domain knowledge. The rocket assembly use case benefits from multimodal data and a process ontology. The Vegemite case benefits from detailed process measurements and product-quality variables. Factories with poor instrumentation, messy data lineage, or outdated manuals will not receive these benefits by wishing politely.

Third, InfoGuide’s satisfaction result is based on 10 manufacturing professionals. That is enough for early signal. It is not enough for broad claims about workforce adoption, training burden, or long-term trust.

Fourth, the paper discusses operational value such as reducing downtime and production losses, but the reported quantitative results focus mainly on prediction accuracy, forecast error, relevance, factual accuracy, satisfaction, and response time. Actual economic metrics—downtime avoided, scrap reduced, labour hours saved, schedule variance reduced—are not yet demonstrated.

Fifth, the ForeSight results should be read with granularity. The average improvement over LSTM is promising, but performance varies across product subtypes. In production planning, variance is not a footnote. It is where the expensive surprises live.

These limitations do not undermine the paper. They keep it in the correct category: strong prototype and architecture evidence, not universal deployment proof.

What leaders should copy from SmartPilot

The most transferable lesson is the design pattern.

A serious manufacturing copilot should not be built as a single conversational layer sitting above disconnected systems. It should be organised around operational questions and connected to the data required to answer them.

That suggests a practical blueprint:

This is where SmartPilot becomes relevant beyond the specific rocket and Vegemite cases. It gives manufacturers a way to think about AI copilots as operational systems, not conversational accessories.

The uncomfortable implication is that buying a generic model will not be enough. The value comes from the fit between model architecture, process data, plant knowledge, and operator workflow. The generic model may be useful. The generic deployment fantasy is still fantasy, now with nicer typography.

Conclusion: the copilot is the architecture

SmartPilot’s main contribution is not that it predicts anomalies, forecasts production, or answers questions. Manufacturing AI systems have done versions of those things before.

Its contribution is the way it connects them.

PredictX gives anomaly prediction with multimodal and ontology-informed learning. ForeSight adds production forecasting with structured process features. InfoGuide turns manuals and live outputs into operator-facing answers. OPC-UA integration brings the system closer to real-time use. The result is a compact multiagent copilot that treats factory intelligence as a workflow problem rather than a chatbot problem.

For business leaders, the message is practical. If you want AI copilots in manufacturing, do not start with the chat interface. Start with the decision loop. Decide what must be predicted, what must be forecast, what must be explained, what live data is available, and what domain knowledge must constrain the model.

Then, and only then, give it a chat box.

Cognaptus: Automate the Present, Incubate the Future.