ESG in the Age of AI: When Reports Stop Being Read and Start Being Parsed

Reports are meant to be read.

ESG reports, unfortunately, are often meant to be admired, navigated, skimmed, quoted, selectively screenshotted, and occasionally endured. They arrive as glossy PDFs full of charts, tables, diagrams, narrative claims, compliance language, decorative layout choices, and headings that may or may not behave like headings. The result is a familiar corporate ritual: a firm publishes hundreds of pages of sustainability disclosure, investors and regulators ask what it means, and everyone quietly discovers that the document is more presentation object than data infrastructure.

That is the problem tackled by Pharos-ESG, a framework for parsing ESG reports into structured, multimodal, hierarchically labelled data.¹ The interesting part is not that another model can extract text from a PDF. We have enough PDF-to-text tools to form a small and irritating civilisation. The interesting part is that the paper treats ESG understanding as a sequence of infrastructure problems: first recover reading order, then reconstruct hierarchy, then ground images in nearby context, then attach finance-relevant labels.

This is the correct order of operations. A model cannot reliably classify a disclosure if it does not know which text comes before which chart. It cannot compare emissions policy across firms if the relevant section boundary is missing. It cannot reason about a visual element if the legend, explanatory paragraph, and section heading are separated by layout rather than meaning. ESG intelligence does not begin with a bigger language model. It begins with not feeding that model a shredded document and asking it to be brave.

The wrong lesson is “use a stronger multimodal model”

The tempting interpretation is simple: ESG reports are visual and textual, therefore the solution is a powerful vision-language model. Point the model at the page, ask for structure, perhaps add a heroic prompt, and hope that the sustainability gods approve.

Pharos-ESG is more interesting because it argues, in effect, that this is the wrong abstraction. The bottleneck is not only perception. It is document evidence architecture.

ESG reports create three linked problems. The first is reading order. A page may contain two columns, a callout box, a table, an image, a caption, and a floating statistic. Human readers use layout conventions to infer the intended sequence. Machines often flatten the page in the wrong order, which turns coherent disclosure into corporate confetti.

The second problem is hierarchy. Long ESG reports are weakly structured. Heading size, numbering, indentation, and visual emphasis are inconsistent across firms. A heading may appear in the table of contents, reappear with slightly different wording in the body, or be implied by nearby content rather than explicitly printed. For downstream analysis, this matters. “Energy consumption” under environmental performance is not the same analytical object as a decorative energy-related chart placed in an executive summary.

The third problem is multimodal context. ESG visuals rarely explain themselves. A chart may require a legend. A map may require nearby text. A photograph may be meaningless without the section path that tells us whether it concerns community investment, workplace safety, or renewable energy. Treating images as isolated objects produces fluent but shallow captions. Yes, the model can describe circles and colours. Congratulations, it has reached kindergarten geometry.

Pharos-ESG therefore does not simply “read” a report. It turns the report into an ordered, hierarchical, multimodal evidence object.

Step one: recover the order before pretending to understand the content

The first mechanism is reading-order modelling. Pharos-ESG represents pages as content blocks: text, tables, images, bounding boxes, block types, and page indices. Instead of relying on a global top-to-bottom or left-to-right heuristic, it predicts pairwise successor relationships between blocks. In plain language: for each block, the system estimates which block should be read next.

Those pairwise relations are then converted into a directed graph and topologically sorted into a globally consistent reading sequence. This is a neat technical choice because it reframes reading order as local relational structure rather than a fragile single ranking pass.

The business interpretation is straightforward. Bad reading order contaminates everything downstream. If a chart is read before its explanatory paragraph, if a caption attaches to the wrong image, or if a callout interrupts a policy section, later classification is already compromised. You do not have an ESG analytics problem yet. You have a plumbing problem.

In the paper’s main Chinese-report evaluation, Pharos-ESG achieves a Reading Order Kendall’s Tau score of 0.92, ahead of dedicated document parsers and general-purpose multimodal models. That number matters because reading order is not a decorative metric. It is the foundation layer for turning disclosure into comparable data.

Step two: use the table of contents as an anchor, not an ornament

The second mechanism is table-of-contents-guided hierarchy reconstruction. This is where the paper becomes more than a parsing exercise.

Many ESG reports include a table of contents, but the table of contents is not automatically usable. Entries may be visually grouped, split across lines, styled inconsistently, or only loosely aligned with body headings. Pharos-ESG uses RAP, or Region-Aware Prompting, to parse ToC structure by using visual and semantic cues such as colour similarity, spatial proximity, textual adjacency, and region-level grouping.

Then comes ALIGN, an anchor-based alignment procedure that maps ToC headings back into the body. It starts with exact matching, expands with fuzzy and containment matching, and uses context-aware insertion when headings cannot be directly matched. This last part is important: the system is not only finding structure; it is repairing missing or implicit structure.

For business users, this is the difference between search and evidence navigation. Search can find the word “emissions.” Hierarchy tells you whether that word appears under strategy, risk management, performance metrics, supply-chain policy, or a glossy CEO message. Only one of those is likely to support serious comparison.

The headline result here is ToC-body title alignment. Pharos-ESG reports 92.46% TBTA on Chinese ESG reports, while dedicated parsers perform extremely poorly and general-purpose multimodal models remain much lower. The likely reason is not mysterious. Generic parsers often trust local visual cues. ESG reports punish that innocence.

Step three: images need witnesses

The third mechanism is contextual image-to-text transformation. This is where the paper addresses a subtle failure mode in multimodal AI: visual description without document understanding.

Pharos-ESG first aggregates each target visual element with surrounding content, guided by hierarchy and reading order. It then generates an image description using the visual element plus its section path and nearby ordered evidence. The paper’s case study shows why this matters. A baseline multimodal model treats related visual elements separately and produces generic descriptions. Pharos-ESG identifies that one image functions as a legend for another and uses nearby text to interpret the visual content more accurately.

The likely purpose of this case study is not to prove aggregate performance by itself. It is exploratory and explanatory. It illustrates the mechanism behind the quantitative results: ESG visuals are often relational. The meaning is not inside the image alone; it is distributed across the image, legend, text, and hierarchy.

That is exactly the kind of detail that matters in assurance, audit, and investment research. A system that captions a chart as “a circular chart showing categories” is technically producing language. It is not producing evidence.

Step four: labels turn recovered structure into financial signals

Once the report is ordered, structured, and visually grounded, Pharos-ESG adds multi-level labels. The MLPDH module maps content blocks into ESG category, GRI indicator, and sentiment. It uses text semantics, heading-path context, and global reading order, then enforces hierarchy-aware consistency so that child labels remain logically aligned with parent labels.

This matters because ESG analysis is rarely satisfied by raw text. Investors, auditors, and regulators need comparable categories. They need to know whether a block concerns environment, social, governance, or non-ESG content; whether it maps to a particular GRI-style indicator; and whether its tone is positive, neutral, or negative.

The label-prediction results are strong. MLPDH reaches a macro-F1 of 86.32 and a hierarchy logic accuracy of 94.78, outperforming traditional and neural baselines including SVM+TF-IDF, XGBoost, BERT-base, HAN, and HMCN. The improvement over HMCN is especially relevant because HMCN already models hierarchy; Pharos-ESG gains from combining hierarchy with reading order and richer contextual embeddings.

That is the business pattern again: the value comes from preserving document structure before classification, not from treating every paragraph as a free-floating text snippet in the sustainability soup.

The evidence supports the pipeline, not just the leaderboard

The paper’s experiments are useful because they test different parts of the mechanism rather than only presenting one grand score. The main evidence comes from expert-annotated ESG reports across China, Hong Kong, and the United States, with the most detailed parsing evaluation on Chinese reports, which the authors describe as structurally more complex.

The ablation study is particularly important. With the core modules disabled, the system records an F1 of 76.95. Adding a general-prompting ToC parser nudges performance to 78.79. Replacing that with RAP lifts it to 83.57. Adding reading-order modelling brings it to 88.14. Adding partial ToC-body alignment reaches 90.05. The full system reaches 93.59.

That progression is the paper’s real argument in miniature. Performance improves as the system restores the document’s missing evidence architecture. The model is not merely becoming more powerful. The input is becoming more intelligible.

Aurora-ESG is the dataset layer, with one reporting wrinkle

The paper also introduces Aurora-ESG, a structured ESG-report dataset spanning Mainland China, Hong Kong, and U.S. markets. It includes document-level metadata, page-level references, content-block structure, reading order, hierarchy, content type, ESG category, GRI label, and sentiment label.

This is potentially valuable because ESG research has often relied on proxies: third-party ratings, disclosure indicators, small samples, or text-only corpora. A dataset with full multimodal blocks and hierarchy creates better conditions for cross-company comparison, year-over-year consistency checks, greenwashing research, and long-document model evaluation.

There is, however, a small reporting wrinkle worth preserving rather than politely sanding away. The paper states that Aurora-ESG contains 24,409 reports. Later, the market-level counts listed for China, Hong Kong, and the U.S. are 3,369, 13,057, and 7,539 reports respectively, which sum to 23,965. This does not invalidate the contribution, but it does mean the dataset scale should be cited with care. “Over 23,000 reports” is safer than treating every count as perfectly reconciled.

This is not pedantry. Dataset scale is a business claim. If a system is going to support compliance benchmarking or investment research, the first governance habit is to keep the inventory straight. Very unfashionable, very necessary.

What this means for investors, auditors, and regulators

The practical relevance of Pharos-ESG is not that it magically detects virtue. It does not. The practical relevance is that it can turn unstructured sustainability disclosure into structured evidence that other systems can inspect.

For investors, that means ESG reports become more comparable. Analysts can examine which topics appear, where they appear, how they are framed, whether quantitative evidence accompanies claims, and whether tone shifts over time. This is not the same as proving sustainability performance, but it is a stronger basis than treating the PDF as a branding artefact.

For auditors and assurance teams, the pipeline suggests a route toward better disclosure traceability. A structured report can support sampling, anomaly detection, missing-section checks, and evidence-linking workflows. The human reviewer still matters. The difference is that the reviewer is no longer manually spelunking through visual chaos with a highlighter and a mild sense of despair.

For regulators, the appeal is cross-firm and cross-market comparison. If disclosures can be parsed into standardised categories and aligned with reporting indicators, supervisory review becomes more systematic. Regulators could compare coverage, omissions, tone, and structure across issuers. Again, this is not automated judgment. It is automated preparation for judgment.

For companies, the implication is sharper. Once ESG reports become machine-readable, decorative ambiguity becomes less useful. Firms that rely on beautiful formatting to hide thin disclosure may find that parsing systems are annoyingly indifferent to vibes. The machine wants structure, evidence, and consistency. It has no patience for a full-page forest photograph unless the forest photograph has metadata and analytical purpose.

The boundary: parsing is not assurance

The paper directly shows that Pharos-ESG improves ESG report parsing, reading-order recovery, hierarchy reconstruction, contextual visual narration, and multi-level labelling on the reported benchmarks. That is already meaningful.

What Cognaptus infers is broader but separate: systems like this can become infrastructure for ESG analytics, audit preparation, benchmarking, and disclosure intelligence. The path is plausible because structured evidence is a prerequisite for those workflows.

What remains uncertain is whether these structured outputs reliably detect greenwashing, predict investor behaviour, satisfy regulators, or improve capital allocation decisions. The paper discusses such applications, but the experiments do not prove those outcomes. Greenwashing detection, in particular, requires external ground truth: actual emissions, operational records, regulatory findings, controversy data, or assurance evidence. A parsed report can reveal inconsistency. It cannot, by itself, tell whether the company is lying.

There are also deployment questions. The expert-annotated evaluation set is valuable but still limited relative to the diversity of global ESG disclosure. Cross-market results are encouraging, especially the strong U.S. performance, but the Hong Kong and U.S. samples are much smaller than the Chinese evaluation. And any pipeline that uses model-generated descriptions and labels needs audit trails, confidence scoring, and version control before it becomes part of regulated workflows.

The mature business interpretation is therefore not “AI solves ESG.” That sentence should be placed in a locked drawer with other crimes against thought. The better interpretation is that ESG analysis becomes more credible when the disclosure layer is converted into structured, inspectable, multimodal evidence.

The real shift is from documents to evidence systems

Pharos-ESG is best understood as a document infrastructure paper wearing an ESG jacket.

Its mechanism-first lesson is simple: before AI can reason over corporate sustainability claims, it must reconstruct the document as an evidence system. Reading order creates sequence. ToC anchors create hierarchy. Contextual narration turns visuals into usable text. Multi-level labels turn blocks into comparable signals. Aurora-ESG then packages those outputs into a dataset that can support research and downstream tooling.

That is a quieter contribution than promising automated ESG truth. It is also more useful.

The future of ESG reporting will not be decided only by better disclosure rules or longer sustainability reports. It will also be decided by whether those reports can be parsed, aligned, compared, challenged, and audited at scale. Once reports stop being merely read and start being parsed, the theatre of disclosure has to share the stage with evidence.

A terrible day for decorative ambiguity. A decent day for governance.

Cognaptus: Automate the Present, Incubate the Future.