Data-Quality

Clinical AI has a paperwork problem. Not the usual paperwork problem, where doctors drown in documentation and everyone promises that software will save them. The more interesting problem sits one layer below: the paperwork used to judge the software may itself be wrong. That is the uncomfortable center of Scalable Stewardship of an LLM-Assisted Clinical Benchmark with Physician Oversight, a paper that audits MedCalc-Bench, a benchmark for testing whether language models can compute medical risk scores from patient narratives.1 The paper’s target is not a toy dataset. MedCalc-Bench covers 55 medical calculators and includes 10,053 training instances plus 1,047 test instances. Its labels were produced through an LLM-assisted pipeline: GPT-3.5 matched patient contexts to calculator questions, GPT-4 extracted clinical features, and Python scripts aggregated those features into final scores. ...

Accuracy is a comforting number. That is precisely the problem. A neural network can score well on a test set and still be operationally suspicious. The labels may be corrupted. The input may be degraded. A small patch may have quietly hijacked part of the model’s learned behavior. The model may be confident, calibrated enough for a dashboard, and still untrustworthy in the one place where the business actually needs it to behave. ...

Bad data is not one problem. It is at least three problems wearing the same cheap trench coat. There is bad data that appears once and disappears. There is bad data that keeps appearing, but becomes rarer as the corpus grows. And there is bad data that settles in at a stable rate, like a permanent tenant with poor hygiene and legal representation. Business discussions about AI training data often compress these into one vague category called “noise”. Convenient, yes. Informative, no. ...

Phones are where automation goes to embarrass itself. A desktop workflow can often be forced into a neat sequence: open tab, click menu, submit form, pretend the enterprise software was designed by someone who likes people. Mobile apps are less polite. They hide features behind drawers, gestures, modals, permissions, scrolling lists, bottom sheets, dark-pattern-ish confirmations, and the occasional button that looks decorative until it suddenly matters. A human user handles this with a mixture of visual attention, memory, muscle habit, and mild resentment. A mobile control agent has to do it with pixels, UI trees, and a policy that decides where the next finger should land. ...

Workshop logs are not glamorous. They are where predictive-maintenance dreams go to meet misspelled component names, missing codes, wrong vehicle identifiers, and dates that imply a truck was both under repair and happily accumulating kilometres. Industrial AI, as ever, is less a matter of elegant algorithms than of persuading messy operational records to stop lying. ...

TL;DR for operators VeriMinder is a useful reminder that the most dangerous analytics failure is not always a bad SQL query. Sometimes the SQL is correct, the dashboard loads, the stakeholder nods, and the decision is still built on a question that should never have passed quality control. The paper introduces VeriMinder, an interactive system that sits before or alongside a natural-language-to-SQL workflow and checks whether the user’s question is biased, under-specified, or poorly aligned with the decision being made.1 Its target is not SQL syntax. Its target is analytical intent. ...

TL;DR for operators Generative AI is useful for data processing when the work is painfully simple for a human and painfully awkward for software. That sounds like a joke until you meet the actual enterprise data stack: PDFs with shifting layouts, scanned documents with OCR scars, multilingual reports, product descriptions pretending to be industry classifications, and a graveyard of “temporary” spreadsheets that somehow became critical infrastructure. ...