Track · AI in Product4 min read
36 Ways to Define One KPI: Why AI Agents Still Can't Replace Analysts
A supply chain engineer found 36 undocumented definitions of one delivery KPI and planners ditching AI for Excel — proof agentic workflows still break on corporate messiness.

Context
A consumer goods company used 36 undocumented ways to define one on-time-delivery KPI, uncovered over 12 weeks of workshops.
Planners abandoned AI assistants because the same question and input file produced two different Python scripts.
A deterministic Wagner-Whitin redesign at a Swiss manufacturer is estimated to cut production costs by 30%.
The author draws on 60+ supply chain projects across Asia Pacific and Europe since 2020.
His 2027 goal: avoid becoming a 'meat proxy' who only relays requests to AI.
Twelve weeks of workshops at a consumer goods company revealed 36 different undocumented ways to answer a single question: was this store delivery late or on time? That number, from a supply chain engineer writing under the banner of his analytics blog Logigreen, is the clearest evidence yet of why AI agents keep stalling inside corporate environments — and where analytics professionals should stake their claim through 2027.
The author, a supply chain professional who learned data science to optimize manufacturing and logistics operations across 60+ projects in Asia Pacific and Europe since 2020, opens with a confession. His first session with Claude Code felt like a turning point: from one prompt, it adapted an algorithm he'd spent months designing and deployed it on his server. Eighteen months later, he wakes up asking where his value sits. His stated goal for 2027 is blunt: avoid becoming what he calls a "meat proxy" — a human whose only job is relaying requests to an AI and passing answers back.
The good news for analysts: the field evidence says agents rarely meet automation expectations.
Why can't an agent read a company from its database?
The first obstacle is what practitioners now call ontology — mapping how the business actually runs. Companies are not monolithic systems with one documented data source. They're legacy ERP, CRM and WMS platforms, patched for years to satisfy teams that each generate and use data their own way.
The late-delivery project proves the point. On paper, the formula was trivial: compare the ERP's Delivery Time field with Requested Delivery Time and output a boolean. In practice, the France teams used Shipping Time because stores send their own trucks, while the Brazil team flagged a delivery late only if it arrived 10 days past the requested date, because variable sea-freight lead times made tighter rules meaningless. None of it was written down.
Extracting that knowledge required a human with connections inside the company, the ability to structure workshops, and industry experience that doesn't exist online. "Do you think that plugging Claude Code into the SQL database of the ERP would help to get these insights?" he asks rhetorically. In most AI projects he's seen, someone still had to know what each team takes in, changes, and hands to the next — and in most organizations, that person doesn't exist, making this a management problem more than a technical one.
When does determinism beat an LLM?
The second obstacle is stochasticity. A manufacturing company equipped its planners with AI assistants, with the supply chain director aiming for analytics-as-a-service: on-demand answers to questions like how many units to produce next week. It collapsed fast.
"It became an operational chaos," the director said. One planner admitted she'd switched back to her Excel spreadsheet because "AI can give me different outputs for the same question from one day to another." The diagnosis confirmed it: for the same question and the same input file, the agent generated two different Python scripts.
After workshops on what planners actually needed, the conclusion was clear — they didn't need AI at all. Deterministic tools written in code, like inventory management rules and Wagner-Whitin lot-sizing, return the same output every run. The director's verdict on what was missing: "We needed someone to challenge our requests, propose a structured process to generate insights from data and design robust tools we can rely on."
Which two skills keep analysts relevant?
First: learn the whole chain your data passes through — input (how data enters systems), processing (how it's stored and updated), output (how teams read it and act). Applied to the FMCG company, the framework compressed 12 weeks of workshop findings into three lines: customer service edits requested dates, a French script rewrites them against SLA thresholds, and American teams ignore delivery dates entirely because they don't own transportation.
Second: translate, and challenge, the request. At a Swiss mechanical parts manufacturer, the ask was an optimization algorithm. The real problem was process: orders arrived by email into per-group mailboxes, and planners built production lots by hand in Excel, never grouping SKUs that could share batches. The delivered solution ran in three validated steps — orders consolidated in Anaplan, a Wagner-Whitin algorithm grouping batches, and optimization embedded in planners' daily workflow with a dashboard and email alerts. Production costs are estimated to drop by 30%, mostly from lower inventory, and other teams reused the deterministic service for their own simulations.
A coding assistant, the author notes, would have written the algorithm requested without ever questioning the process around it. Before building anything, ask what decision the tool will change and who will make it — agents still don't.
He points practitioners to free resources on his blog and YouTube channel, including a playlist of 5-minute videos covering 25+ analytics concepts, each with a real use case, source code and anonymized dataset. As agentic workflows spread into more processes through 2027, the analysts who survive won't be the best prompt writers — they'll be the ones holding the process knowledge no database contains.
via towardsdatascience.com (Source)
More from Daniel Okafor
Show full bio
Staff writer covering technology and product news in product management at Roadmap File.
60 articles