Shipping Intelligence: How AI Products Drive Real Business Value

By Smita Madam MBA 18

Prepared for the UC Davis Graduate School of Management & Master of Science in Business Analytics (MSBA) Publications, Sept. 2026

Introduction: The Shift from Deterministic to Probabilistic Systems

Traditionally, software development was largely defined by deterministic architecture. Developers wrote explicit logic: defined inputs flowed through predictable algorithms to produce fixed, reproducible outputs. Analytics teams built dashboards and statistical models to evaluate past performance, but the boundary between human decision-making and software execution remained sharp.

Over the past decade, artificial intelligence has fundamentally transformed that paradigm. Modern technology platforms are increasingly powered by probabilistic systems—intelligent models capable of reasoning through unstructured data, evaluating context under uncertainty and performing multi-step tasks autonomously. Having built and scaled products across software engineering, digital payments, global e-commerce and domain-specific enterprise AI, I have had a front-row seat to this evolution.

Translating cutting-edge AI capabilities into enterprise-grade products requires more than model fine-tuning or API integrations. It demands a deep understanding of real-time data pipelines, confidence scoring, trade-offs between precision and latency, and robust human-in-the-loop guardrails. Below, I explore how artificial intelligence transformed business-critical workflows across three key stages of my product leadership journey: fraud mitigation at Capital One, catalog enrichment and agentic commerce at Amazon and automated legal document generation at AI Demand Pro.

1. Fraud Detection & Security: Real-Time Risk Intelligence at Capital One

In high-volume digital payment ecosystems, security and user experience sit in constant tension. Every additional authentication challenge or decline rule reduces fraud exposure, but it can also introduce user friction, cart abandonment, and lower customer retention. During my time in Digital Payments and Identity at Capital One, our core objective was to build intelligent risk evaluation systems that protected transactions seamlessly.

The Challenge

Legacy financial risk architectures across the fintech industry relied heavily on expert-driven, rule-based heuristics (such as flagging any transaction over a specific dollar threshold or outside a user’s home state). While simple to audit, these deterministic rules failed to capture complex, multi-variable fraud patterns. Moreover, they generated high rates of false positives, rejecting legitimate purchases and frustrating cardholders.

The Machine Learning Solution

Rather than relying on rigid rules or static card attributes, modern machine learning models evaluate real-time behavioral signals across multiple dimensions simultaneously:

• Smart Velocity & Location Telemetry: Comparing device location, IP connection paths and transaction pacing against a user's normal spending habits.
• Merchant Scoring with Virtual Card Numbers: Combining predictive risk scoring with dynamic card tokenization. By evaluating merchant trustworthiness on the fly and generating Virtual Card Numbers (VCNs) for online transactions, platforms shield primary accounts if a merchant's database is ever compromised.

Image

Key Takeaways for Product Leaders

1. Low-Latency Feature Engineering: A high-performing ML model is only as effective as the data pipeline supporting it. In payment authorization, feature extraction and model inference must execute in under 50 milliseconds to avoid degrading transaction speed.
2. Optimizing Precision vs. Recall: In fraud mitigation, over-indexing on recall (catching every possible fraudulent event) severely damages precision, leading to high false-positive rates. Product and analytics leaders must continuously calibrate threshold boundaries based on the lifetime value of customer trust versus the net financial cost of fraud losses.

2. Generative AI & Agentic Workflows: Scaling E-Commerce Infrastructure at Amazon

Managing global e-commerce catalog operations requires processing immense volumes of unstructured product data. As a Senior Product Manager, Tech in Amazon eCommerce Selection Monitoring team, I focused on generative AI and LLM-driven catalog enrichment products across global marketplaces, while contributing to next-generation agentic commerce initiatives like Buy For Me.

Catalog Enrichment via Large Language Models

In a global marketplace featuring millions of sellers and suppliers, incoming product data is inherently fragmented. Vendors submit listings with missing attributes, varied terminology, non-standardized units and incomplete localized descriptions across international marketplaces.

• The Generative AI Solution: Large language model (LLM) extraction and normalization pipelines process raw seller titles, bullet points and unstructured specifications. The models automatically extract missing attributes (such as material composition, dimensions, technical compatibility and brand classifications) and map them directly into a structured catalog schema.
• Business Impact: Automated catalog enrichment directly improves discoverability and search relevance. When product attributes are accurately populated, customer search filtering becomes significantly more effective, reducing catalog friction and improving customer discovery across retail and business-to-business storefronts.

Image

Moving from Generative to Agentic AI: "Buy For Me"

While catalog enrichment leverages generative models for data transformation, agentic commerce represents a fundamental shift in how consumers interact with software. Traditional e-commerce requires users to manually execute every phase of the buying journey: searching, comparing features across multiple listings, checking shipping constraints and completing payment steps.

Through publicly announced initiatives like Buy For Me, e-commerce is evolving toward autonomous, multi-agent execution:

• Intent Decomposition: The system translates natural language user prompts (for example, finding an item with specific price and delivery constraints) into multi-step execution plans.
• Autonomous Task Orchestration: AI agents interact directly with underlying microservices—querying live inventory, validating seller ratings, checking delivery logistics and preparing purchase authorizations.

Measurable Business & Customer Impact:

• Eliminating Friction & Conversion Drop-off: By compressing complex, multi-site research and checkout flows into a single autonomous action, agentic workflows collapse the conversion funnel—dramatically reducing cart abandonment and decision fatigue.
• Hyper-Personalized Match Accuracy: Agents evaluate real-time constraints (price limits, exact delivery windows, seller trust scores) in milliseconds, delivering higher customer satisfaction and lower return rates compared to traditional keyword search.
• Unlocking Conversational Commerce at Scale: Transforms e-commerce platforms from passive digital catalogs into proactive digital personal shoppers, driving higher average order value (AOV) and long-term customer lifetime value (LTV).

Image

Key Takeaways for Product Leaders

1. Mitigating Hallucination via Human-in-the-Loop (HITL) Architecture: In enterprise applications of LLMs, probabilistic error is an operational risk. Deploying automated confidence scoring ensures high-confidence outputs pass directly into production, while low-confidence predictions route to human oversight loops.
2. The Emergence of Agentic Protocols: Future software products will increasingly function as autonomous agents capable of tool call execution. Analytics and product teams must design APIs and data stores specifically for machine consumption and automated decisioning.

3. Domain-Specific Synthesis: Automated Legal Workflows at Arches Legal (previously AI Demand Pro)

General-purpose foundation models excel at broad conversational tasks, but enterprise applications in specialized fields—such as law, insurance and medicine—require high precision, strict document grounding and deterministic safety boundaries. At Arches Legal, my focus has been on building AI products that automate complex legal workflows, specifically Demand Letter Generation.

The Challenge

Drafting comprehensive legal demand letters in personal injury or insurance contexts is labor-intensive. Legal professionals must manually collect, review, cross-reference and summarize scores of unstructured documents, including police incident reports, medical provider charts, itemized billing statements and loss-of-income documentation. Synthesizing these disparate sources into a cohesive, legally sound document traditionally requires hours of manual drafting per case.

The Domain-Specific AI Solution

AI Demand Pro converts raw, unstructured case files (like medical records, billing statements and police reports) into fully formatted, attorney-grade settlement demand drafts—complete with medical chronologies and hyperlinked exhibits—in 15 to 30 minutes. Built specifically by and for personal injury lawyers, the platform automates record synthesis and fact extraction so law firms can eliminate drafting backlogs, maintain consistent narrative quality and scale caseloads without increasing administrative overhead.

Image

Key Takeaways for Product Leaders

1. Context Optimization over Data Dumping: Unconstrained prompt sizing does not replace structured data preparation. Overloading a model's context window with hundreds of pages of raw, unstructured records causes information loss (the "lost-in-the-middle" effect), increases latency and degrades output precision.
2. Verifiability as a Core Product Metric: Enterprise AI platforms must provide an audit trail for every conclusion. Designing interface components that visually link model outputs directly to specific source document references is critical for user trust, accuracy validation and workflow adoption.

Strategic Playbook for the Next Generation of MSBA & MBA Leaders

As graduates of the UC Davis Graduate School of Management MBA and Master of Science in Business Analytics programs step into product and technology leadership roles, they enter an industry undergoing rapid transformation. Based on my experience across fintech, e-commerce and specialized AI products, I encourage future leaders to ground their work in three core tenets:

1. Anchor AI Initiatives in Core Business Metrics

It is easy to get caught up in the excitement of novel foundation models or benchmark scores. However, technology for its own sake rarely builds enduring business value. Every AI initiative should map directly to measurable business outcomes:

• Reducing fraud losses and transaction friction (Capital One)
• Expanding catalog coverage and search discoverability (Amazon)
• Decreasing document generation cycle times (AI Demand Pro)

2. Design for Probabilistic Systems

Traditional product managers define predictable user journeys with fixed inputs and deterministic outputs. AI product managers must design systems where model outputs are probabilistic. Building product guardrails requires defining clear confidence thresholds, establishing fallback strategies when models fail and constructing continuous human feedback loops (RLHF/HITL) to refine system performance over time.

3. Cultivate the Tech-Business Translation Capacity

The highest-value professionals in today's technology landscape are those who can bridge technical data engineering teams and business executives. Understanding model mechanics, vector databases and agent orchestration allows leaders to evaluate technical feasibility accurately, while a strong business grounding ensures those technical capabilities address real customer needs.

Conclusion

The evolution of artificial intelligence from simple statistical rule sets to multi-agent autonomous execution marks a fundamental turning point in software development. By focusing on real-world business impact, robust architecture design and human-centered safety guardrails, business and analytics leaders can harness these powerful technologies to build products that redefine industries.

Image

About the Author

Smita Madam holds a Bachelor of Engineering in computer science and an MBA from the UC Davis Graduate School of Management. She has held technical and product leadership roles across companies including Microsoft, Capital One, Tableau, Amazon and Arches Legal (previously AI Demand Pro). She is active in the product management community, serving as a mentor through Women In Product, the Braven Professional Mentor Program, and the UC Davis GSM mentor network, and sharing insights on product leadership, artificial intelligence, innovation, and the evolving intersection of technology, business, and human potential through her LinkedIn newsletter, Product, AI & Beyond. She was recognized in Marquis Who’s Who for her contributions to product leadership.