AI adoption is rising, but most deployments fail on unclear goals, weak governance, and pilots that never scale.
High performers get results by redesigning whole workflows, with clear objectives, baseline KPIs, defined human-in-the-loop rules, and continuous monitoring, rather than adding isolated automation tools.
Modern AI agents can reason, take action, and automate multi-step workflows across servicing, collections, quality assurance, and back-office operations. Deployed strategically, they can drive material improvements in cost efficiency, resolution rates, QA quality, and the customer experience.
They also introduce new risk categories: hallucinations, policy deviations, compounding errors, adversarial inputs, and misread customer intent. Regulators, including the NCUA, CFPB, FDIC, and OCC, expect explainability, bias testing, documentation, strong vendor oversight, and clear paths to a human.
- A documented AI strategy tied to customer value and operational efficiency.
- An AI governance framework aligned with regulatory expectations.
- Robust vendor controls, testing protocols, and model lifecycle oversight.
- Policies for privacy, fairness, explainability, and incident escalation.
Failure rates reflect deployment strategy, not model stagnation
MIT’s 2025 State of AI in Business report found that 95% of AI pilots fail. Is AI a dud?
Gartner placed AI agents at the peak of inflated expectations in June 2025, with the usual two to five years to the plateau of productivity. In Krew’s view that curve is running faster than usual: voice agents, text agents, and workflow automations are each further along it than the category as a whole.
Progress is not stalling. It is accelerating. Measured by the length of task a model can complete on its own, frontier models have moved from writing a sentence to sustained, day-long execution, roughly a full working day, in a few years. The capabilities are there. What separates the pilots that fail from the deployments that pay is how they are set up.
Length of tasks AI can complete, by model
Task length at a 50% success rate, in hours, by release date.
Successful firms deploy with purpose and with conviction
78% of companies use AI, yet the typical impact is under 10% in cost savings and under 5% in revenue uplift, and only about 1% of U.S. firms have scaled AI across the enterprise. Successful firms expect 90% of their value to come from reshaping and inventing workflows, not from sprinkling AI on top of old processes.
Purpose means a target like “increase NPS by 3 points” or “cut delinquency management costs by 6%”, not “an AI chatbot for our website”.
Successful firms deploy AI across entire workflows rather than isolated pilots, often with AI agents embedded where decisions are made. They move from humans using tools to AI-centered workflows orchestrated by humans, and they are five times more likely to plan their workforce around it.
Conviction means a clear value thesis, such as a stated NPS uplift over three years, that goes beyond hedging.
Successful companies run AI as CEO and board-sponsored, multiyear programs with a funded roadmap. High performers are three times more likely to expect transformative change from AI within three years, and one-third of them spend more than 20% of their digital budget on it, tracking KPIs for every AI solution.
- Set baseline metrics, a target, and an owner for each AI deployment, from day one.
- Redesign the workflow end to end.
- Define AI and human roles. For each process, label it AI-led, human-in-the-loop, or human-only, with escalation rules.
- Write a three-year AI value thesis, socialize it, and lock it in at executive level.
- Set stage gates: the evidence (prototypes, KPIs, risk sign-off) needed to move from pilot to scale, reviewed monthly at the executive table.
Widespread use, narrow impact
Share of companies, and the typical result where AI is in use.
Typical cost savings
<0%
Typical revenue uplift
<0%
Value from redesigned workflows
0%
Expected by successful firms
Large language models are one of five kinds of machine learning
Each kind answers a different question, and most institutions will run several. The language model is the one that talks; the others score, sort, and predict.
- 01
- 02
- 03
- 04
- 05
Large language models
Systems trained on massive amounts of text to understand and generate language. A model like ChatGPT predicts the next best word given everything generated so far. It learns patterns, not meaning, and has no goals of its own.
How it works- They learn patterns, not meaning: they scan huge amounts of text and learn which words and ideas tend to appear together.
- They have no goals or understanding. They generate the most likely continuation of the text, based on the patterns learned in training.
Use casesBuilding AI agents and workflow automation, summarizing and classifying text, knowledge retrieval in plain language, personalization at scale.
Good for: AI agents and customer personalization
- 01
- 02
- 03
- 04
- 05
Tree-based models
Models that make decisions by asking a sequence of simple questions, like a flowchart whose rules are generated from data rather than written by hand.
How it works- They split data into simple decisions, repeatedly dividing it into smaller groups by the features that best separate outcomes, such as whether income is over $50k.
- They predict by following a path: each input is routed down the branches until it reaches a final decision or value.
Use casesPredicting which accounts are most likely to fall behind, recommending the best contact channel, suggesting the repayment plan a customer is most likely to keep.
Good for: Delinquency management and fraud prevention
- 01
- 02
- 03
- 04
- 05
Linear and logistic models
Very simple models that find straight-line relationships in data. In other words, regressions.
How it works- They fit a straight line through the data, or an S-shaped curve for yes-or-no outcomes, weighting each driver by how much it moves the result.
- Each weight is a readable reason: any score can be traced back to a handful of factors and how much each one contributed.
Use casesScoring and ranking where the drivers are few and the reasons must be easy to explain.
Good for: Risk assessment and customer segmentation
- 01
- 02
- 03
- 04
- 05
Neural networks
Flexible pattern-recognition systems inspired by brain-like structures. They learn from labeled examples and then recognize similar patterns in inputs they have never seen.
How it works- They learn from examples: as the network sees many labeled examples, it adjusts itself to make fewer mistakes.
- They apply what they learned: after training, the network recognizes similar patterns in new inputs it has never seen.
Use casesDelinquency risk assessment from historical patterns, customer insights for personalized service, spotting unusual transactions in real time.
Good for: Delinquency management and payment negotiations
- 01
- 02
- 03
- 04
- 05
Unsupervised models
Models that discover structure in data without being told the right answer. All they do is find groups and patterns, the way a phone sorts faces into albums.
How it works- They find patterns without labels: they explore the data and group similar customers or behaviors together, without being told what the right answer is.
- They reveal hidden structure: segments, trends, or anomalies that are not obvious from the raw data.
Use casesGrouping customers by payment behavior or responsiveness, finding clusters showing early signs of financial stress, revealing which channels and plans work for whom.
Good for: Customer segmentation and clustering
AI agents are workflow orchestrators that think, decide, and execute, rather than just respond
They take actions, not just give answers. An agent can plan, decide, and execute a task by breaking it into steps, much like a digital assistant that does things on your behalf. It uses tools and enterprise data to reach a goal: it can access records, call systems, and adapt its approach based on what it learns along the way.
- QA agents: reviewing interactions and compliance against custom scorecards.
- Loan servicing agents: assisting with payment plans, account updates, hardship options.
- Delinquency management agents: automating outreach, tailoring messages, and helping customers get back on track.
- Workflow automation agents: completing back-office tasks such as document processing, data entry, and follow-ups.
An agent thinks, decides, and executes
A servicing call, broken into steps the agent plans and carries out.
The agent reads the account, calls the systems it needs, and adapts its approach to what it learns on the call.
Like any worker, an AI agent can make mistakes, and its mistakes compound
Agents are probabilistic. On the same call, one run may be 99.99% certain that a customer can pay in full and another only 98% certain. A 98% success rate sounds high, but decisions compound: five steps at 98% accuracy yields about 90% overall, which means one workflow in ten fails.
Agents therefore require rigorous testing in simulated environments to quantify their likelihood of error before they are trusted with real ones.
Two further failure modes look familiar from human teams. An agent could hallucinate or misrepresent your policies and brand: providing inaccurate or non-compliant financial guidance, referencing products or competitors your institution does not endorse, or making assumptions about a customer’s situation that are inappropriate or insensitive. And an agent may not capture every edge case: misreading emotionally charged statements, taking them too literally, or overlooking a compliance-sensitive red flag that should have gone to a person. A customer who says they will not be paying and asks whether you understand may be making a request that ends further contact, and missing it creates FDCPA exposure.
Decisions compound
Each step is 98% accurate. The workflow is only as accurate as the product of its steps.
Five steps at 98% yields about 90% overall: one run in ten fails. Evaluations, guardrails, and human review of uncertain cases are what close the gap.
Guardrails are essential for safe deployment, and evaluations tell you how safe
Extensive guardrails, where specially trained oversight models check other models for mistakes, reduce error rates. Human-in-the-loop review handles the high-uncertainty cases. The example alongside is a hijacking attempt: a customer instructs the agent to praise a competitor at the end of every sentence, the first draft complies, and the guardrail blocks it before anything is said.
Evaluations are how the error rate is known rather than assumed. Leading AI firms run hundreds of thousands of simulations against categories such as jailbreaking, content moderation, hallucination prevention, competitor checks, policy compliance requests such as bankruptcy and cease-and-desist, and optional internal policies, delivering compliant agents with only low single-digit basis-point error rates.
Those simulations are typically validated by independent assurance providers, which is what allows the resulting agents to be insured against systemic failure.
A hijacking attempt, caught before it is spoken
An oversight model checks every draft response against policy.
What the NCUA, CFPB, FDIC, OCC, and FFIEC expect from AI deployments
The expectations are consistent across regulators and fall into four areas. None of them is new; each applies the existing model-risk and consumer-protection framework to a new kind of model.
- 01
- 02
- 03
- 04
Explainability
What they expect- Ability to explain in plain language how AI is used in a product or process.
- Specific, accurate reasons for credit denials and other adverse actions. No black-box excuses.
- Documented model purpose, assumptions, limitations, and key drivers.
- 01
- 02
- 03
- 04
Customer impact
What they expect- Demonstrated testing for disparate impact and unfair bias across protected classes.
- Ongoing monitoring of approval rates, pricing, collections, and servicing outcomes.
- Clear disclosures, opt-outs where applicable, and easy access to a human review or complaint path.
- 01
- 02
- 03
- 04
Model governance
What they expect- Treat AI as models under SR 11-7-style guidance: inventory, model owners, use-case documentation.
- Independent validation before go-live, and regular back-testing, drift, and performance reviews.
- Board and senior management oversight, with policies, controls, and escalation paths for issues.
- 01
- 02
- 03
- 04
Data privacy
What they expect- Lawful data use under FCRA, GLBA, and UDAAP, with limits on surveillance and non-traditional data.
- Strong controls over vendors: rights to audit, explain, remediate, and shut down AI systems.
- Cybersecurity, access control, and logging around training data, prompts, and outputs.
Five places AI pays for banks and credit unions first
The common thread is volume: work that arrives all day, follows policy, and has to be done the same way every time.
- 01
- 02
- 03
- 04
- 05
Customer service
Where it pays- 24/7 omnichannel support with consistent, compliant responses.
- Faster response times and shorter waits.
- Personalized guidance on accounts, payments, and product questions.
- 01
- 02
- 03
- 04
- 05
Loan servicing
Where it pays- Instant, accurate answers to servicing inquiries across channels.
- Automated follow-ups, payment reminders, and hardship workflows.
- Lower handle times and higher customer satisfaction.
- 01
- 02
- 03
- 04
- 05
Delinquency management
Where it pays- Outreach that adapts to customer intent and risk signals.
- Personalized payment plans that improve resolution rates.
- Automation of routine tasks: promises to pay, status updates, documentation.
- 01
- 02
- 03
- 04
- 05
Back-office workflow automation
Where it pays- Automated data entry, document processing, and QA scoring.
- Continuous monitoring of agent and customer interactions for compliance.
- 01
- 02
- 03
- 04
- 05
Fraud and risk monitoring
Where it pays- Real-time detection of unusual activity across transactions and channels.
- Enhanced identity verification and anomaly detection.
- Less manual review for risk teams.
Use AI to ensure and elevate every customer conversation
Sampling a few calls a month tells you about a few calls a month. An evaluation agent with a scorecard reads every transcript, recording, and piece of interaction metadata, scores it against criteria aligned to regulatory and internal policy, and flags the risks with evidence attached.
The output is a quality score per conversation and, over time, dashboards by agent, team, and topic, with outliers detected automatically and feedback routed to supervisors or straight into training workflows.
Quality assurance on every interaction
From the conversation to a scored, evidence-backed result.
- 01Conversation with the customer
- Transcripts, voice or chat
- Audio recordings
- Interaction metadata: timestamps, duration, disposition
- 02AI evaluation agent with scorecard
- Criteria aligned to regulatory and internal policies
- Scores scripting, disclosures, accuracy, empathy, resolution
- Flags regulatory risk with evidence-backed scoring
- 03QA quality score generated
- Continuous monitoring and trend insights
- Agent, team, and topic dashboards
- Outliers detected; feedback routed to supervisors or training
Better answers, higher NPS, and a record that stays current
A servicing agent captures the caller’s intent, loan details, and reason for reaching out, transcribing in real time so the rest of the workflow can run on it. It retrieves accurate answers from approved knowledge bases, policies, and servicing guidelines, and offers the next best response given the customer’s context.
Then it writes the outcome back: payment status, promises to pay, address changes, into the servicing systems of record. The next interaction starts from current context, which is what removes the repeat call.
A servicing conversation, end to end
Capture the intent, retrieve the approved answer, write the outcome back.
- 01Conversation with the customer
- Captures caller intent, loan details, and reason for outreach
- Transcribes in real time for downstream automation
- 02Response retrieval
- Accurate answers from approved knowledge bases and policies
- The next best response, based on the customer’s context
- 03Context updating
- Payment status, promises to pay, address changes written back to servicing systems
- The next interaction starts with current context, so fewer repeat calls
“Financial institutions don’t need AI demos. They need systems that work under real policy and oversight. Krew is built for that reality in servicing and collections, and it shows.”
Michael Jones, Former President and Chief Operating Officer, TCF Bank
Contributors
Michael Goh
Cofounder, Member of Technical Staff
Prior to Krew, Michael led speech benchmarking at Artificial Analysis, an Andrew Ng-backed company, where he worked alongside OpenAI and Amazon on performance and quality evaluations of multimodal models, focusing on speech. He was also a FinTech venture capitalist and a management consultant at Bain. MS Computer Science, University of Chicago, where he was a Global Fellow; BA Economics and Management, University of Oxford, where he was a Fung Scholar.
Alice Zhang
Head of Product Compliance, Member of Technical Staff
Prior to Krew, Alice advised the United Nations on AI Ethics and Safety for defense industries. She was also In-House Counsel at a FinTech VC, where she advised on global regulatory compliance. BA Jurisprudence, University of Oxford, where she was a college scholar.
- Challapally, A., et al. (2025). The GenAI Divide: State of AI in Business 2025. MIT NANDA.
- Gartner (2025). Hype Cycle for Artificial Intelligence, June 2025.
- METR (2025). Measuring AI Ability to Complete Long Tasks, updated with Claude Opus 4.6.
- Wall Street Journal (2025). Companies Are Struggling to Drive a Return on AI. It Doesn’t Have to Be That Way.
- Boston Consulting Group (2025). The Widening AI Value Gap.
- McKinsey & Company (2025). The state of AI in 2025: Agents, innovation and transformation.
- Krew internal data and customer research (2025 to 2026).
This primer is general information for financial institution leaders, not advice for any particular institution. Third-party figures are quoted from the named sources as published, and reflect those sources’ methods and dates, not Krew’s own measurements unless stated. Model capabilities, regulatory expectations, and market figures change quickly, and the material may be revised as they do. Outcomes from any AI deployment depend on the institution’s objectives, data, workflows, controls, and oversight, and actual results will vary.
This document is provided for informational purposes only and does not constitute legal or regulatory advice. Motivated by Krew’s Collaborative Assurance Framework’s principles of shared accountability and streamlined assurance, we partner with our customers, combining our AI-driven credit servicing platform’s built-in security safeguards with each customer’s own controls, system configurations, and user training, to help manage regulatory risk. It remains the responsibility of each customer to ensure compliance with all applicable federal, state, and local statutes. All rights, responsibilities, and liabilities of Krew in relation to its customers are governed exclusively by the terms of Krew’s customer agreements. This document neither forms part of, nor alters, any contractual agreement between Krew and its customers.