Short answer
Why It Matters
Health systems are moving quickly to deploy AI across patient financial services, often without a clear framework for where automation adds value and where it introduces risk. When technology is placed too close to a patient's financial decisions without oversight, mistakes compound quickly: incorrect balances, mispriced payment plans, and eroded trust that surfaces on the customer service line and in patient complaints.
There is also a process-integrity risk. Automating a workflow that was never validated to begin with does not fix the workflow - it scales the defect. As Amanda Hines, Patient Financial Services Director at Essentia Health, put it: "Sometimes you could automate a process that's a bad process. And now you're just automating a bad process."
Key Takeaways
- Apply the "can vs. should" test. Capability is not permission. Just because AI can perform a task does not mean it should perform that task without governance.
- Reserve automation for repetitive, transactional work. Moving accounts between work queues, updating demographic data, retrieving statement information, and initiating pre-defined actions are strong candidates.
- Keep humans on decisions that affect a patient's wallet. Determining balances, setting payment plan amounts, and evaluating affordability are high-stakes interactions. Mistakes here directly increase patient stress and complaint volume.
- Validate the process before you automate it. If the underlying workflow is broken, automation multiplies the defect.
- Chatbots work for information retrieval, not advocacy. They can answer bounded questions ("What was my last balance?" "Send me a 1099") but cannot hear confusion, sense frustration, apply empathy, or advocate on the patient's behalf.
- Governance is non-negotiable. AI deployment requires risk assessment, transparency, auditing, human review, and defined escalation paths. You cannot outsource accountability to a robot.
- Redesign human roles around exception handling. As automation absorbs repetitive work, the remaining human work becomes more complex. QA, exception management, and AI supervision become senior-level competencies.
Expert Perspective
Amanda Hines frames the decision as a risk-tiered question rather than a technology question. In her words: "Just because AI can do something doesn't automatically mean we should let it do it and without oversight." She draws a specific line at the patient's wallet: "If the decision gets closer to a patient's wallet, you really got to think about it. Because that's their livelihood."
She illustrates the boundary with Essentia Health's MyChart chatbot, Emmy. The tool handles bounded requests - 1099 retrieval, statement balances, payment plan initiation - well. But she is direct about its limits: "It can't use empathy, it doesn't have really judgment. It doesn't have context. A person can hear confusion. They can hear if the patient is getting frustrated."
Her most quoted line in this conversation captures the governance principle in a single sentence: "You can't outsource accountability to a robot."
Hines is not anti-technology. She is deploying it actively at Essentia Health. Her point is sequencing and scope: automate the transactional layer first, keep humans on judgment and advocacy work, and build the governance infrastructure (auditing, QA, human review, escalation) before scaling
A Practical Framework: Two Buckets for AI in RCM
Bucket 1 - Good candidates for AI and automation: - Routing accounts to work queues - Address and demographic updates - Data entry and tick-mark tasks - Statement balance retrieval - 1099 and document requests - Simple status inquiries - Payment plan initiation against pre-approved parameters
Bucket 2 - Requires human judgment (or tightly governed AI with human oversight): - Determining what a patient owes - Setting or negotiating payment plan amounts - Financial assistance and affordability decisions - Handling frustrated, confused, or emotionally distressed patients - Advocacy conversations that require context beyond the script - Escalations involving disputes, hardship, or complex insurance situations
Governance Checklist Before Deployment
- Has the underlying process been validated?
- What is the risk tier: informational, transactional, or financial-decision?
- How will accuracy be audited, and at what cadence?
- Who reviews exceptions, and how quickly can a patient reach a human?
- What sentiment or frustration signals trigger escalation?
- Are QA staff trained to supervise AI output rather than perform the underlying task?
- What post-go-live metrics will confirm the deployment is working?
