Health systems are moving quickly to deploy AI across patient financial services, often without a clear framework for where automation adds value and where it introduces risk. When technology is placed too close to a patient's financial decisions without oversight, mistakes compound quickly: incorrect balances, mispriced payment plans, and eroded trust that surfaces on the customer service line and in patient complaints.
There is also a process-integrity risk. Automating a workflow that was never validated to begin with does not fix the workflow - it scales the defect. As Amanda Hines, Patient Financial Services Director at Essentia Health, put it: "Sometimes you could automate a process that's a bad process. And now you're just automating a bad process."
Amanda Hines frames the decision as a risk-tiered question rather than a technology question. In her words: "Just because AI can do something doesn't automatically mean we should let it do it and without oversight." She draws a specific line at the patient's wallet: "If the decision gets closer to a patient's wallet, you really got to think about it. Because that's their livelihood."
She illustrates the boundary with Essentia Health's MyChart chatbot, Emmy. The tool handles bounded requests - 1099 retrieval, statement balances, payment plan initiation - well. But she is direct about its limits: "It can't use empathy, it doesn't have really judgment. It doesn't have context. A person can hear confusion. They can hear if the patient is getting frustrated."
Her most quoted line in this conversation captures the governance principle in a single sentence: "You can't outsource accountability to a robot."
Hines is not anti-technology. She is deploying it actively at Essentia Health. Her point is sequencing and scope: automate the transactional layer first, keep humans on judgment and advocacy work, and build the governance infrastructure (auditing, QA, human review, escalation) before scaling
Bucket 1 - Good candidates for AI and automation: - Routing accounts to work queues - Address and demographic updates - Data entry and tick-mark tasks - Statement balance retrieval - 1099 and document requests - Simple status inquiries - Payment plan initiation against pre-approved parameters
Bucket 2 - Requires human judgment (or tightly governed AI with human oversight): - Determining what a patient owes - Setting or negotiating payment plan amounts - Financial assistance and affordability decisions - Handling frustrated, confused, or emotionally distressed patients - Advocacy conversations that require context beyond the script - Escalations involving disputes, hardship, or complex insurance situations