A Pennsylvania lawmaker and emergency physician has introduced a bill that would force healthcare providers to disclose when artificial intelligence is used in patient care and ensure a human clinician remains the final decision-maker. The proposal comes as patients like Harrisburg-area retiree Mark Warble face unexplained prescription changes and repeated charges for unordered supplies - without any right to know whether AI made those calls.
What the bill would require
Current Pennsylvania law does not mandate that patients be told when AI contributes to their diagnosis, treatment, or billing. Rep. Arvind Venkat, an emergency doctor at Allegheny General Hospital and the prime sponsor of House Bill 1663, wants to change that. His legislation would compel insurers and providers to create an internal governance structure around AI use that lets patients and regulators examine how a system arrived at its conclusions and whether it followed privacy, safety, and anti-discrimination rules.
"Right now, patients don't have insight as to when artificial intelligence, which may be even becoming close to autonomous in its use in healthcare, is being used," Venkat said. Venkat uses AI in his own practice but calls the technology "largely a black box in terms of outputs."
Bias, hallucinations, and the danger of rubber-stamp medicine
AI can replicate bias present in its training data and sometimes produces outright fabrications - "hallucinations" that could have severe consequences when the technology flags cancers or recommends care. Erin Gabriel, a spokesperson for the Pennsylvania Health Access Network, said most people remain more comfortable with human oversight. "AI is great at helping, at assisting, but I think anybody who has used AI on their phone for example knows it can make mistakes and frequently does," Gabriel said.
Venkat warned that overreliance on AI could erode clinical judgment, turning practitioners into a perfunctory approval step. "We are at the point or close to the point, where the human is basically just a perfunctory step in this process, simply saying yes yes yes yes, as opposed to actually applying that output to the specific circumstance," he said.
Billing arms race adds cost without patient benefit
AI scribes that handle electronic prescribing, scheduling, and billing are now the most common form of AI in healthcare. While vendors pitch them as tools that free up clinicians, Venkat argued they have not lowered costs. Instead, he sees an escalating "arms race" between insurers and providers, especially around downcoding - when insurers reduce the treatment level and compensation originally determined by a provider.
"The arms race is likely adding expenses to the system without benefits to the patient. There's a larger issue that the public does not trust that any large institution is deploying AI to the benefit of the public. I think the concern, rightly so, is that AI is being deployed largely for financial reasons and for the benefit of the elite and the wealthy," Venkat said.
For billing and coding professionals, the dynamics Venkat describes are already reshaping workflows. Learning how to manage automated claim reviews and defend against inappropriate downcodes is becoming essential, as covered in courses like AI for Medical Billers.
Some clinicians see practical gains - with guardrails
Andrew Lutzkanin, president of the Pennsylvania Academy of Family Physicians, said AI has meaningfully reduced administrative burdens in his primary care practice at Penn State Health. It cuts time spent on note-taking, code selection, and clinical research without sidelining physician judgment.
"As long as it's implemented in a thoughtful way, it doesn't remove the ability from individual physicians for having a final decision-making capacity. I still am the one making the decision. I have to sign off on it," Lutzkanin said. He also stressed that openness with patients is non-negotiable: "It's important that we are open and honest with how we're using the AI tools that we have with our patients. We're dealing with human beings and not everything can fall down to the simplicity of the computer algorithm."
The broader debate over how to integrate AI into clinical settings while protecting patients extends across the field. For a deeper look at implementation and ethics, see AI for Healthcare resources.
Why this matters for healthcare professionals
Transparency requirements, like those proposed in Pennsylvania, would directly affect clinical and administrative workflows. Professionals need to be ready to explain exactly how AI contributed to a decision when a patient asks - and to spot errors that automated systems may introduce. For billers and coders, the shift means auditing AI-driven claims more aggressively to catch downcoding before it cuts reimbursement. For clinicians, the message is clear: AI may speed up documentation and triage, but the final signature still belongs to a human who understands the limits of the machine.
Your membership also unlocks:
