Doctors Are Using ChatGPT With Patient Data: What HIPAA Means for AI Data Security
Generative AI is becoming part of everyday healthcare workflows. Physicians and healthcare professionals are using AI tools to summarize information, create documentation, analyze content, and support administrative and clinical tasks.
But when patient information enters an AI system, a critical question follows:
What happens to that patient data after it is submitted to the AI tool?
A recent article from Law Commentary highlights the growing privacy implications for doctors who use ChatGPT with patient records. The article explains that healthcare providers cannot simply assume that an AI tool is appropriate for handling protected health information (PHI) because it is useful for clinical or administrative work.
For healthcare organizations adopting generative AI, the issue goes beyond choosing an AI model. Organizations also need to understand what information employees are entering into AI systems, what protections are applied to that information, and whether the AI environment is appropriate for the data being processed.
Why Patient Data and ChatGPT Require Careful Consideration
Healthcare organizations handle some of the most sensitive information about individuals, including medical histories, diagnoses, laboratory results, medications, treatment information, insurance information, and other identifying details.
When identifiable health information is entered into an external AI service on behalf of a healthcare provider, HIPAA requirements may apply.
The Law Commentary article notes that healthcare providers may need a Business Associate Agreement (BAA) when an outside service creates, receives, maintains, or transmits protected health information on the provider’s behalf. It also distinguishes between regulated clinical AI environments and ordinary consumer AI accounts.
That distinction is important.
An employee using an AI tool for work is not necessarily operating in the same privacy environment as an individual using an AI application for personal purposes.
HIPAA Compliance Is More Than Choosing a "Secure" AI Tool
One common misconception is that selecting an AI provider that offers security features automatically solves the healthcare organization’s data protection responsibilities.
It does not.
The organization still needs to understand its own data flows and risks.
The U.S. Department of Health and Human Services (HHS) explains that the HIPAA Security Rule requires regulated entities to perform an accurate and thorough assessment of potential risks and vulnerabilities affecting the confidentiality, integrity, and availability of electronic protected health information (ePHI).
This means organizations should consider questions such as:
- What patient information is being entered into AI tools?
- Which employees or teams are using AI?
- Which AI applications are being used?
- Is the AI application approved by the organization?
- Where does the information go?
- Is the information transmitted or stored by a third party?
- What security controls are applied?
- Are sensitive data flows being monitored?
- Are employees unintentionally submitting PHI to consumer AI applications?
- What happens when employees copy and paste information into an AI prompt?
The Shadow AI Problem in Healthcare
There is another challenge that healthcare organizations need to consider: Shadow AI.
Shadow AI occurs when employees use AI applications that have not been formally approved, evaluated, or governed by their organization.
For example, an employee might copy information from a patient record into an AI chatbot because they want to:
- Summarize a long document
- Rewrite clinical notes
- Generate an email
- Analyze a report
- Extract information from a document
- Create administrative content
But from a security perspective, the data has still left the organization’s controlled workflow.
This creates a gap between AI adoption and AI governance.
The organization may have an approved AI policy, while employees are simultaneously using dozens of external AI tools that security teams cannot easily see or control.
Data Protection Needs to Happen Before Sensitive Information Reaches AI
One practical approach is to introduce data protection controls into the workflow itself.
Instead of relying exclusively on employees to determine whether information is safe to submit, organizations can use technology to help identify and protect sensitive information before it reaches an AI system.
Depending on the use case, this can involve:
Detection
Identifying sensitive information such as names, addresses, identification numbers, medical information, or other confidential data.
Masking
Replacing sensitive values with protected or anonymized representations before information is submitted to an AI system.
Redaction
Removing sensitive information from documents and files before they are shared or processed.
Monitoring
Improving visibility into how AI tools are being used and where sensitive information may be entering AI workflows.
Policy enforcement
Applying organizational controls around which information can be submitted to AI applications.
This approach creates another layer of protection between sensitive organizational data and external AI systems.
De-identification Can Change the Privacy Equation
Not every use of healthcare information in an AI workflow is necessarily the same.
The Law Commentary article notes that properly de-identified health information is treated differently under HIPAA because it is no longer considered PHI under the Privacy Rule.
However, organizations should not assume that simply removing a name makes information de-identified.
Healthcare organizations need to evaluate whether their data has actually been de-identified according to applicable HIPAA standards and should consider the possibility of re-identification based on the information being processed.
This is another reason why data classification and appropriate security controls are important before healthcare information enters an AI workflow.
AI Adoption and Patient Privacy Don't Have to Be Opposing Goals
The answer is not necessarily to prevent healthcare professionals from using AI.m
AI can provide meaningful productivity benefits when implemented appropriately. It can help professionals summarize information, organize documentation, reduce administrative workload, and support other workflows.
The challenge is enabling these benefits while maintaining appropriate controls around sensitive information.
HHS emphasizes that HIPAA compliance involves risk analysis, reasonable and appropriate security measures, and ongoing evaluation rather than a one-time compliance exercise.
That principle is particularly relevant to AI because the technology landscape changes quickly.
New AI applications appear frequently. Employees discover new tools. Organizations introduce AI agents and integrations. Existing applications add new capabilities.
Security controls therefore need to evolve alongside AI adoption.
A Practical AI Data Security Checklist for Healthcare Organizations
Before allowing employees to use generative AI with healthcare information, organizations should consider the following:
1. Identify approved AI tools
Maintain an inventory of AI applications that employees are authorized to use.2. Classify sensitive information
Clearly identify PHI, ePHI, confidential information, credentials, financial information, intellectual property, and other sensitive data categories.3. Understand AI data flows
Determine what information is being sent to AI applications and where that information is processed or stored.4. Establish AI usage policies
Define what employees can and cannot enter into AI systems.5. Protect sensitive information before submission
Use appropriate controls such as detection, masking, redaction, or other data-loss-prevention mechanisms6. Monitor AI usage
Where appropriate, maintain visibility into AI applications and sensitive-data interactions.7. Review third-party requirements
Organizations should determine whether their use of an AI provider requires contractual protections such as a Business Associate Agreement.8. Conduct ongoing risk assessments
HHS describes risk analysis as a foundational component of HIPAA Security Rule compliance and recommends that organizations understand their systems, data, threats, vulnerabilities, and existing safeguards.How SecurePledge Fits Into the AI Data Security Conversation
SecurePledge is AI data security software developed by StrategyWiz Consulting.
Its purpose is to help organizations protect sensitive information across AI interactions and digital workflows.
The SecurePledge ecosystem includes:
- Masker for AI data protection and data loss prevention
- Redactor for sensitive document and PDF redaction
- Protector for protection across browser and desktop AI usage
For healthcare organizations, technologies such as these can form part of a broader AI data security strategy by helping reduce the risk of sensitive information being exposed through AI workflows.
SecurePledge should not be viewed as a replacement for HIPAA compliance, organizational policies, legal advice, or a comprehensive security program. Rather, it can be considered as part of the technical controls organizations may evaluate when developing their approach to AI data protection.
The Bigger Question: Can Healthcare Adopt AI Without Losing Control of Its Data?
Healthcare organizations are likely to continue expanding their use of generative AI.
The more important question is how they will govern that adoption.
A healthcare organization needs visibility into where sensitive information is going, controls over what employees can share with AI systems, and safeguards that reduce the likelihood of accidental disclosure.
HIPAA provides an important regulatory framework, but responsible AI adoption also requires organizations to understand their technology environment and continuously assess their risks.
The goal should not be to choose between AI innovation and patient data protection.
It should be to build workflows where both can exist together.
Sources
- Law Commentary: Doctors Using ChatGPT With Patient Records Face HIPAA Privacy Rules
- U.S. Department of Health & Human Services: Summary of the HIPAA Security Rule
- U.S. Department of Health & Human Services: Guidance on Risk Analysis
- U.S. Department of Health & Human Services: HIPAA Security Rule Compliance FAQ
Disclaimer: This article is provided for general informational purposes and does not constitute legal or compliance advice. Healthcare organizations should consult qualified legal, privacy, and security professionals regarding their specific HIPAA obligations and AI deployments.