No. The moment someone pastes a tax return or bank statement into ChatGPT, every name, SSN, and account number in it leaves your control and is stored on OpenAI’s servers. On free and Plus plans it can also be used to train future models. Deleting the chat doesn’t guarantee it’s gone, and a client’s Social Security number can never be changed. If it leaks, it leaks for life, and the breach is yours to report.
This has already gone wrong
- Samsung, 2023. Engineers pasted confidential source code into ChatGPT. Samsung couldn’t get it back and banned generative AI company-wide.
- Deleted chats, 2025. In the New York Times lawsuit, a federal court ordered OpenAI to preserve ChatGPT logs, including chats users had deleted.
- Google search results, 2025. Shared ChatGPT conversations, some containing personal details, were indexed by Google and other search engines where anyone could find them.
None of these needed a hacker. Each one started with a person typing something into a chat box.
What it can cost you
- Tax preparers: disclosing return information without consent is a federal crime under Section 7216, with up to a year in prison per violation.
- Healthcare: pasting patient records into a tool without a signed HIPAA business associate agreement is a HIPAA violation.
- Everyone: client contracts, state breach notification laws, and, for financial firms, the FTC Safeguards Rule. Explaining to a client that their SSN went to an AI company costs you the client.
“But we’re on a business plan”
| Free and Plus | Business, Enterprise, API | |
|---|---|---|
| Trains on your chats | Yes, unless you turn it off in Data Controls | No, by default |
| Stored on OpenAI’s servers | Yes | Yes, under business terms |
| HIPAA agreement available | No | Some plans, by request |
Source: OpenAI’s business data page and ChatGPT’s Data Controls settings. Terms change; check them before relying on this table.
Not training on your data is different from not having it. On every plan the file sits on someone else’s servers, under their retention rules, within reach of court orders and breaches. And the plan your firm pays for doesn’t cover the employee who uses a personal ChatGPT account on their phone because it’s faster.
How to use ChatGPT without sending PII
Swap the personal details for placeholders before the file leaves
your office, and swap them back in the answer. “Jane Smith”
becomes PERSON_1, her SSN becomes
SSN_1, and ChatGPT still sees the balances, dates,
and line items it needs to do the work.
Doing that by hand is slow and easy to get wrong. Our PII Cleanser does it automatically on a small computer in your office, for Claude and ChatGPT alike. If a file can’t leave in any form, a private AI server runs the model entirely in your office instead.
See what ChatGPT would see
Bring a few sample files with fake names and numbers. We’ll show you what the Cleanser removes and what comes back.
This page is general information, not legal or tax advice. ChatGPT and OpenAI are trademarks of OpenAI; On Premises is not affiliated with OpenAI. See our Terms for how site content should be read.
Last updated: October 23, 2026