What should you do after an AI data leak?

What should you do in the first hour after an AI data leak?

Stop the account activity, preserve the exact conversation, and rotate every secret that appeared in it. Do not start by deleting the chat. First record the tool, account, timestamp, prompt, attachments, shared links, and exposed data. Then move the incident to one owner who can involve your IT lead, privacy counsel, insurer, and affected client contacts. If you have none of those roles, name the most technical manager available and record the escalation gap.

That order matters. Deleting evidence can make the later assessment harder, while a pasted API key can remain useful to someone else even after the employee closes the browser. I would treat the event as a security incident until the facts say otherwise.

How do you contain the exposed AI conversation?

Ask the employee to stop using the conversation and not edit the prompt. Capture screenshots or export the chat, including the model name, account email, timestamps, files, and any public sharing URL. Save the record in the incident folder with restricted access. The FTC's Data Breach Response guide recommends securing operations and preserving evidence during breach response. NIST's SP 800-61 Rev. 3 gives the broader incident-response structure.

Revoke any shared link. If a password, API key, OAuth token, private certificate, database connection string, or recovery code appeared in the chat, rotate it now through the system that issued it. Do not wait for proof that the model or another person used it. A secret is already compromised when it leaves the controlled system. Check access logs after the rotation and preserve those logs.

Deleting the conversation may be part of the final cleanup, but it is a poor first move. Record first, contain second, delete only after your owner and counsel agree on the evidence plan.

How can you find out what data was exposed?

Make a short inventory from the preserved copy. Record whose data appeared, what business purpose it served, and whether it included customer personal data, employee records, health or payment information, source code, credentials, contract material, or a client's confidential information. Include filenames and the smallest useful description. Do not paste the same material into another AI tool to classify it. That would be a second leak.

Then establish the account facts: consumer or business account, personal or company email, exact product and model, browser extensions, connected storage, retention setting, training setting, memory features, and whether the user created a share link. Vendor labels change, so verify the current setting in the account and the vendor's documentation. OpenAI's data controls FAQ describes controls for ChatGPT, while Google's Gemini Apps Privacy Hub explains how activity and retention work for Gemini Apps. Microsoft documents enterprise protections for Microsoft 365 Copilot.

A business plan may have stronger contractual controls than a consumer plan. It does not make a sensitive prompt harmless. The actual account, settings, files, and contract decide the risk.

When should you notify a regulator, customer, or insurer?

Do not promise that the event is harmless because the AI provider says it does not train on business data. Training is only one question. You still need to assess access, retention, human review, connected services, contractual confidentiality, and the sensitivity of the material.

If personal data is involved, ask privacy counsel or your data-protection officer to assess the applicable law and your role in the processing. Under the GDPR's Article 33, a controller may have to notify a supervisory authority within 72 hours of becoming aware of a reportable personal-data breach. The ICO breach guide explains the risk test and documentation duties. The clock and threshold are legal questions, not a reason to wait for a perfect forensic report.

In the United States, notification rules vary by state and by data type. Sector rules and contracts can add duties. The FTC's Health Breach Notification Rule is one example of a rule that applies to particular businesses and health information. Ask your insurer about notice requirements before making public statements. If a client supplied the data, read the contract and call the client's security contact promptly.

Your incident log should say what happened, when you learned it, what you contained, what you still do not know, who made each decision, and why you concluded that notification was or was not required. That log is useful even when the answer is no notification.

Should you delete the chat or ask the AI company to remove it?

Usually, do both, but only after evidence preservation and an intentional containment decision. Turn off training or activity retention where the product offers that control, remove saved memory and shared links, delete the conversation, and submit the vendor's privacy or deletion request if appropriate. Record the request and any confirmation.

Do not describe deletion as erasure from every backup or operational system. You cannot promise that without a vendor response and a contract that supports it. Also check connected apps. A chat may have pulled from a drive, CRM, browser extension, or custom GPT, and deleting the visible thread may not revoke those permissions.

How do you prevent the next AI data leak?

Start with an approved tools list and a one-page rule: public or approved material may go into approved tools; confidential, personal, regulated, credential, and client-restricted material needs an explicit approved workflow. Give people a safe route for real work, such as a company-managed workspace with SSO and an agreed retention setting. A ban by itself pushes the work into personal accounts.

In the next seven days, I would do four things.

1. Review identity-provider OAuth grants and browser extensions for common AI tools.
2. Put billing and admin ownership on every approved AI account.
3. Run a 20-minute exercise using fake customer data, with one person practising reporting the incident.
4. Add an AI incident field to the same ticketing system used for other security events.

Keep the exercise boring. Someone should know whom to call before the real version arrives in a Friday afternoon inbox.

For a small company, the practical control is clear ownership. Name one person who maintains the registry and one person who can approve a sensitive use case. Review vendor data settings at renewal, because product names and defaults move faster than most policy documents. NIST's AI Risk Management Framework Generative AI Profile is useful background, but you do not need a committee to apply its basic risk questions.

If this happened to your team, document the facts before debating blame. If you want help turning the response into a simple workflow, contact us.