Keeping private documents private in RAG and AI search tools
Make sure your AI search and RAG assistants only show each employee the documents they are already allowed to see.
The new internal assistant can answer questions across Google Drive, SharePoint and the ticketing system. In the first week, a junior employee asks about bonus plans and gets a neat summary of a compensation file shared with leadership only. The document was never opened directly. The AI simply found it, read it and helpfully explained it.
This happens when documents are copied into an AI index without their access rules. The original system knew who could open each file, but the search layer does not. Over-shared folders, old links and stale permissions, which were mostly harmless before, become easy to find once an assistant can search everything in seconds.
Can we just tell the AI not to reveal confidential files?
No. Instructions in the prompt are not access control and can be bypassed or ignored. Restricted documents should be filtered out before the model ever receives them, based on the user's real permissions.
Should we clean up file sharing before launching AI search?
Yes. AI search makes existing over-sharing easy to discover. Review public links, all-company folders and old shares for sensitive content first, and fix them in the source system, not only in the index.
Is a vector database itself sensitive?
Yes. Embeddings and stored text chunks can reveal document content, so protect the index with the same access controls, encryption and retention rules as the original documents.