Retrieval augmented generation

Start with what it is not. OWASP describes retrieval augmented generation as a model adaptation technique that improves the performance and contextual relevance of responses by combining pre-trained language models with external knowledge sources. The model is not retrained. Your documents are converted into a form the system can search, and the relevant pieces are pulled in at the moment a question is asked.

That distinction decides which questions matter. Nothing has been baked into a model somewhere. There is a store holding your material, and the real subject is who is allowed to read it.

OWASP names three ways that goes wrong, and the first is the one to raise with any vendor selling a shared product. In multi-tenant environments where several classes of users or applications share the same vector database, there is a risk of context leakage between users or queries. Second, attackers can invert embeddings and recover significant amounts of source information, so the common reassurance that the store holds only mathematical representations rather than your documents is not by itself a guarantee. Third, poisoned data can arrive from insiders, prompts, data seeding or unverified data providers.

There is also an effect nobody warns you about. OWASP notes that retrieval augmentation can alter the underlying model's behavior, and that while factual accuracy and relevance may increase, qualities like emotional intelligence or empathy can diminish. An assistant that becomes more correct can become worse at handling an upset customer.

NIST supplies the standard the store has to meet. It says systems that maintain confidentiality, integrity and availability through protection mechanisms preventing unauthorized access and use may be said to be secure, and lists among common security concerns the exfiltration of models, training data or other intellectual property through AI system endpoints. The endpoint is the assistant itself.