Algeria AgencyArtificial intelligence
RAG without the jargon: what an assistant on your documents still invents
Connecting a model to your documents does not make it honest. It changes what it invents, and moves the problem to what you give it to read.
Three letters have been in every proposal for two years: RAG. You will be told they solve the invented-answer problem, and that is neither true nor false — it is true of half the problem and false of the other half, and the remaining half is the one that will cost you time.
The mechanism fits in one sentence and needs no technical knowledge: before answering, the system fetches a few passages from your own documents and pastes them into the question. The model no longer answers from memory, it answers with the text in front of it.
This article explains what that gesture fixes, what it does not, and the work it transfers to you. That work is real, it is yours, it has nothing to do with the technology you buy, and it is almost always what decides whether the project holds.
What a model does when it does not know
A language model does not look for the truth, it looks for the plausible continuation. That is its nature and not a manufacturing fault: it was taught to produce text resembling correct text, and resemblance is exactly what it optimises.
The consequence is that it does not know the state "I do not know" the way a person does. Faced with a question about your opening hours, your returns policy or the price of a service, it produces an answer shaped like an answer on those subjects. The shape is right, the content comes from nowhere.
That is why a model’s falseness is hard to spot: it carries none of the signs we associate with error. No hesitation, no vagueness, no strange phrasing. A round number, a clean sentence, a confident tone.
Everything that follows comes from there. The problem is not that the machine is ignorant of something, it is that it cannot distinguish ignorance from knowledge unaided, and the only possible help is to put the text in front of it.
The answer fits in one sentence
Before answering, the system searches your documents for the few passages most like the question, and places them into the request sent to the model. The model writes from those passages rather than from what it retained of the internet.
That does not rule out training, which solves a different problem: the three cases where training a model is worth it are all about the shape of answers, never about their accuracy.
Nothing more. There is no training, no model "trained on your data", no transfer of your documents into anybody’s weights. Your texts stay texts, in a database of yours, consulted on every question.
That simplicity has a verification virtue: on every answer, you can ask to see the passages that were used. A system that cannot show them is not doing what is described here, whatever word appears in the quote.
It also has an economic consequence rarely stated: it is the cheapest of the three arrangements you will be offered, because it demands no heavy computation. What it demands is tidying, and tidying is paid for in your team’s hours rather than in hard currency.
The split decides what can be found
Your documents are not sent whole: they are cut into pieces of a few paragraphs, and it is the piece that is retrieved, not the document. Where you cut is therefore a decision, and it is the first in the chain.
Cut too short, a passage loses its context: the sentence "this period is extended to thirty days" no longer says which period, and it will be retrieved for a question it does not answer. Cut too long, a passage carries three subjects, and the model picks one.
The case that breaks most often is the table. A price list, a scale, a timetable lose their header in the split, and what is left is a column of numbers nothing attaches to what they measure. A system that answers your tables badly does not understand less well: it received less.
There is no universally good split, which is why "how do you split?" is a useful question in a meeting. The honest answer describes your documents — your contracts, your sheets, your notes — and not a default value.
Retrieving is not understanding
Two ways of retrieving coexist and do not find the same things. The first compares words: it is unbeatable on a reference, a proper noun, an article number. The second compares meaning, representing each passage as a series of numbers: it retrieves "how do I cancel" from "I want to stop my subscription".
Each misses what the other finds. Words alone fail as soon as the reader uses different vocabulary from yours — which is the rule, not the exception. Meaning alone fails on a reference, because two neighbouring codes look very alike and denote nothing close.
A serious system does both and merges the two lists. That is an implementation detail, except on one point that concerns you: the "meaning" part has a preparation cost, each passage having to be converted once, and that conversion is redone every time the text changes.
A tuning decision hides behind all of this and nobody mentions it: how many passages to return. Too few and the answer misses a piece that existed; too many and the model receives noise it will treat as signal, because nothing inside a passage says it is off topic. That number is set by trying, on your questions, and it has no correct default.
In Arabic, the first method suffers an extra problem nobody mentions before go-live: the definite article attached to the word, possessive suffixes and optional vowels mean one notion is written several ways. An engine that does not handle them retrieves one passage in five, and the symptom is an assistant that "works in French".
What is not retrieved does not exist
The quality of the answer is capped by the quality of the search, and that cap is absolute. If the right passage is not among the few pieces returned, no model, however recent, can make up for it — it will write very well from the wrong ones.
That is why changing model is rarely the right move when an assistant answers badly. The model is the last step and the most visible; the search is the step before and the only one that decides what it has in front of it.
The diagnosis takes two minutes and needs no technical access. Ask the question, ask to see the retrieved passages, and read them. If they contain the answer and the answer is wrong, the problem is the model. If they do not, any tuning of the model is time lost.
That distinction is worth demanding in writing in a quote: when the assistant gets something wrong, who looks at what, and in what order. A supplier who answers "we will improve the prompt" to every failure has no method.
Silence is an answer, and it has to be allowed
An assistant that finds nothing must be able to say so. That seems obvious and it is the most frequently absent setting, because a system that always answers demonstrates better than a system that admits a limit.
The refusal has a wording that matters. "I did not find this information in the documents available to me" is correct. "We do not offer that service" is an invention of a graver kind, because it denies something instead of being ignorant of it, and it is delivered with the same confidence.
That confusion happened to us on our own assistant, on a question the corpus did not yet answer, and the reply denied a service the company sells. Nothing was broken: the search had returned little and the model filled the gap in the most natural direction.
The rule that came out of it fits on one line, and it holds for any assistant: finding nothing is not evidence of absence. It has to be written into the system’s instructions, and checkable by you by asking a question whose answer you know is nowhere.
A citation proves less than you think
An assistant that cites its sources inspires confidence, and deservedly: without citations, verifying an answer means finding the document yourself, which nobody does twice.
But a citation proves one thing — that this passage was retrieved. It does not prove that it says what the answer claims, nor that it is the right passage, nor that it is current. A correct link under a false sentence is the hardest defect to see, because the link’s presence stops the checking.
There is a design consequence you can judge without being a technician: sources must be assembled by the system from the passages actually used, never written by the model. A model asked to cite produces references the same way it produces everything else, which is to say plausibly.
The test is one question: open the cited source and look for the sentence. On a correct system you find it every time. On a system where the model writes its own references, you often find the right document and never the right sentence.
What this arrangement does not prevent
Three failures survive the connection to your documents, and they are the more awkward because the system looks reliable everywhere else.
The first is internal contradiction. Two documents say different things — a 2023 note and a 2026 procedure — and the search returns both. The model picks one, without knowing which is authoritative, and it will not tell you it decided.
The second is multi-step reasoning. "Is this client entitled to the discount?" requires reading the scale, then the contract, then drawing a conclusion. Search is good at finding a passage and bad at finding a chain, and the missing link gets filled in.
The third is the question whose answer is a calculation or a state. A balance, an availability, a period under way: these are living data no document contains. An assistant answering those from a corpus of texts is answering from a photograph, and nobody told it when it was taken.
The corpus is a business job
The part nobody sells is the one that decides the result: gathering the documents, naming them, saying which are authoritative, and throwing out the expired ones. That work is not technical and cannot be subcontracted, because it requires knowing which of two versions is right.
In most businesses we see, that corpus does not exist in usable form. There is a shared inbox, a few binders, a folder of PDFs sent by suppliers, and the real knowledge is in two people’s heads. An assistant cannot read a head.
The good news is that the first sort is short and is done without us: the twenty questions your customers ask most, and the document answering each. When a question has no document, you have just found something more useful than an assistant.
The less good news is that the sort has an owner, and a corpus with no owner degrades at the speed of your business. It is the same observation as for any installation: what nobody watches stops being true without warning.
Updating has no symptom at all
The day your returns policy changes, the page changes and the corpus does not. The assistant carries on answering, confidently, citing a source that still exists, and the answer has become false without any alert firing.
It is the arrangement’s most dangerous failure mode, because it resembles normal operation in every respect. Nothing is broken, nothing is empty, nothing returns an error: the assistant has simply stayed in last month.
The remedy is dull and sufficient. The corpus is rebuilt on every publication, automatically, and a table somewhere gives the date of the last rebuild. We apply it to ourselves, and we were caught once: a daily rebuild had been failing for four days with nothing to signal it, because the job finished "successfully" having done nothing.
The lesson transfers unchanged. Do not ask whether the update is automatic, ask what will warn you on the day it stops being, and where you can read the date of the last one.
The check: twenty questions, one hour
Write down the twenty questions your customers actually ask, taken from your messages rather than from memory. Add five whose answer you know is written nowhere.
Put the twenty-five to the assistant you are being offered. For each, note three things: is the answer right, can the source be opened, and is the quoted sentence really in the source. Three columns, a tick or nothing.
The five unanswerable questions are the most instructive and are the ones nobody asks in a demonstration. The system has to say it does not know. If it answers anyway, you know its behaviour on the day a customer asks for something you have not written down.
That sheet fits on one page, fills in within an hour, and is redone identically six months later. It is the only quality measure of this kind of system that depends on no promise, and it combines with the sort by family the reader may already have done.
What we do, and what we refuse
We gather the corpus with you before connecting anything: which documents go in, which is authoritative when two disagree, and who keeps them current. We then measure the twenty-five questions and hand you the table filled in, including the rows where the system fails.
We refuse to deliver an assistant that cannot show its passages. Without that there is no way — for you or for us — to tell a grounded answer from an invented one, and the reliability promise becomes unverifiable.
We also refuse to connect an assistant to a corpus nobody at your end is responsible for. That is not a methodological requirement: it is what separates a system that stays true from a system that was true on delivery day.
What you can do without us is the corpus sort, and it is the part that counts. Many businesses discover while doing it that their answers are good and unfindable, which is a search problem and not an artificial-intelligence one.
Frequently asked questions
Do we need to train a model on our data?
Almost never, and it is the most frequent question. Connecting search to your documents gives a better result at an incomparable cost, and above all it corrects: change a document and the answer changes the same day, whereas a trained model keeps what it learned.
Do our documents go to the model’s supplier?
The retrieved passages do, because they are in the question. The rest of the corpus does not move. That is an important distinction for a contract or a client file: what is sent is what the search judged relevant, and that can be logged.
How many documents do we need to start?
Far fewer than people think. Twenty pages answering the twenty real questions beat three hundred pages where nobody knows which is authoritative. Volume only becomes a subject long after go-live.
Does it work in Arabic and darija?
It works, provided the search treats Arabic for what it is: attached definite article, suffixes, optional vowels. An engine that does not retrieves a fraction of the passages, and the misleading symptom is an assistant that seems good in French and mediocre in Arabic.
Can we tell when the assistant invented something?
Not from the inside, which is why sources matter. What you can do is check by sample: ten answers a quarter, source opened, sentence looked for. It is little work and it is the only honest measure available.
Should the corpus contain our prices?
That is a commercial decision before it is a technical one. A price in the corpus will be quoted, and quoted long after it changed if nobody rebuilds. For that reason we generally advise keeping prices out of the corpus and pointing to the person who gives them.
Where we come in
A question on your list with no document facing it is worth more, at this stage, than a quote for an assistant.
- We sort your documents with you and write down, for each contradictory pair, which one wins.
- The twenty-five questions run in front of you and the table leaves with you, failures included.
- The date of the last index is displayed where somebody will see it without looking for it.
While no named person keeps your documents current, we will connect nothing to them.
Read next
Site search: the cost of zero results
The least-opened report in your shop is your best demand signal. What it contains, and how much of it is really an engine problem.Your engine will never be better than your catalogue
When search finds nothing, the cause is nearly always in the product records. The weekly work that prevents it.The words transcription does not know
The voice agent understands the sentence and gets the name wrong. That is the opposite of what people fear, and it is what makes an appointment unusable.
Let us talk about your project
A free audit, no commitment: we look at your online presence and tell you what is holding it back.