Every company we work with has a folder of standard operating procedures. Almost none of them can tell you whether the team knows what is in it. The document exists, someone signed that they read it, and six weeks later a customer gets the wrong answer anyway.
AI Mentor is what we built after accepting that the document is not the problem. The problem is that reading something once does not put it in anyone's head.
Retrieval, with the receipt attached
The first job is answering questions from the procedures rather than from the model's general knowledge. Documents are split into knowledge items, embedded with pgvector, and retrieved as context at answer time.
The detail that matters is the citation. An answer that says "escalate to tier two after 30 minutes" is useless on its own, because the person reading it cannot tell whether that number came from the SOP or from the model's sense of what a support policy usually looks like. Every answer carries the paragraph it came from. When the model has nothing to cite, the correct behaviour is to say the procedure does not cover it — which is itself a useful finding, because it means someone needs to write that paragraph.
Assessment generated from the same source
A procedure that has just been written is not yet training material. Someone has to turn it into questions, and that someone never has time.
So the questions are generated from the document: quiz items and flashcards, tied back to the knowledge items they came from. A new procedure becomes trainable the day it is published. The generation is not magic and it is not always right — it is reviewable, and a bad question is edited rather than accepted.
The part that actually makes it stick
This is where most internal training tools stop, and it is the reason most of them do not work. A quiz measures whether someone knew a thing today. It does nothing about the fact that they will not know it in three weeks.
Scheduling is done by FSRS. Each card has its own difficulty and stability, and the next review date is computed per person per card from their answer history. Someone who finds the refund policy obvious sees it rarely. Someone who keeps getting it wrong sees it often, and sees it at the point they are about to forget it rather than on a fixed weekly cadence that wastes their time on what they already know.
A fixed schedule optimises for the person who administers the training, not the person taking it. It shows everyone the same card on the same day regardless of whether they have known it for a year. FSRS models forgetting per person per item, so review time goes where forgetting is actually happening.
Practice is not a quiz
Knowing a procedure and applying it under pressure are different skills, and multiple choice only measures the first. The practice mode puts the trainee in a conversation with a difficult customer and scores the handling against the procedure - whether they verified identity before discussing the account, whether they offered the remedy the policy allows rather than the one the customer demanded.
The report that changes the document
The analytics are per person and per topic - a forgetting curve, retention statistics, which cards keep failing. The most useful output is not the ranking of employees. It is the list of procedures the whole team gets wrong.
When eight people out of ten fail the same item, the staff is not the problem. The paragraph is ambiguous, or it contradicts another paragraph, or it describes a process that no longer matches what the team actually does. That is a document defect, and it is invisible until you measure comprehension instead of attendance.
Where the model runs
The LLM provider is configuration: OpenAI, Gemini, or a self-hosted Ollama or vLLM endpoint. For a company whose procedures are the business - underwriting rules, pricing logic, incident response - the ability to keep the whole loop inside their own network is not a preference, it is the condition for using the system at all.