Everything in the context,
then RAG

Youth AI Lab · Workshop 2 of 6
Build your own chatbot

❓ The question of the series

A chatbot only answers from what it reads before it answers. Every workshop of the series is one answer to the same question.

What enters the model's context, when, and who decides?
📥
What?
rules, documents, search results, live data
⏱️
When?
always, at startup, or for each question
🧑‍⚖️
Who decides?
you, a search engine, or the model itself

🧭 Today: who picks the pieces?

🎭
1. Instructions
📚
2. Knowledge, RAG
today
🔎
3. Search, skills
📡
4. Tools
🗺️
5. Wiki
🧪
6. Fine-tuning
Mode B: everything enters the context, every time. Nobody picks.
RAG: a search engine picks a few pieces for each question, before the model.

🗓️ The plan: 2 hours

👋
Welcome
📚
Everything
✂️
Unplugged
🧩
Pieces
🎛️
Settings
⚖️
RAG vs B
🏁
Closing
25 min
mode B
15 min
paper
15 min
query
20 min
size, top-k
20 min
columns
10 min
10 min
📄 Follow on your screen: the README.md of your Codespace, or the one-page sheet.

💻 Your workspace: one Codespace per pair

A Codespace is VS Code in your browser: nothing to install. Everything you need is already inside.

⏱️ Before you open it, once per GitHub account: github.com/settings/codespaces → Default idle timeout → 120 → Save. It only applies to the Codespaces you create after.

1Open the Codespace link of the workshop, at the top of its README
2In the terminal, type workshop-login and paste the key of your pair
3Wait for Key OK, then type workshop bare. First time: choose a theme, then ↓ to Yes, I trust this folder (Enter alone exits). To leave: Ctrl+C twice
4Then type workshop-start and the number of the workshop. With a save from last time, add --from my-work.zip
⚠️ The key is not shown while you paste it: that is normal. If the Codespace stops after a long pause, reopen it: nothing is lost.

💻 No Codespace? The same repository opens on your own computer: dev containers guide, Part 3.

🐳 Open WebUI starts by itself

1Drag my-work.zip into the file list.
2workshop-start 2 --from my-work.zip: your work is back, and Open WebUI starts in the background.
3Second terminal: tail -f .workshop/openwebui-start.log. No need to wait: it must say READY before Part 4 (about 4 to 5 minutes the first time). Ctrl+C stops following, not Open WebUI. Other commands: first terminal.
🚫 Never make port 3000 public. This Open WebUI has no accounts: anyone with the link could use your key.

📍 Question 12: your territory

1Read the section Question 12: your territory of test-questions.md.
2Write the question, the answer and the address of the official page in row 12.
3replay bare 12, then replay A 12.
4score bare 12 --blind, then score A 12 --blind: your score first, then the judge's.
⏱️ 5 minutes. With row 12, a complete column is out of 24. In workshop 3, its page enters your chatbot.

📚 Everything in the context

What does reading everything cost?

15 min · mode B

🌡️ The gauge: how full is the context?

The context is everything the model reads before it answers. Its maximum size is the context window. A line above the input box shows how full it is.

🪟
200,000 tokens
the size of the window (a token is a piece of a word)
💶
Every message pays
for everything in the window
🤝
Same model for all
so that the pairs can compare
👁️ Watch the gauge at every change: it shows what entered the context.

📚 Mode B: nobody picks

A
your AGENTS.md
B
37 files
about 100,000 tokens: half
✅ The model can find any fact, and see two documents that disagree.
💶 Every message pays for all of it, again.

📏 How to score an answer

Each answer gets 0, 1 or 2. The score rewards honesty as much as knowledge.

2
Right, and it says where it comes from. Or "I don't know" when nobody knows.
1
Honest but not enough: no source, only partly right, or "I don't know" although the answer exists.
0
Wrong or invented.
💡 An invented answer is worse than "I don't know".

🎬 Your turn: mode B

1workshop B: ask question 10, then look at the gauge. Ctrl+C twice.
2Predict: which questions go up from A to B?
3replay B, then score B 3 7 10: the judge proposes, you accept (Enter) or correct. Time left? score B 1 2 4 5 6 8 9 11 12.
❓ The kit fills half of the window. And with ten times more documents?

🪟 Three times the kit?

The window: 2 m of paper, 1 cm = 1,000 tokens
kit
kit
kit: does not fit
200,000 tokens ▲
❯ How many hectares burnt outside the EU in 2025?
  ⎿  Prompt is too long · the request is ~237206 tokens (limit 200000) …
     A single-exchange conversation cannot be compacted …
🚫 No answer at all: the model read nothing.
❓ So who chooses what enters?

✂️ Unplugged: be the search engine

Pick 3 pieces. Answer with them only.

15 min · paper strips · two pairs together

✂️ How it works

🔎 Pair 1: the search engine

Reads the question card, looks through the strips, and passes 3 strips (round 1) or 2 strips (round 2). It must always pass strips.
🤖 Pair 2: the model

Answers the question with the strips it received, and nothing else: not its memory, not the other strips.
Round 1: one sentence per strip, question 10, then question 7. Round 2: one paragraph per strip, question 10, roles swapped.
❓ How did the search engine choose: by the words of the question, or by their meaning?

🧩 Pieces and embeddings

What does the RAG give the model?

20 min · openwebui query · no model, no cost

🧩 A RAG in three words

✂️
Chunk
a piece of a document, cut before any question (here, at most 1,000 characters)
🔢
Embedding
a list of numbers for a text: close meanings, close numbers
🔝
Top-k
the number of pieces the model receives (here, 5)
💡 The search works before the model. The model never sees the rest.

📐 How two texts are compared

Text AText B

Each text is an arrow. The angle between two arrows says how close they are.

Text A (0.6, 0.8), text B (0.8, 0.6): 0.48 + 0.48 = 0.96

The two arrowsAngleCosine (the sum)Score
Same direction0°11
Text A and text B (made up)16°0.960.98
wildfire and fire in the woods (measured)51°0.6250.813
Right angle: no link90°00.5
Opposite180°-10

🔢 Predict: which pairs are close?

Each text gets a list of 384 numbers: its place on a map of meanings. Near 1: same direction. Around 0.5: no link for the small model.

  • wildfire and forest fire
  • wildfire and campfire
  • wildfire and birthday cake
  • wildfire and incendie de forêt
  • burnt outside the EU and burnt beyond the EU
  • Access to the forest is forbidden. and Access to the forest is allowed.

🔢 What the small model says

Text 1 Text 2 Score
Access to the forest is forbidden. Access to the forest is allowed. 0.964
burnt outside the EU burnt beyond the EU 0.944
wildfire campfire 0.865
wildfire forest fire 0.844
wildfire birthday cake 0.618
wildfire incendie de forêt 0.559
💡 The search finds what a text is about, not whether it is true or whether it answers you.

🔎 One question, from start to end

Before: 37 files, 326 pieces, 384 numbers for each piece. Done once.
1. Question 10 becomes 384 numbers.
2. Each of the 326 pieces gets a score: from 0.861 down to 0.490.
3. Sort, cut after 5. Summary 90 (2.2 million) is 5th (0.847). The 1 092 095 hectares are 7th (0.832): out.
4. Open WebUI pastes the 5 excerpts into the message, in front of your question.
5. The model reads this message and writes. It searched nothing.
💡 Retrieval (1 to 3), Augmented (4), Generation (5).

🎬 Your turn: read the pieces

1openwebui query "How many hectares burnt outside the EU in 2025?"
2Do summary 10 (1 092 095 hectares) and summary 90 (2.2 million) come back? Look at the score of each excerpt.
3The same for question 7 (Campania 2025) and question 3 (Bouches-du-Rhône).
We measured (pieces of 1000, top-k 5)
Q10 the 2.2 million, not the 1 092 095
Q7 5 excerpts, although nothing answers
Q3 the first page of document 43 and a summary about Charente-Maritime, never the articles that answer

🎛️ Change the settings

Bigger pieces, or more pieces?

20 min · --size · --top-k · Open WebUI

🎛️ What the pieces contain

Bigger pieces
openwebui query --size 2000 "How many hectares burnt outside the EU in 2025?"

Question 10: both figures come back.
More pieces
openwebui query --size 1000 --top-k 10 "How many hectares burnt outside the EU in 2025?"

Question 10: both figures come back too.
Both send more text to the model at every question. Question 3 is still missing.
💡 What matters is what the pieces contain.

⚖️ Question 10, two sizes, one model

Wildfire RAG 1000 · judge: 1
"2 242 195 − 1 079 538 = about 1 162 657 hectares" · "my calculation, not a figure from the documents" · "I can't tell which reading is correct from the excerpts alone."
Wildfire RAG 2000 · judge: 2
"Non-EU European countries: 1,092,095 hectares" · "Middle East and North Africa: 70,562 hectares" · "The figures add up only if the 2.2 million includes the EU."
🎬 In Open WebUI: question 10 to each model, then click the sources. Optional: openwebui top-k 10, question 10 again to Wildfire RAG 1000, then openwebui top-k 5: replay RAG needs 5.

⚖️ RAG against everything

Same questions, another way to read

10 min · replay RAG · score RAG 3 7 10

🎬 Your turn: column RAG

1replay RAG: each question to "Wildfire RAG (workshop kit)", the same knowledge base as Wildfire RAG 1000 (pieces of 1000), top-k 5.
2score RAG 3 7 10: a column RAG appears after B in scores.md. Time left? score RAG 1 2 4 5 6 8 9 11 12.
3Compare rows 3, 7 and 10 of columns B and RAG, and the costs. Totals only if both columns are complete.
We measured Cost of 11 questions Judge, 11 questions (3 runs)
Mode B $0.47 TO BE COMPLETED
RAG, pieces of 1000 about $0.14 (estimate) 18 to 19/22
RAG, pieces of 2000 about $0.16 (estimate) 16 to 19/22

🏁 When to choose a RAG

Everything in the context (B) RAG
Who picks nobody: everything enters a search engine, before the model
Size of the knowledge limited by the window much bigger
Cost of a question everything, every time only the pieces
Its failure stops when the knowledge grows always gives something: the failure is invisible
💡 The test questions show the failures that the chat does not show.

🔀 Some chat tools switch on their own

Claude Projects (paid plans)

When the knowledge "approaches the context window limit, Claude will automatically enable RAG mode", with "a visual indicator". You cannot choose.
Gemini

Going beyond the window "could lead to responses that don't take into account all the content provided". No switch to a search is documented.
📦 The same Knowledge box can read everything, search for pieces, or miss part of it.

💾 Your chatbot as a RAG, then save

🎬 Second terminal: openwebui chatbot --name "<your chatbot>" --rag, then <your chatbot> chatbot (RAG) in Open WebUI: Who are you?, then your question 12. Your instructions, the same search as "Wildfire RAG".
💾 Save

workshop-save
right-click on my-work.zip → Download...
Bring the newest one (maybe my-work (1).zip).

Next time: workshop-start 3 --from my-work.zip
(or its exact name between quotes, e.g. "my-work (1).zip")
📝 Exit card, alone
  1. Who picks the pieces in a RAG, and when?
  2. One question where the size or the number of pieces changed the answer.
  3. Why is a RAG's failure harder to see than one of mode B?
📚 A knowledge base for a group: yal-openwebui-guide, Step 1 and Step 4 · on a computer of your organisation: yal-local-ai-guide · github.com/Youth-AI-Lab

🆘 If you are stuck

Every command explains itself. From workshop 3, a few slash commands inside the chatbot show what it is using.

❔
In the terminal
workshop help
replay --help
score --help
🔍
In the chatbot
/context what fills the window
/status the key in use
/mcp the tools (from workshop 4)
Workshops 1 and 2: they answer "Unknown command". Watch the gauge above the input box.
🙋
Something breaks?
Ask the facilitator. Your files stay in the Codespace.
💡 To leave a conversation: Ctrl+C twice. In an open .md file, to read it as a page: Ctrl+Shift+V (Cmd+Shift+V on a Mac). In the terminal, copy / paste: Ctrl+Shift+C / Ctrl+Shift+V (Cmd+C / Cmd+V on a Mac). To change mode: workshop <mode>.

🧯 Backup: what a RAG hands to the model

If Open WebUI does not start: what we measured for question 10, "How many hectares burnt outside the EU in 2025?", top-k 5, same model.

1. The documents are cut into pieces (chunks), before any question.
2. Each piece becomes a list of numbers: its embedding.
3. The 5 pieces closest to the question enter the context.
4. The model answers from them only. It never sees the rest.
Pieces of 1000 · context: 2,291 tokens · judge: 1
Excerpts from summaries/10, corpus/text/10, summaries/90
✅ "2.2 million" (summary 90)
❌ 1 092 095 (document 10): other pieces of it came
➡️ "I can't tell which reading is correct from the excerpts alone."
Pieces of 2000 · context: 3,145 tokens · judge: 2
Excerpts from corpus/text/10, summaries/10, summaries/90, corpus/text/90
✅ "2.2 million" and ✅ 1 092 095
➡️ "The figures add up only if the 2.2 million includes the EU."
💡 Who decides what enters the context? A search engine, before the model. Question 7: 5 excerpts, although nothing answers.

🗣️ Say (EN) - Welcome to the second workshop. Today, your chatbot gets all the knowledge of the kit. - First we put everything in the context, then a search engine picks a few pieces: that is a RAG. - Please open the Codespace of this workshop, and bring my-work.zip from last time.

Fiche animateur : « Facilitator sheet » de l'atelier 2 sur le site des diapositives (préparation, bandes à imprimer, mesures, replis). Bandes de papier : https://youth-ai-lab.github.io/yal-workshop/2/strips.html (une enveloppe par manche et par groupe de deux binômes).

🗣️ Say (EN) - A chatbot only answers from what it reads before it answers. - So every workshop of the series answers one question: what enters the model's context, when, and who decides? - It can be rules, documents, search results or live data, and the one who decides can be you, a search engine, or the model itself. - We come back to this question in each part of each workshop.

La question qui organise toute la série. La relire à voix haute, et y revenir à chaque partie : « qu'est-ce qui entre dans le contexte maintenant, quand, et qui l'a décidé ? ». Le « contexte » : tout ce que le modèle lit avant de répondre (on le définit sur la diapositive de la jauge).

🗣️ Say (EN) - Last time, the chatbot had instructions and no knowledge: it said "I don't know". - Today it gets the 37 files of the kit: first all at once, mode B, then through a search engine, a RAG. - The question of the day: who picks the pieces?

Rappel de l'atelier 1 : Feurisson et Prudence avaient le même total (mesuré avant les règles de protection du 2026-10-31) ; aujourd'hui, avec des documents, Feurisson peut répondre.

🗣️ Say (EN) - Two hours in seven parts: welcome, everything in the context, a paper activity, the pieces of a RAG, its settings, a comparison, and a closing.

25 + 15 + 15 + 20 + 20 + 10 + 10 = 115 min (accueil 20 + question 12, 5), plus la démonstration de saturation de l'animateur à la fin de la partie 2 (5 min) : plus de marge, ses 10 min sont allées à la démonstration et à la question 12. Les parties 2 et 3 n'ont pas besoin d'Open WebUI : environ 35 min pour qu'il soit prêt. Si un binôme n'est pas READY à 1:00, faire les parties 4 et 5 sur l'Open WebUI de l'animateur, au grand écran ; le binôme rejoint dès READY (si STOPPED : openwebui start && openwebui sizes).

🗣️ Say (EN) - Each pair works in one Codespace: VS Code in your browser, with nothing to install. - Before you open it, the first time on your GitHub account: go to github.com/settings/codespaces, set Default idle timeout to 120, and Save. It only applies to the Codespaces you create after, so do it first. - Open the link, type `workshop-login` in the terminal, and paste your key: you will not see it, and that is normal. - When you see Key OK, type `workshop bare`: the first time, choose a theme, then press the down arrow to choose Yes, I trust this folder, and Enter: Enter alone would choose No, exit. To leave, press Ctrl+C twice. - Then type `workshop-start` with the number of today's workshop. If you saved your work last time, add `--from my-work.zip`: it brings your work back first. It opens the page of the day. - If the Codespace stops after a long pause, just reopen it: nothing is lost. - No Codespace, because the network blocks it or the quota is used up? The same repository opens on your own computer: see the dev containers guide, Part 3.

Un dépôt par atelier : https://codespaces.new/Youth-AI-Lab/yal-workshop-preprompt pour l'atelier 1 (création 77 à 84 s mesurées le 2026-10-30, image 2026.10.31, dépôt de l'atelier 1). Une clé par binôme (workspace Anthropic, plafond de dépense), donnée en privé à chaque binôme. Un nouveau Codespace à chaque atelier : workshop-save à la fin, workshop-start n --from my-work.zip au début du suivant. Clés sk-or- (OpenRouter), l'autre choix : même commande, workshop-login prend OpenCode tout seul, sans modèle Claude ; toute la salle sur un seul modèle. Claude Code les refuse. Messages : « Key OK (Claude Code, Anthropic, model claude-sonnet-5-5). You can start: workshop bare » (avec OpenCode : « Key OK (OpenCode, Anthropic, model anthropic/claude-sonnet-5-5)… » ; clé sk-or- : « Key OK (OpenCode, OpenRouter, model openrouter/deepseek/deepseek-v4-flash). You can start: workshop bare »). HTTP 401/403 : clé refusée. HTTP 400 : presque toujours une clé créée sans workspace (SETUP.md). Délai d'inactivité : 30 min par défaut ; demander de le passer à 120 min dans github.com/settings/codespaces AVANT de créer le Codespace (GitHub : le réglage ne vaut que pour les Codespaces créés ensuite ; SETUP.md, étape 1).

🗣️ Say (EN) - Drag your save into the Codespace, then type workshop-start 2 with --from: your system prompt and your scores come back. - The same command starts Open WebUI in the background, and cuts the documents into pieces of two sizes. Do not wait: parts 2 and 3 do not need it. - If you downloaded your save more than once, drag the newest one, maybe my-work (1).zip, and check the date in the line "Restored from".

Sans sauvegarde : workshop-start 2 copie Feurisson dans AGENTS.md. Mesuré dans deux vrais Codespaces neufs (2026-10-30, image 2026.10.32, 2 cœurs) : READY en 3 min 51 et 4 min 13 (téléchargement ~2 min 15). Fin du journal : « READY: Open WebUI is ready, with the two sizes. » ou « STOPPED: … ». Une fois sur quatre (image 2026.10.32), la base de 1000 ne renvoyait rien après READY. Depuis l'image 2026.10.33, start et sizes font une recherche test et redémarrent Open WebUI une fois ; sinon : openwebui stop puis openwebui start && openwebui sizes.

🗣️ Say (EN) - Question 12 is yours: one precise fact about your territory, from an official page. - Write the question, the expected answer and the address of the page in row 12. - Then replay it with the model alone and with your instructions, and score both answers blind. - Today, "I don't know" is the honest answer: it gets a 1. In workshop 3, you add the page to your chatbot, and this question should get a 2.

Décision du 2026-11-06 : la Q12 passe de l'atelier 1 à l'atelier 2 (l'atelier 1 garde les 11 questions du kit, totaux sur 22). Deux rôles : évaluation et appropriation. 5 min prises sur la marge (il n'en reste plus : 5 pour la démo de saturation, 5 ici). Totaux sur 24 dès que la ligne 12 est écrite ; Bare et A reçoivent leur 12e note ici. Un binôme qui l'a déjà écrite à l'atelier 1 passe à la partie 2. La page de la Q12 prépare le document de l'atelier 3 (corpus-add). Les binômes peuvent l'écrire dans leur langue. Les mesures de cette série restent sur 11 questions (sur 22).

Partie 2 (15 min) : workshop B, jauge après la question 10, prédiction, replay B (complet), score B 3 7 10 (mode recommandé) ; les autres questions s'il reste du temps, avec leurs numéros (score B 1 2 4 5 6 8 9 11 12 : score B seul renote tout). Puis la démonstration de saturation (5 min, l'animateur seul, au grand écran : bande de 2 m et Haiku 4.5), diapositive « Three times the kit? ». Rappel de la jauge (diapositive suivante) et du barème (avant « Your turn »), communs avec l'atelier 1.

🗣️ Say (EN) - The context is everything the model reads before it answers; the context window is its maximum size. - The window holds 200,000 tokens, and a token is a piece of a word. - Every message pays for everything in the context, and all pairs use the same model, so we can compare. - Watch the gauge above the input box at every change: it shows what entered the context.

Jauge : mod Claude Code token-weather. Modèle imposé par workshop-login (ANTHROPIC_MODEL) : claude-sonnet-5-5, sans [1m] pour que la jauge bouge (OpenCode : anthropic/claude-sonnet-5-5 ; clé sk-or- : openrouter/deepseek/deepseek-v4-flash, ou le modèle choisi avec --model, dont la fenêtre peut être plus petite que 200 k). Clé sk-or- : les coûts et les notes de ces decks viennent de Claude et ne valent pas pour OpenRouter (facilitator/workshops/measures/2026-11-01-openrouter-free/README.md). Jauge en interactif, 2026-10-30 (vrai Codespace, image 2026.10.34) : bare 1,6 k après la Q1 et 1,7 k pour « Hello », A avec Feurisson 2,6 k. B : 100,7 k à la répétition du 2026-10-21 (image plus ancienne). Elle reste à 0 avant le premier message (`0 / 200k`). Ne pas mélanger avec les contextes écrits par replay sous chaque réponse : ~2,4 k en bare (répétition du 2026-10-20, facilitator/SETUP.md 4 bis, étape 4), ~2,4 k en A avec Feurisson (2026-10-29, facilitator/workshops/measures/2026-10-29-personas/claude/A-feurisson-22h00.md).

🗣️ Say (EN) - In mode B, your system prompt and the 37 files of the kit enter the context at every message: 17 summaries and 20 documents. - That is about 100,000 tokens: half of the window, before you have asked much. - The model can find anything, and see a contradiction. But every message pays for everything.

Mesuré à la répétition (2026-10-21) : 100,7 k / 200 k après la question 10 ; la barre B est à l'échelle, la barre A symbolique.

🗣️ Say (EN) - Each answer gets 0, 1 or 2 points, and you write them in `scores.md`. - 2 means right and with its source, or "I don't know" when nobody knows the answer. - 1 means honest but not enough, and 0 means wrong or invented. - Remember: an invented answer is worse than "I don't know".

Barème dans test-questions.md ; notes dans scores.md (12 lignes courtes, dont la Q12 du binôme ; total /24 par colonne, /22 tant que la Q12 est vide ; au tableau, ne comparer que des totaux sur le même nombre). Questions 7 et 9 (inconnue, hors sujet) : la bonne réponse est de le dire.

🗣️ Say (EN) - Type workshop B. The first time, press the down arrow to trust the folder. The gauge shows 0 / 200k until your first message: the kit goes with it. - Ask question 10 in mode B and look at the gauge. Then predict, replay, and score questions 3, 7 and 10 of column B. - If you have time, score the other questions too, with their numbers. Compare the totals only if both columns are complete. - No save, so no column A? Type replay A and score A first. - This time the judge proposes a score: press Enter to accept, or type your own.

Mesuré (2026-10-21) : replay B 1 min 35, 0,47 $ (environ sept fois les 0,07 $ du mode A de la même répétition ; cinq fois les 0,09 $ du mode A de l'atelier 1). Discussion : quand le groupe ajoute un document par semaine, quand est-ce que ça s'arrête ? La réponse vient avec la démonstration de la diapositive suivante.

🗣️ Say (EN) - This strip is the window: two metres, one centimetre for a thousand tokens. The kit fills one metre: half, as the gauge said. - Two kits fill the window. Three times the kit: will the chatbot answer? Hands up. - For this demo only, I use another model, Claude Haiku 4.5: its window is exactly 200,000 tokens, like the gauge. You keep the model of the day. - Look: "Prompt is too long". The chatbot did not answer badly: it did not answer at all. And the gauge stays at zero. - Compacted means: the conversation is replaced by a shorter summary. Here there is nothing to summarise yet. - So we cannot send everything. Someone has to choose what enters: that is the next part, with paper.

Démonstration de l'animateur, 5 min prises sur la marge (fiche animateur, « The overflow demo » : préparation, bande, replis). Préparation pendant la partie 1, dans le Codespace de l'animateur, jamais dans un dépôt : copie dans /tmp/three-kits (76 copies), puis au grand écran WORKSHOP_HOME=/tmp/three-kits workshop B --model claude-haiku-4-5, question 10 ; la première fois, faire confiance au dossier. Ensuite Ctrl+C deux fois et rm -rf /tmp/three-kits. Mesuré le 2026-11-05 avec la vraie API (un appel, forme -p, image 2026.11.4) : ce message, code 1, 0 $ affiché par Claude Code, une requête refusée par l'API (400, 237 206 > 200 000), 2 s. Facture dans la console : pas encore regardée. Écran interactif avec la vraie API (le même jour) : même message, jauge à 0 / 200k, plus une petite requête payée (Haiku 5.5, 1 178 jetons lus, 25 écrits, 0,00013 $ selon Claude Code ; sans doute le titre de la conversation). 237 206 et pas 300 000 : Haiku 4.5 compte environ 22 % de jetons de moins que Sonnet 5.5 sur le même texte (81 418 contre 104 723 pour un kit, count_tokens). Deux copies tiendraient (159 325) et la question serait payée : toujours trois. Pourquoi pas Sonnet 5.5 : fenêtre de 1 M (table d'Anthropic), la jauge s'arrête à 200 k mais la requête part ; trois kits seraient lus et payés (environ 0,60 à 0,80 $ au prix de liste : 2 $ le million en lecture, 2,50 $ en écriture du cache ; tarif au-delà de 200 k non vérifié ; non essayé). Première fois : flèche du bas sur « Yes, I trust this folder », puis Entrée (Entrée seule choisit « No, exit »). À l'écran interactif, le message finit par « · /clear to start fresh » (vu avec le faux seulement). Mesures : facilitator/workshops/measures/2026-11-05-saturation-real/.

Partie 3 (15 min). Une enveloppe par manche et par groupe de deux binômes. Cartes Q10 et Q7.

🗣️ Say (EN) - Two pairs work together. One pair is the search engine: it passes a few strips. The other is the model: it answers with these strips only. - Every strip is copied from a summary of the kit. Then compare with questions 10 and 7 of test-questions.md.

Attendu, pas encore observé avec un groupe (TO BE COMPLETED à la répétition) : on cherche « outside the EU », or le résumé 90 dit « beyond the EU ». Q10 demande les bandes 11 et 22 en manche 1 (parmi 29, 3 passées), G et C en manche 2 (parmi 7, 2 passées) : les deux tiennent dans ce qu'on passe. En manche 1, « Outside the EU » mène à la bande 1, qui ne donne aucune surface en hectares, et la bande 11 commence par « Of these » : on s'attend à ce qu'elle soit oubliée plus souvent. En manche 2, G porte « Outside the EU » à côté des deux chiffres du résumé 10, C le « 2.2 million » du résumé 90. Q7 : des bandes sur la Campanie, pas sur 2025 (« always something »). Réutilisable avec des 12-15 ans, sans ordinateur.

Partie 4 (20 min). Mots du jour : chunk, embedding, top-k. Open WebUI doit être READY.

🗣️ Say (EN) - Open WebUI cut the 37 files into pieces of at most 1,000 characters: chunks. It cuts at the end of a paragraph, so most pieces are shorter. - A small model turns each piece, and each question, into numbers: embeddings. Close meanings give close numbers. - For each question, the 5 closest pieces go to the model: top-k is 5. The model never sees the rest.

Modèle d'embeddings d'Open WebUI par défaut : sentence-transformers/all-MiniLM-L6-v2 (fiche du modèle : `language: en`). Longueur moyenne réelle des extraits : 770 caractères pour la base de 1000 (le découpeur coupe aux paragraphes).

🗣️ Say (EN) - Think of each text as an arrow from the centre of the map. All arrows have the same length, so only the direction counts. - The search does not measure an angle. It multiplies the two lists number by number and adds up. With 384 numbers, the sum is computed the same way. This sum is the cosine of the angle. - Open WebUI shifts it so that the score stays between 0 and 1. That is why "no link" is 0.5 here, and not 0.

open_webui/retrieval/vector/dbs/chroma.py rend (2 - distance) / 2, soit (1 + cosinus) / 2 pour des vecteurs normés. Mesures : facilitator/atelier-2-measures/2026-10-31-embeddings/pairs.txt. Exemple A et B inventé ; 0,96 fait 16°. Dessiner deux flèches au tableau prend une minute. Le pourcentage affiché au clic sur une source n'est pas ce score : sa formule n'a pas été cherchée.

🗣️ Say (EN) - Each text gets 384 numbers. Nobody wrote them: the small model learnt to place texts by training on a very large number of sentences. - With your partner, sort these pairs. Which are close? Which are far apart? You have one minute.

Deux minutes, à voix haute, avant la diapositive suivante. Les réponses sont aussi dans la page, repliées. Mesures : facilitator/atelier-2-measures/2026-10-31-embeddings/.

🗣️ Say (EN) - Outside and beyond: different words, close numbers, 0.944. Question 10 says outside, summary 90 says beyond, and its piece comes back. - Forbidden and allowed get the highest score: both sentences talk about the same thing, and only one word differs. For the search they are almost the same. The model that reads the excerpts can tell them apart, and so can you. - Incendie de forêt means forest fire, and the small model places it further from wildfire than a birthday cake.

Score d'Open WebUI : (1 + cosinus) / 2. Ne pas entrer dans cette formule en salle. Le modèle ne lit que 256 jetons (environ 1 230 caractères d'anglais, 780 de français, mesuré) : voir la partie 5.

🗣️ Say (EN) - Each step can be explained. - The 1 092 095 hectares miss the cut by less than 0.02. The piece is not bad, it is just seventh. - The model only receives a longer message. It does not know that 321 other pieces exist.

Mesures sans modèle, base de 1000, top-k 5. Le message est celui capturé en sortie d'Open WebUI (message-sent-to-the-model.json, faux modèle, mode Legacy).

🗣️ Say (EN) - openwebui query shows the pieces the RAG would give the model, without asking the model: it costs nothing. - Read the five excerpts for question 10, then for questions 7 and 3.

Mesures sans modèle (2026-10-29, ATELIER-2-OUTILS.md §2). Q3 : document en français, question en anglais. Testé le 2026-10-31 : en français, 4 extraits sur 5 viennent du document 40, et les deux passages INTERDITS vont des rangs 61 et 63 aux rangs 30 et 70 sur 326 (2026-11-02, après les corrections des résumés 20, 31, 60, 71 ; le 2026-10-31 : 56 et 58 sur 322). La langue change le résultat sans tout expliquer. L'effet de la forme de ces passages (des listes de niveaux) n'est pas testé : à dire comme tel. Une question sur un gâteau obtient 0,573 au mieux, la question 7 monte à 0,831 sans réponse dans le corpus : un bon score dit « même sujet », pas « la réponse est là ».

Partie 5 (20 min). Penser à openwebui top-k 5 à la fin.

🗣️ Say (EN) - Bigger pieces of 2000 characters bring back both figures of question 10. So do 10 pieces of 1000. - Both cost more text at every question. And question 3 still fails: no setting brings back the article that answers.

Mesuré (2026-10-29) : Q10 complète en 2000 dès top-k 3, en 1000 à top-k 10 ; Q3 jamais. Choix retenu pour la séance : montrer 1000 à top-k 10 (2000 à top-k 3 n'est pas proposé aux binômes).

🗣️ Say (EN) - Here are the two answers we measured to question 10, with the same model. The only difference is the size of the pieces. - Both answers see that the sources disagree. With pieces of 1000, the model only has the totals: it subtracts them and cannot tell which reading is right. With pieces of 2000, one piece also holds the detailed figures: the model checks that they add up, and explains the disagreement. - Try it in Open WebUI: choose Wildfire RAG 1000, then 2000, and click the sources under each answer. - If you change the top-k to try, set it back to 5 before the next part: replay RAG refuses otherwise.

Citations exactes de facilitator/atelier-2-measures/2026-10-30-RAG-1000.md et -2000.md (question 10).

Partie 6 (10 min). Le top-k doit être 5. Les consignes du chatbot RAG sont fixes (RAG_PROMPT d'openwebui), pas l'AGENTS.md du binôme : une partie de l'écart B / RAG peut venir des consignes.

🗣️ Say (EN) - Replay the whole test set with the RAG, then score questions 3, 7 and 10: a RAG column appears after B. The other questions if you have time. - Compare the answers, and the cost: about a third of mode B. - The RAG reads about 40 times less, but costs only about 3 times less: writing is most of its bill, and mode B pays a lower price for text it sends again (the prompt cache).

Mesure payante du 2026-11-02 (11 questions, sans Q12) ; coûts du RAG estimés d'après les jetons d'Open WebUI au tarif de claude-sonnet-5-5 (0,142 $ et 0,157 $). Q7 = 2 aux deux tailles (le modèle dit que les extraits ne répondent pas) ; Q3 = 1 ; Q10 = 2 aux deux tailles. Le 2026-10-30 : 18/22 et 19/22, Q10 à 1 en 1000. Les réponses à Q10 disent la même chose les deux jours : seule la note du juge a bougé, un écart de quelques points entre deux passages ne veut rien dire. Note B au juge : non mesurée dans la série (TO BE COMPLETED). Trois passages du juge : 18, 19, 18/22 en 1000 ; 19, 16, 19/22 en 2000 (2026-10-30, 2026-11-02, 2026-11-04). Les deux premiers avec le gabarit RAG par défaut d'Open WebUI (qui disait de répondre de ses propres connaissances), le troisième avec celui du kit (depuis devcontainer-2026.11.2, publiée sans être épinglée ; les binômes l'ont avec 2026.11.4) : mêmes extraits, même coût (0,135 $ et 0,151 $), Q3/Q7/Q11 inchangées, Q10 à 1 en 1000 comme le 2026-10-30. Notes dans l'écart des deux passages précédents ; l'écart mêle réponses et juge, aucun second passage du juge sur les mêmes réponses (atelier-2-measures/2026-11-04-rag-paid-template/).

🗣️ Say (EN) - Mode B reads everything: simple, but it stops when the knowledge grows, and every message pays for all of it. - A RAG grows, and costs less per question. But it always gives something, so when it picks the wrong pieces, nothing tells you. - That is why we keep testing with the same questions. - Workshop 3 starts from this failure: what if the model looked for the pieces itself, and you could see where it looked, and when it found nothing?

Message à retenir de la fiche animateur : lire tout ne grandit pas ; un RAG grandit, mais donne toujours quelque chose.

🗣️ Say (EN) - Some tools make this choice for you. Claude Projects, on paid plans, switch to a RAG when the knowledge nears the window, and show an indicator. - Gemini's help says that content beyond the window may be missed, and documents no switch to a search. - So the same Knowledge box can read everything, search for pieces, or miss part of it, depending on its size and the tool. Claude Projects shows when it searches, but you do not choose. Gemini's help does not say how you would know that part was missed.

Pages d'aide lues le 2026-10-29 (ATELIER-2-OUTILS.md §3, sources et citations exactes). Rien n'a été essayé dans un vrai compte Gemini : ne pas présenter le silence de l'aide comme un comportement observé. Si la salle demande : claude.ai est réservé aux plus de 18 ans, et les Gems des comptes personnels de plus de 18 ans passent en lecture seule le 2026-11-17 (supprimées le 2026-12-17) ; moins de 18 ans : « at a later date », mais les Gems de Classroom attribuées à un compte personnel d'élève s'arrêtent le 2026-11-17 ; Workspace for Education : juin 2027. Ni l'un ni l'autre n'est donc, pour l'instant, un endroit où rebâtir un chatbot avec des 12-15 ans ; la clôture de l'atelier 5 y revient.

🗣️ Say (EN) - Now give your own chatbot a RAG. In the second terminal, openwebui chatbot with the name of your chatbot and --rag. In Open WebUI, choose your chatbot with "(RAG)" at the end, ask "Who are you?", then your question 12. Your instructions, your knowledge, and before each answer the same search as "Wildfire RAG". - Without --rag, the same command makes a chatbot that searches by itself: you will build it in workshop 3. - Save your work and download my-work.zip for workshop 3. - Then the exit card, alone. And to build a chatbot with a knowledge base for a group, the Open WebUI guide shows each step.

Ancres vérifiées : yal-openwebui-guide#step-1---gather-your-documents-knowledge, #step-4---try-it ; yal-local-ai-guide#what-we-measured (mêmes deux échecs avec des modèles locaux). openwebui chatbot --rag (image 2026.11.4) : avec un faux modèle, sans réseau, le 2026-11-04 (openwebui-prototypes/skills-rag/) : aucun outil, 5 extraits, consignes du binôme. Avec qwen3:8b (Ollama, ThinkPad, 2026-11-05, un essai, openwebui-prototypes/real-model-local/results-thinkpad/) : Q7 « pas dans les documents » (2), Q3 bons niveaux sans les heures (1) : mêmes notes que « Wildfire RAG ». Non essayé avec Claude ni dans un Codespace (sans --rag : 37 s pour 37 fichiers en local). Ne pas le sauter si le temps manque : c'est la clôture de l'atelier (décision du 2026-11-03) ; couper plutôt les autres questions de la partie 6.

🗣️ Say (EN) - If you are stuck, every command explains itself, for example `workshop help`. - In workshops 1 and 2 the chatbot has no slash commands: they answer "Unknown command". From workshop 3, `/context` shows what fills the window and `/status` shows the key in use; from workshop 4, `/mcp` shows the tools. - To leave a conversation in any mode, press Ctrl+C twice, and to read an open `.md` file as a page, press Ctrl+Shift+V (Cmd+Shift+V on a Mac); in the terminal, copy and paste are Ctrl+Shift+C / Ctrl+Shift+V (Cmd+C / Cmd+V on a Mac). - If something breaks, ask me: your files stay in the Codespace.

Repères de SETUP.md (copié dans chaque dépôt d'atelier). Vérifié à la répétition : en bare, A et B (--disable-slash-commands), /exit répond « Unknown command » ; Ctrl+C deux fois quitte dans tous les modes. /context, /mcp, /status : à partir de C.

🗣️ Say (EN) - Open WebUI did not start, so here is what we measured. A RAG cuts the documents into pieces before any question, turns each piece into numbers, and gives the model the 5 pieces closest to the question. - Question 10, pieces of 1000: the excerpts come from summary 10, the readable copy of document 10 and summary 90, but not the paragraph with 1 092 095. The model only has the totals, and cannot tell which reading is right. - Pieces of 2000: one piece holds both detailed figures. The model checks the sum, and explains the disagreement. - And for question 7, the RAG still gives 5 excerpts, although nothing answers: it always gives something.

Annexe : diapositive de secours si Open WebUI ne démarre nulle part (pas même sur l'écran de l'animateur) ; citée par la fiche animateur (Fallbacks), facilitator/openwebui/README.md et facilitator/SETUP.md. Adaptée de la 29 de l'ancien deck (git show e4f64bb:facilitator/slides/slides.md). Sources : réponses payantes du 2026-10-30, top-k 5 (facilitator/atelier-2-measures/2026-10-30-RAG-1000.md et -2000.md, question 10 : fichiers des extraits, une ligne par fichier, jetons de contexte, citations exactes) ; notes du juge et marqueurs sans modèle du 2026-10-29 (ATELIER-2-OUTILS.md §2 et « Mesure payante complète » : 1 092 095 absent des extraits en 1000 / k 5, présent en 2000 / k 5 ; Q7 : 5 extraits). Le texte exact des extraits n'a pas été gardé : ne pas le citer de mémoire. Q7 notée 2 aux deux tailles (le modèle dit que les extraits ne répondent pas).