← All workshops · Slides of workshop 1

Workshop 1 - Instructions, and how to test a chatbot: facilitator sheet

For the person who runs the workshop. The participants' page is the README.md of the repository of the workshop, yal-workshop-preprompt (source: workshops/1-preprompt/ of the facilitator's repository), and their one-page sheet sheet.md.

ℹ️ Note. This sheet is not in the repositories that the participants open, on purpose: it gives expected scores, and from workshop 3 on, in modes C, D and E, the model can read every file of the repository.

Question of the day: what does the chatbot read before it answers? Message to leave with: the text of my question is not the only thing that shapes the answer: knowing what is in the context helps me read the answer. Instructions change how a chatbot behaves, not what it knows; and a total does not show everything. The piece the participants reuse with young people is the Open WebUI chatbot of Part 5: do not skip it, even when time is short.


Before the workshop (about 1 hour)

Time What How you check it
15 min One Anthropic API key per pair, all in one workspace for the room, capped at $3 × the number of pairs (see Who pays and what we measured, below). Your key is accepted by workshop-login: Key OK (Claude Code, Anthropic, model claude-sonnet-5-5)
15 min Your own run: a new Codespace from https://codespaces.new/Youth-AI-Lab/yal-workshop-preprompt, workshop-login, workshop-start 1, replay bare, replay A --agents examples/AGENTS.sophist.md, score answers/A-sophist-….md --blind, openwebui start, cp examples/AGENTS.feurisson.md AGENTS.md (stands in for Part 4: the closing needs a filled AGENTS.md), openwebui chatbot --name "Feurisson" Each command ends as the participants' page says. The last one prints Instructions: your AGENTS.md, 39 lines. It starts with: "You are Feurisson, a friendly hedgehog who helps young people prevent…" (not Warning: your AGENTS.md is still empty), and Who are you? in Open WebUI answers as Feurisson
10 min Print the one-page sheet (one per pair). For the exit cards, one blank card or half sheet per person: the three questions are on the page (Part 5, Step 3), on the deck and below (Exit card) -
5 min Self-assessment form ("Before"): open it, check that its QR code opens it on a phone, and that it collects no names, e-mail addresses or ages (the self-assessment; form link TO BE COMPLETED) the form opens on a phone
- Translate the one-page sheet (sheet.md) into the language of the group if needed, before you print it: keep the commands and what the terminal prints in English, as they appear on the screen -
10 min Assign the personalities (table below) and prepare the board table -
10 min Spare plan: one spare key; the measured answer files (Fallbacks) on a USB stick or in a shared folder -

How to create a key and set a limit: Open WebUI guide, Your Anthropic API key.

Or OpenRouter keys (sk-or-): same commands, no Claude model, and the whole room on one model so that totals compare (OpenRouter guide). The figures of this sheet come from Claude and do not hold for OpenRouter (its measures). Cost per pair with the default model, DeepSeek V4 Flash: under one cent ($0.0045) for the terminal commands of this workshop and about $0.12 for workshops 1 to 5, played once on 2026-11-05 in a container of the image 2026.11.4 (OpenCode 2.1.4), not in a Codespace and not with a group (measures). Not counted: replay RAG and the Open WebUI chatbots. A pair that asks more questions spends more: take it as a minimum. In mode bare, DeepSeek V4 Flash answered 3 questions of 11 (1, 2 and 10) with a tool call written as text (<|DSML|tool_calls>: a web_search for 1 and 10, a shell curl for 2, tools it does not have) instead of an answer: it is not a broken kit, Haiku and Sonnet did the same in mode A before workshop told them that they have no tools. The workshops are for adults: no young person types in the Codespace or holds a key, and OpenRouter asks for users of 18 or older. Young people only use a chatbot in Open WebUI, in their browser. When what they wrote goes through OpenRouter, leave out names and personal details, and type workshop-login --europe first (SETUP.md, Young people and OpenRouter; OpenRouter guide, Part 8). With --europe, the judge of score is Mistral Small 3.2: on 181 answers, it gave the same score as Haiku 135 times (75 %), 44 times one point apart, twice two points apart (measures, section Juges). Read its scores as a rough guide, and do not compare them with totals scored by Haiku.

Other ways to pay for the model (Gemini, subscriptions), for your own tests: OpenCode guide, five ways to pay for the model.

ℹ️ Note. Who pays. Each partner organisation pays for its own sessions: its own Anthropic account, workspace and keys, on its project budget. One workspace for the room, capped at $3 × the number of pairs: 8 pairs, $24. At Antwerp, the most advanced pairs used less than $2 in a 2-hour workshop. The limit is on the workspace, not on each key, so one pair can use up the share of another: keep the Anthropic console open during the workshop and watch the spending. Not checked: if the console limit is per month, the workshops of the same month share it. Then count $3 × pairs × workshops of the month, or raise the limit before each workshop. Indicative cost per pair for the whole series: TO BE COMPLETED at the rehearsal; this workshop's measured cost is under "What we measured".

GitHub quota. GitHub Free gives 120 core-hours a month (60 hours on the 2-core machine of the kit) and 15 GB-months of storage. Open WebUI takes 6.1 GB of it. Each pair uses its own account.

Who gets which personality

Pairs in the room Personalities, in this order
2 Feurisson, Prudence (overcautious)
3 + Doctor Proof (sophist)
4 + Captain Certain (liar)
5 + Grandpa Beaver (storyteller)
6 or more then again from Feurisson: two pairs with the same personality compare their human scores

Feurisson and Prudence come first: their same total is the main discussion of Part 3 (measured again with the protection rules of Feurisson on 2026-11-06: still 13/22 from the judge, the same score on each question; Prudence was not measured again).

The board table

Personality Total of the pair (as score prints it) Total of the judge (as score prints it) Disagreements The trick, in one sentence
Feurisson
Captain Certain (liar)
Grandpa Beaver (storyteller)
Doctor Proof (sophist)
Prudence (overcautious)

Each pair writes its totals with their "out of", as score prints them (page, Part 3, Step 4). In this workshop, every total is out of 22: question 12 (the pair's own question about its territory) is written in workshop 2, so row 12 stays empty today. Compare two totals only when they are out of the same number: an answer marked ERROR is not scored. The fallback files are out of 22 too.


Timing

Time Part What you do What the room should have at the end
0:00 1. Welcome (20) Self-assessment "Before" (5 min, before the Codespace): QR code of the form (link TO BE COMPLETED; no QR slide in the deck yet), a personal code with no name: see the self-assessment. Question of the day. Setup (SETUP.md, steps 1 to 5), then workshop-start 1, then openwebui start in terminal 2. Walk around: the paste of the key is the usual problem. Key OK, Ready for workshop 1
0:20 2. The test set (20) Read the scoring rules aloud. Ask for a prediction on paper. replay bare, scoring by hand. Question 12 is not written today: it is a step of workshop 2 (decision of 2026-11-06), so row 12 stays empty and totals are out of 22. column Bare scored, out of 22
0:40 3. Five personalities (35) Give the personalities. Remind: do not open the file before scoring. Say why the pairs score before they see the judge: without a score made by a person, nobody can tell when the judge is wrong, so they score first, then compare and look at the disagreements (the judge is checked in turn). Fill the board table. Lead the discussion (below). the board table full
1:15 4. Your system prompt (25) AGENTS.md, workshop A, prediction, replay A, score A --blind. column A
1:40 5. Closing (10) openwebui chatbot, workshop-save and the download, exit cards. my-work.zip downloaded by each pair; cards collected
1:50 Margin (10) 5 of them go to the self-assessment of Part 1, which the 20 minutes of the Welcome do not include (timing to check at the rehearsal). -

What we measured

2026-10-29: throwaway container of the image 2026.10.31, model claude-sonnet-5-5, judge claude-haiku-5-5 (score --auto), 11 questions (before question 12 existed). Costs of OpenCode are its own estimates at the catalogue price.

ℹ️ Note. The Feurisson figures of the table below were measured on 2026-10-29, before the protection rules (from "Never ask for personal information" to the end) were added to examples/AGENTS.feurisson.md on 2026-10-31 and made longer on 2026-11-01. Measured again with these rules on 2026-11-06 (image 2026.11.5, Claude Code 2.1.311, Feurisson in AGENTS.md, judge only, one try per question, measures): 13/22, the same score on each question, $0.09, 1 min 14, about 330 more tokens per answer (2,728 to 2,768). No question of the test set gives these rules a reason to act. The gauge in a conversation and the other personalities were not measured again.

Personality Claude Code 2.1.303: cost, time, judge OpenCode 2.1.4: cost, time, judge
Feurisson $0.09, 1 min 13, 13/22 $0.07, 1 min 09, 11/22
Captain Certain (liar) $0.10, 1 min 29, 2/22 $0.06, 1 min 02, 2/22
Grandpa Beaver (storyteller) $0.15, 2 min 11, 7/22 $0.13, 1 min 58, 7/22
Doctor Proof (sophist) $0.17, 2 min 16, 2/22 $0.14, 2 min 05, 3/22
Prudence (overcautious) $0.06, 0 min 48, 13/22 $0.03, 0 min 34, 13/22

The judge cost about $0.002 per answer file (11 answers). replay bare cost $0.10 (2026-10-21).

One pair's path, in a real Codespace (2026-10-30, repository yal-workshop-preprompt, image 2026.10.31, 2-core machine, real key, driven by gh codespace ssh; 11 questions, question 12 left empty except for one test replay):

Step Time Cost
Codespace created and ready 77 to 84 s -
replay bare 1 min 42 $0.10
replay A --agents examples/AGENTS.liar.md 1 min 20 $0.11
score answers/A-liar-….md --auto (judge: 3/22) 4 s $0.002
Conversation in mode bare: 3 messages (hello, question 1, "where does this figure come from?") 16 s $0.03
Conversation in mode A (Feurisson): 4 messages (hello, who are you, questions 9 and 7) 14 s $0.04
replay A (Feurisson) 1 min 12 $0.09
score A --auto (judge: 13/22) 6 s $0.002
openwebui start, first time (1.6 GB downloaded) 3 min 03 -
openwebui chatbot --name Feurisson (0 knowledge files) 2 s -
3 questions to the chatbot in Open WebUI - $0.016
Total about $0.39

The costs of replay, score and the conversations are those printed by the commands; the cost in Open WebUI is computed from its token counts (2,427 in, 1,113 out) at the price of claude-sonnet-5-5 ($2 and $10 per million tokens). A pair that talks more, or replays twice, spends more: $1 per pair is a safe estimate, below each pair's $3 share of the room's limit. The real durations of the five parts with a group are still TO BE COMPLETED at the rehearsal.

In Open WebUI, the chatbot of this repository has only its instructions (no documents): it answered "I don't know" to questions 7 and 1, as in mode A, with about 800 input tokens per question (Claude Code sends about 2,400 in mode A: its own instructions come on top). How these 800 were measured is not recorded; they look like questions asked through the API, which sends no tool definitions. In the browser, Open WebUI also sends the definitions of the 6 tools that search the knowledge base (image 2026.11.4, Open WebUI 0.12.4; since 0.12.5, also display_file), about 1,300 tokens estimated with a fake model on 0.12.4 (about 5,000 before, for 27 tools), not measured with the real model.

Opening Open WebUI (2026-10-30, real Codespace of yal-workshop-preprompt): the Ports tab shows port 3000 as Private, forwarded automatically. The globe (Open in Browser) opens Open WebUI directly on "Wildfire RAG (workshop kit)", with a green notice ("Vous êtes désormais connecté." in a French browser: there are no accounts). The owner of the Codespace sees no GitHub warning page; it appears only when the link is opened in another browser session. From image 2026.11.6, openwebui start makes no "Wildfire RAG" model in this repository (it has no documents) and removes the one an older image made: the list holds only the pair's chatbot, once openwebui chatbot is done. Until openwebui chatbot is done, the page opens on no model (not seen on screen yet). The path of Part 5 to read the paragraph ## Your knowledge (Workspace → Models, the chatbot, the line Prompts) comes from the Open WebUI guide: not seen on screen in the kit's mode without accounts.

The gauge, in a real Codespace (2026-10-30, repository yal-workshop-preprompt, image 2026.10.34, Claude Code 2.1.303, claude-sonnet-5-5, one new conversation per line; the page uses these figures):

Mode Message Gauge
any before the first message 0 / 200k
bare question 1 1.6k
bare then "Where does this figure come from? Give me your source." 2.2k
bare Hello 1.7k
A (Feurisson) Hello 2.6k
A (Feurisson) question 1 2.6k

About 900 to 1,000 more tokens in mode A: the system prompt, plus the line of workshop that says the chatbot has no tools. The trust screen appears again at the first workshop A (another folder).

What happened (the same with both engines):

The answer files (download one to use it as a fallback):

Personality Claude Code OpenCode
Feurisson A-feurisson-22h00.md A-feurisson-22h10.md
Captain Certain A-liar-22h02.md A-liar-22h11.md
Grandpa Beaver A-storyteller-22h03.md A-storyteller-22h12.md
Doctor Proof A-sophist-22h05.md A-sophist-22h14.md
Prudence A-overcautious-22h08.md A-overcautious-22h16.md
Scores of the judge scores-personas.md scores-personas.md

Leading the discussion of Part 3

  1. Start with the same total. "Feurisson and Prudence have the same score. Are they the same chatbot?" Read aloud their answers to question 1 (both got 1 from the judge). Feurisson: "I don't know… I can't open the documents in summaries/ or corpus/ in this conversation". (Its words "in this conversation" come from the line workshop adds in mode A: no tools, no files; its "I don't know" comes from AGENTS.md.) Prudence: "I can't be sure… any number I gave you might be out of date". Both send to EFFIS. The difference is the reason: Feurisson lacks documents, Prudence refuses by principle. Then ask: "And when they get the documents, in workshop 2?" Feurisson will answer; Prudence will still refuse. The total of today hides it.
  2. Then the sophist. "Who gave Doctor Proof more than the judge?" Look at its answer to question 2: the number of levels can be right while the names come from another scale (EFFIS). Each fact may be true, the conclusion is still wrong.
  3. Then the storyteller. "Did the facts survive the story?"
  4. Close with the liar. Its tone is the most confident of the five. A confident tone is not a source.
  5. Link to WikiYouthBot: this is "I Heard a Rumor" (a chatbot can sound sure of itself and be wrong), from inside the chatbot.

⚠️ Important. Say it once: the deceptive bots are for adults who test chatbots. They are labelled, and their errors make the rules stricter. They are not for a group of young people.


Exit card

One card per person, collected at the end. Same three questions in every edition, to compare.

  1. What does the model read before it answers, in mode A?
  2. Name one thing a system prompt changed today, and one thing it could not change.
  3. Two chatbots have the same total. What else do you look at before you choose one?

What a good answer contains: (1) the general instructions of the tool (in mode A, with one added line: it has no tools and cannot open any file), its system prompt (AGENTS.md) and the conversation so far: no document; (2) the tone, the refusals or "I don't know" / the knowledge, the figures; (3) the answers themselves: invented details, sources, refusals, the questions where they differ.


Fallbacks

Problem What to do
No Codespace (network, quota) The repository of the workshop opens on a computer with Docker: dev containers guide. Or pair the group with another pair.
A key is refused Check the workspace in the Anthropic console first: the limit is shared, and when the room reaches it every key stops, the spare key too (not tested). Raise the limit if your budget allows. Otherwise, give the spare key.
replay is too slow A subset: replay A --agents examples/AGENTS.sophist.md 1 2 4 7 9 10.
replay fails for a pair Give it the measured answer file of its personality (column Claude Code of the table in What we measured): download it, drag it into answers/, then score answers/<file> --blind.
The judge does not answer score says Judge unavailable: the pair scores alone.
Open WebUI is not ready at the closing Skip Step 1 of Part 5. Show your own chatbot in Open WebUI on the big screen.
A pair uses OpenCode with an OpenRouter key Same steps, judge and Open WebUI included. Its totals are not comparable with the others (other model, other judge). A replay of 11 questions took 2 to 4.5 minutes with the default model (two tests). openwebui chatbot with this key: tried once, on a laptop, on 2026-11-05 (4 min 07, the right tools on 4 questions, one answer in French to a question in English; measures). If it fails, the pair watches your chatbot on the big screen.

After the workshop

Reuse with young people. The demo personalities are for adults. With a group, keep the test set (the "Press Conference") and the sections of AGENTS.md (the Instructions box). To build a chatbot for a group: Open WebUI guide, Part 2 - Build a chatbot, then Part 3 - Test before you open.