Cluemaster
Introducing Cluemaster 1.0
A 3B Codenames spymaster.
Cluemaster 1.0 plays the spymaster in Codenames. It reads the board and gives your team one word and a number, aimed at as many of your words as it can reach safely, and away from the opponent's words, the neutral ones and the assassin.
It scores 60% on Cluebench, against 20% for the next best 3 to 4B model, and gives a legal clue on the first try 96% of the time.
Spymaster quality
Cluebench rates every clue good, ok or bad in a blind rating. Fresh boards, midgame and endgame are scored separately and averaged, so the mix of boards cannot move a score.
Cluebench
mean of the three board types
By board type
Cluemaster leads on every board type. Midgame, with 4 or more of your words left, is its weakest at 40, against 72 on fresh boards and 67 on endgames.
Fresh boards
Cluebench score
Midgame
Cluebench score
Endgame
Cluebench score
Legal clues
A clue is illegal if it is a board word, a root of one, or a word that contains one. Cluemaster's single answers were legal on the first try 96% of the time, against 25 to 76% for the other models.
Legal on the first try
single answer, no retry
Rated good
legal and rated good
Acceptable
legal and rated good or ok
The Studio can also ask for several clues and drop any that break the rules before choosing. With that check on, Cluemaster gave a legal clue on every held-out board it was tested on.
Speed
A single answer takes about 2.5 seconds on a 6 GB graphics card. Granite 4.1 3B is faster and shorter, at 0.6 seconds and 19 tokens per answer. The model's own answer includes a short brainstorm before the clue, which is where the extra tokens go.
Seconds per board
lower is faster
Tokens per answer
lower is shorter
Try it today
Cluemaster 1.0 is on Hugging Face as a Q4_K_M GGUF (2.1 GB). It needs about 3 GB of free memory and loads in Ollama, llama.cpp and LM Studio. For Ollama, put this Modelfile next to the file and run ollama create cluemaster -f Modelfile.
FROM ./cluemaster-1.0-Q4_K_M.gguf
TEMPLATE """{{ if .System }}<s>[SYSTEM_PROMPT]{{ .System }}[/SYSTEM_PROMPT]{{ end }}{{ if .Prompt }}[INST]{{ .Prompt }}[/INST]{{ end }}{{ .Response }}</s>"""
PARAMETER stop <s>
PARAMETER stop [INST]
PARAMETER temperature 0.3
PARAMETER num_ctx 4096How to use it
The easy way: use Cluemaster Studio. Type in the board, press Get clue, and it handles the prompt and checks the clue against the rules.
Running it yourself: Cluemaster isn't a chat model. It does one job: you give it a board, and it gives you a clue.
Recommended system message
You are a Codenames spymaster. Your teammate can see every word on the board but does not know which ones are yours. You give a one-word clue and a number: the number is how many of your team's words the clue points to.
The clue must not be a word on the board, or a form or part of one. Avoid clues that could lead your teammate to the opponent's words, the neutral words, or the assassin.Recommended user message
Put your own words in capitals.
BOARD
My team: BUNK, CENTER, BAND, SNAKE
Opponent: MAKEUP
Neutral: WELL, KITCHEN, STATE, HORSE
Assassin: MICROWAVE
Give a clue that connects 1 to 4 of your team's words. Connect 2 or 3 when there is a real link between them. If the words you have left have no good link, give a clue for a single word instead of forcing a weak pair.
Reply with your clue, the number, and the team words it connects, like this:
Clue: <one word>
Number: <how many team words it connects>
Words: <those team words, separated by commas>What you get back
Brainstorm:
- RUBBER: BAND, SNAKE. Found 2 of 2. Risk: WELL (neutral).
Clue: RUBBER
Number: 2
Words: BAND, SNAKEUse the Clue, Number and Words lines. The brainstorm above them is its working. Set the temperature to about 0.3 and stop after 400 tokens, because it can keep writing extra answers after the first.
It only gives clues, so any other message gets a clue back.
Model card
Model information
- Description
- Cluemaster 1.0 is a 3.4 billion parameter model that plays the Codenames spymaster.
- Model dependencies
- A LoRA fine-tune of Ministral 3 3B Instruct. Rank 16, alpha 32, on all attention and MLP layers, 24.7 million trainable parameters.
- Inputs
- A text prompt in the format it was trained on: the board and the clues so far. The harness builds the prompt.
- Outputs
- A brainstorm, then
Clue,NumberandWords. The model generates all of it, and it was trained to answer in this format.
Model data
- Training dataset
- 2,494 examples (2,405 train, 89 validation), synthetic, built from 50 real boards and expanded into many board states. Clues were checked by simulation and by the legality check used to score the benchmark. Clues the model wrote were kept when rated good or ok, and replaced with human-written ones when bad.
- Data processing
- The sealed benchmark boards, and boards close to them, are kept out of training.
Implementation
- Hardware
- One Nvidia Tesla T4, trained in 16-bit.
- Output format
- GGUF, Q4_K_M, 2.1 GB.
Distribution
- Channels
- Hugging Face.
- Requirements
- About 3 GB of free memory. A 6 GB graphics card runs it fully on the GPU.
Evaluation
- Approach
- Cluebench v1.0. Held-out boards sealed before training, in three kinds: fresh boards, midgame (4 or more of your words left) and endgame (3 or fewer). Every model got the same board and format, one answer at temperature 0. Every clue was rated good, ok or bad in a blind rating. Cluemaster was benchmarked as the Q4_K_M file.
- Results
- As of October 2026.
Overall
| Model | Cluebench | Rated good | Acceptable | Legal first try | Illegal | Invalid | Seconds per board | Tokens per answer |
|---|---|---|---|---|---|---|---|---|
| Cluemaster 1.0 | 60% | 44% | 68% | 96% | 4% | 0% | 2.5 s | 93 |
| Phi-4 Mini 3.8B | 20% | 17% | 34% | 63% | 37% | 1% | 1.4 s | 33 |
| Granite 4.1 3B | 20% | 12% | 33% | 76% | 24% | 2% | 0.6 s | 19 |
| Ministral 3 3B | 5% | 5% | 19% | 61% | 39% | 6% | 2.8 s | 53 |
| Qwen 3.5 4B | 3% | 13% | 17% | 25% | 75% | 0% | 1.9 s | 29 |
Acceptable is legal and rated good or ok. Illegal is a clue that breaks a rule. Invalid is a legal clue aimed at a neutral or opponent word.
Fresh boards
| Model | Score | Good | Acceptable | Illegal |
|---|---|---|---|---|
| Cluemaster 1.0 | 72 | 45% | 85% | 5% |
| Phi-4 Mini 3.8B | 33 | 32% | 47% | 37% |
| Granite 4.1 3B | 22 | 12% | 37% | 25% |
| Ministral 3 3B | 7 | 8% | 23% | 45% |
| Qwen 3.5 4B | 0 | 2% | 5% | 90% |
Midgame
| Model | Score | Good | Acceptable | Illegal |
|---|---|---|---|---|
| Cluemaster 1.0 | 40 | 32% | 45% | 5% |
| Phi-4 Mini 3.8B | 11 | 5% | 27% | 36% |
| Granite 4.1 3B | 16 | 9% | 27% | 23% |
| Ministral 3 3B | 3 | 0% | 14% | 27% |
| Qwen 3.5 4B | 9 | 23% | 27% | 68% |
Endgame
| Model | Score | Good | Acceptable | Illegal |
|---|---|---|---|---|
| Cluemaster 1.0 | 67 | 54% | 74% | 3% |
| Phi-4 Mini 3.8B | 16 | 16% | 29% | 38% |
| Granite 4.1 3B | 23 | 16% | 34% | 24% |
| Ministral 3 3B | 5 | 7% | 19% | 38% |
| Qwen 3.5 4B | 1 | 15% | 18% | 65% |
Scoring
| Clue | Points on its board |
|---|---|
| Good | 100 |
| Ok | 70 |
| Bad, or no clue | 0 |
| Invalid: a target is a neutral word | −10 |
| Invalid: a target is an opponent word | −15 |
| Illegal, whatever it was rated | −25 |
A segment scores 50% legal, 30% good and 20% acceptable, minus the penalties, never below zero. Cluebench is the mean of the three segments.
Intended usage and limitations
- Intended usage
- Giving clues as the spymaster in English Codenames.
- Midgame
- The weak spot. Cluemaster scores 40 on midgame boards against 72 on fresh boards and 67 on endgames.
- Illegal clues
- About 4% of single answers are illegal, mostly by being or containing a board word. The harness filters these out.
- Benchmark conditions
- The clues were produced with a modified version of the harness with fewer safety checks, so the numbers show the model by itself.
- Raters
- One rater. The comparison between models is solid; small differences are not.
- Guessing
- Trained only to give clues, not to guess.
- Language
- English, with one common Codenames-style word list.