Language model
Ask Alfonso
Ask Alfonso
Downloading the model (7.7 MB, once)… 0%
Small models make mistakes. On questions it had never seen, it answered 78% exactly right and never invented a fact; when unsure, it says so. For anything important, email javs3@uw.edu.
How it works
A language model I built from scratch
Tokenizer, transformer, training, retrieval and guardrails, all written by me and trained only on my real data.
- 3.9M
- parameters, trained from random weights
- 78.1%
- exact answers on 254 questions it never saw
- 0 / 202
- answers that invented a fact
- $0
- to serve: 7.7 MB, runs in your browser
How it answers
Four steps, each one measured
A tiny model will make things up if you let it, so it never answers alone. Numbers are from the 254-question test set.
- 01 · Guard
Private topics stop here
Age, address, phone or finances get a polite refusal before the model even runs.
43/43unsafe questions handled
- 02 · Retrieve
TF-IDF finds the facts
Ranks 34 chunks of my profile and keeps only the confident matches (τ = 0.10).
99.0%right chunk in the top 3
- 03 · Generate
The transformer answers
Reads up to 3 chunks plus the question and writes the reply one token at a time.
768tokens of context
- 04 · Verify
Guardrails check every claim
Each name, date and number must appear in what it read; if not, a safe fallback is shown.
0invented facts on the test set
Inside the model
A 3.9M-parameter transformer
The same decoder-only design as GPT, scaled down about 32× from GPT-2 so it trains on one GPU and runs in a browser tab.
BPE tokenizer
2,053 tokens, ~4 characters each
Token + position embeddings
Each token becomes a 256-number vector
Causal self-attention
4 heads × 64 dims; each token only sees the past
MLP
256 → 1,024 → 256, GELU
pre-LayerNorm + residual connection around each
Output head
Scores all 2,053 tokens; reuses the embedding weights
Where the 3,870,208 parameters live
- MLP2.10M54%
- Attention1.05M27%
- Embeddings722K19%
- Layers
- 4
- Attention heads
- 4
- Width
- 256
- Context
- 768 tokens
- Vocabulary
- 2,053
- Weights shipped
- 7.7 MB
Every layer, the attention math and the training loop are written by hand in PyTorch: no pretrained weights, no nn.Transformer, no Hugging Face. The browser version is a from-scratch TypeScript port.
Training
Six versions to get here
First 3,000 steps of general English (WikiText-2, loss 7.69 → 2.87), then 80 epochs on 4,497 questions about my work, with new phrasings and made-up projects every epoch so it learns to read, not memorize.
Final run: loss per epoch
View as table
| Epoch | Train loss | Dev loss |
|---|---|---|
| 1 | 2.536 | 0.721 |
| 2 | 0.575 | 0.403 |
| 3 | 0.339 | 0.355 |
| 4 | 0.245 | 0.282 |
| 5 | 0.196 | 0.274 |
| 6 | 0.163 | 0.248 |
| 7 | 0.142 | 0.218 |
| 8 | 0.123 | 0.247 |
| 9 | 0.116 | 0.263 |
| 10 | 0.11 | 0.231 |
| 11 | 0.099 | 0.235 |
| 12 | 0.089 | 0.258 |
| 13 | 0.087 | 0.235 |
| 14 | 0.085 | 0.212 |
| 15 | 0.082 | 0.234 |
| 16 | 0.073 | 0.222 |
| 17 | 0.075 | 0.232 |
| 18 | 0.071 | 0.233 |
| 19 | 0.067 | 0.226 |
| 20 | 0.065 | 0.223 |
| 21 | 0.064 | 0.236 |
| 22 | 0.061 | 0.205 |
| 23 | 0.06 | 0.213 |
| 24 | 0.058 | 0.22 |
| 25 | 0.055 | 0.216 |
| 26 | 0.056 | 0.21 |
| 27 | 0.051 | 0.208 |
| 28 | 0.051 | 0.215 |
| 29 | 0.05 | 0.225 |
| 30 | 0.052 | 0.221 |
| 31 | 0.049 | 0.205 |
| 32 | 0.047 | 0.231 |
| 33 | 0.046 | 0.199 |
| 34 | 0.045 | 0.197 |
| 35 | 0.042 | 0.201 |
| 36 | 0.045 | 0.22 |
| 37 | 0.044 | 0.224 |
| 38 | 0.042 | 0.218 |
| 39 | 0.039 | 0.212 |
| 40 | 0.042 | 0.207 |
| 41 | 0.037 | 0.214 |
| 42 | 0.038 | 0.21 |
| 43 | 0.039 | 0.199 |
| 44 | 0.036 | 0.214 |
| 45 | 0.034 | 0.22 |
| 46 | 0.033 | 0.217 |
| 47 | 0.031 | 0.224 |
| 48 | 0.038 | 0.233 |
| 49 | 0.034 | 0.208 |
| 50 | 0.033 | 0.227 |
| 51 | 0.032 | 0.204 |
| 52 | 0.029 | 0.218 |
| 53 | 0.03 | 0.226 |
| 54 | 0.03 | 0.218 |
| 55 | 0.032 | 0.215 |
| 56 | 0.029 | 0.219 |
| 57 | 0.029 | 0.212 |
| 58 | 0.026 | 0.195 |
| 59 | 0.029 | 0.196 |
| 60 | 0.026 | 0.196 |
| 61 | 0.025 | 0.19 |
| 62 | 0.026 | 0.206 |
| 63 | 0.027 | 0.2 |
| 64 | 0.023 | 0.191 |
| 65 | 0.026 | 0.195 |
| 66 | 0.022 | 0.199 |
| 67 | 0.022 | 0.193 |
| 68 | 0.022 | 0.184 |
| 69 | 0.024 | 0.178 |
| 70 | 0.023 | 0.188 |
| 71 | 0.024 | 0.184 |
| 72 | 0.02 | 0.177 |
| 73 | 0.022 | 0.182 |
| 74 | 0.021 | 0.183 |
| 75 | 0.022 | 0.18 |
| 76 | 0.019 | 0.183 |
| 77 | 0.021 | 0.181 |
| 78 | 0.022 | 0.188 |
| 79 | 0.019 | 0.188 |
| 80 | 0.023 | 0.189 |
What changed between runs
- v462.7%
3.9M params · Clean data, balanced phrasings · test 68.1%
- v5a51.7%
11.7M params · 3× bigger, 25× more pretraining
- v5b54.1%
11.7M params · Same, higher learning rate
- v675.5%
3.9M params · v4 + a paraphrase grammar · test 78.1%
Bigger was worse
3× the parameters and 25× the pretraining scored up to 11 points lower on new phrasings. It memorized faster.
Data beat size
Generating fresh phrasings every epoch took the same 3.9M model from 68.1% to 78.1% on the test set.
Dev sets wear out
After five rounds of decisions, dev said 94% and test said 78%. A separate 453-question stress set predicted test within 3 points.
- AdamW + weight decay
- Warmup, cosine decay
- Dropout
- Gradient clipping
- fp16 mixed precision
- Early stopping
- Answer-only loss
- Resumable checkpoints
Results
Tested once, on 254 questions it never saw
The test set was locked away until the final model was chosen, so these numbers weren't tuned to look good.
- 80.0%
- of answers contain every key fact
- 82.2%
- word overlap (F1) with the reference answer
- 89.5%
- of questions find the right chunk first
- 3.8%
- said “I'm not sure” instead of guessing
Shipped for $0
- 7.7 MB
- float16 weights, half of float32
- 86/86
- answers identical to PyTorch
- ~1 s
- per answer, on one CPU thread
- 0
- servers or API calls
The weights, tokenizer and retrieval run in a Web Worker in your browser. A test suite checks that the TypeScript version produces the same tokens, retrieval and answers as PyTorch on 86 reference questions.
Honest limitations
- 18.1% of answers give a true fact for a slightly different question ("What impact did SkySAR have?" gets SkySAR's summary). Wrong, but never invented.
- It only knows what's on this site and only speaks English. It isn't a general chatbot.
- Small on purpose: about a second per answer and 768 tokens of context, in exchange for $0 hosting and nothing leaving your browser.