jargon
A learning app that rewrites technical terms through each learner's own work, so a familiar job can provide a starting point for an unfamiliar concept.
decision
A learner uses the rewrite to approach an unfamiliar term, then checks the canonical entry before relying on the analogy. The verifier decides whether to show, retry or replace a rewrite.
baseline
The canonical definition is the simpler alternative and the fallback. Personalization adds a familiar comparison, but no learning study here shows it improves understanding over that definition.
review
I reviewed the five identified wrong outputs as a pharmacist. A learner can return to the canonical text. Errors, timeouts and a second failed check return that text rather than an unchecked rewrite.
data
The evaluation uses synthetic learner profiles. The linked implementation generates and verifies through model calls, so the prompt includes the term and learner context. This portfolio calls no model. Retention, access and deletion across a deployed model service are not audited here.
platform
The reusable unit is a generate, verify, retry and fallback pipeline consumed by the learning app. Its evaluation imports the same prompts and parsers. The verifier measures acceptance, not truth, and its five misses prevent treating it as an independent safety gate.
limits
One model family and one incomplete run. The five confirmed errors are a lower bound, not a review of every accepted answer. There is no measured learning or clinical benefit.
Evaluation
Run 2026-10-30. claude-opus-4-7 for every pass (generator, verifier, regenerator), default temperature, concurrency 3. 40 seed terms (8 each from AI/LLM, data, security, APIs and cloud) crossed with 3 synthetic learner profiles: a hospital nurse, a retail pharmacist and a high-school teacher. Every profile starts with an empty learned-vocabulary list. A $10 budget guard stopped the run after 79 of 120 planned items. A parity check against the pre-refactor file confirmed that all 747 prompt strings, builder outputs, parser results and constants tested were byte-identical.
Outcomes (n = 79): Accepted first try | 76 | 96.2%
Outcomes (n = 79): Accepted after one retry | 2 | 2.5%
Outcomes (n = 79): Fell back to canonical text | 1 | 1.3%
Verifier verdicts: Pass-1 verifier verdict = pass | 76 of 79 | 96.2%
Verifier verdicts: Retry verdict = pass | 2 of 2 | 100%
Verifier verdicts: technical_accuracy_preserved failed | 0 of 78 | 0%
Verifier verdicts: analogy_target_accurate failed | 0 of 78 | 0%
Verifier verdicts: Accepted outputs that used an analogy | 18 of 78 | 23%
By profile: Hospital nurse | 27 | 26 (96.3%) | 1 (3.7%) | 0
By profile: Retail pharmacist | 26 | 26 (100%) | 0 | 0
By profile: High-school teacher | 26 | 24 (92.3%) | 1 (3.8%) | 1 (3.8%)
Cost and latency: Total spend, 79 runs | $8.77 (function formula: $8.20)
Cost and latency: Mean cost per item | $0.111
Cost and latency: Accepted first try, mean | $0.109
Cost and latency: Accepted after retry, mean | $0.225
Cost and latency: Latency p50 / p95 / max | 10.9 s / 16.9 s / 24.1 s
- 6.4% is a floor, not a full audit. An AI model read all 79 outputs and flagged five, I reviewed those five as a pharmacist and confirmed all five are wrong. The other accepted outputs were read by the AI model only.
- n = 79 from one run of a non-deterministic model, with no repeats. I wrote the term set and the profiles.
- The verifier is the same model as the generator. Read the rates as "the verifier rarely rejects", not as a measured accuracy figure.
- Costs use the pricing constants in the function, not checked against current list prices. Cold-start vocabulary only. The run did not touch the database, auth, caching or the spend guard.