Correct or incorrect is one bit of information per question. It is enough to compute a percentage and almost nothing else. Add how sure you were, before you saw the outcome, and the same attempt becomes genuinely diagnostic.
Four outcomes, not two
- Sure and right. Finished. Push it a long way out.
- Sure and wrong. The dangerous one. An active false belief with no warning attached, and it will not appear on any revision list you write yourself.
- Unsure and right. Possibly a guess, possibly reasoning. Either way not yet knowledge, and it should not be filed as settled.
- Unsure and wrong. The honest gap. Unpleasant but harmless, because you already know to come back.
Only the first of those is done. A percentage score merges the second and the third into the same undifferentiated pile as the first and fourth respectively, which is why practice percentages are such poor predictors.
Two candidates on 70% can be in completely different positions, and the difference is entirely in the confidence data neither of them recorded.
Keep the scale small
Three levels is right: sure, fairly sure, guessing. Five is worse — you will spend attention deciding between adjacent levels that you will never treat differently, and the decision cost is charged on every question.
Rate before, always
The rating has to happen before the answer is revealed. Afterwards you are not reporting your state at the time, you are reconstructing it, and the reconstruction is contaminated by knowing whether you were right. People reliably remember having been more certain about things they got right.
Calibration is a skill, and it transfers
Recording confidence does something beyond sorting your history: it trains your sense of what you actually know. Most people start badly calibrated, usually over-confident, and improve within a few hundred questions simply from getting immediate feedback on the judgement.
That improvement is worth marks directly. Under +4/−1 the decision to answer or leave blank is a judgement about your own reliability, made in seconds, 180 times. A candidate who knows when they are guessing makes that call better than one who does not.
What good calibration looks like
- Things you called sure should be right the large majority of the time. If they are not, your confidence is running ahead of your knowledge.
- Things you called guessing should hover near chance. Much above, and you know more than you credit yourself with, which is its own useful discovery.
- The gap between the two should be wide. A narrow gap means your confidence carries little information, which makes the exam-day blank-or-answer decision a coin flip.
The under-confident case
Less discussed and quite common, particularly among careful candidates: getting things right while consistently rating yourself unsure. It costs marks in a specific way — hesitation, second-guessing, time lost re-checking answers that were correct the first time. If your "guessing" pile is running at 60% correct, the problem is not knowledge, and more studying will not fix it.
Being wrong about how much you know costs marks in both directions.
The discomfort is the point
Most people find it faintly unpleasant to commit to "sure" before submitting, and that reaction is informative in itself. It forces the question of what you actually believe, which is a more useful moment than any amount of reviewing afterwards.
In Recallab the rating is a single tap before you submit, and it drives three live things: the False Certainty and Lucky Guesses apps, and the interval Due Today gives the question.