Choose a small set of job-relevant competencies
Start with four to six competencies tied to the work. For a software engineering role, these might include problem decomposition, implementation reasoning, testing, collaboration and explaining trade-offs. Avoid vague categories such as “culture fit” that different reviewers interpret differently.
Give each competency a question or task that can produce evidence. If a question tests several things at once, note which part of the answer informs each rating. Keep the same core criteria across candidates for the same vacancy.
Use this reusable scorecard structure
Record the role, interview stage, reviewer, date and rubric version. For each competency, use the following fields. Keep evidence and interpretation separate so a later reviewer can disagree with the rating without losing the original response.
- Competency: the ability being assessed and why it matters for this role.
- Question or task: the prompt used and any follow-up that changed its scope.
- Evidence: a short quote or faithful summary, with a transcript reference when available.
- Rating: a defined scale with an explicit “not enough evidence” option.
- Reason: how the response matches the rating anchor.
- Follow-up: what the next interviewer should verify.
Anchor the scale with observable behaviour
For problem decomposition, a simple four-point scale could be: 1 — does not identify the main constraint even after clarification; 2 — identifies part of the problem but needs substantial prompting; 3 — breaks the problem into workable steps and explains the main constraint; 4 — does that and tests a relevant alternative or edge case.
These are example anchors, not a universal engineering assessment. Adapt them to the seniority and task. Add “not assessed” when the interview did not provide a fair opportunity to demonstrate the skill. Do not silently convert a microphone failure or an unanswered ambiguous question into a low ability rating.
Calibrate AI and human review against the same rubric
Have two reviewers independently assess a small sample before using the rubric broadly. Compare the evidence they chose, not just the total score. Disagreement often exposes an unclear question, an undefined anchor or an expectation that was never included in the job description.
When using AI interview scorecards in Recruit, read the response evidence and transcript alongside the recommendation. Distinguish an observed fact from an interpretation. Verify any unsupported claim before it affects a candidate’s progress.
Make the next step explicit
Finish with a recommendation, its rationale and the remaining uncertainty. “Advance to a debugging exercise to verify testing strategy” is more actionable than “good candidate”. Keep the decision owner and the actual decision separate from the generated recommendation.
Store the rubric version with the review so changes can be understood later. Review the employer’s automation settings and give candidates a route to request human review. A scorecard supports a consistent discussion; it does not establish that a hiring process is fair or legally compliant.
For candidate information, read the AI screening notice and privacy policy.