Skills Assessment: How to Measure Employee Competencies
Quick Summary
How to run an employee skills assessment that holds up. Compare 5 methods on reliability and cost, borrow a sample competency rubric, and act on gaps.
A skills assessment is a structured way to measure whether someone can do the work a role requires, at the level it requires. The method has to match the skill. Use knowledge tests for what people know, simulations and work samples for what they can do, and 360 feedback or self-assessment for how their behaviour is experienced and perceived. The assessments that hold up score observable behaviour against a written rubric, and someone acts on the results.
This guide is for employers, L&D teams and sales enablement leads measuring their own people. If you're a job seeker looking to rate your own skills, the US Department of Labor's CareerOneStop Skills Matcher is a better starting point.
What a skills assessment should tell you
A useful employee skills assessment answers three questions:
- Can this person do the task? That's proficiency, measured against a defined standard, not against their colleagues.
- Where is the gap? You need a specific behaviour, such as "doesn't confirm the customer's priority before pitching", not a generic one like "communication".
- What should happen next? That might be practice, coaching, a course, or nothing.
Most programmes manage the first question and stall on the next two. A quarterly quiz produces a score but rarely a specific gap. A self-rating survey produces opinions but no evidence. The fix is mostly in choosing the right method for each competency.
Five skills assessment methods compared
There's no single best method, because each measures a different thing. The reliability ratings below are our judgement of how consistent the scores are when the method is run well. Where the US Office of Personnel Management (OPM) has published guidance, it's linked.
| Method | What it measures | Reliability | Cost to run | Best use |
|---|---|---|---|---|
| Knowledge test | What someone knows: product, policy, process | High, if items are reviewed | Low per test; custom tests take effort to build | Onboarding, compliance, prerequisites |
| Simulation / roleplay | Applied behaviour in a realistic, controlled scenario | Medium–high, with a fixed scenario and an anchored rubric | Medium; lower with AI roleplay than with human actors | Readiness checks before going live |
| Work sample / observed work | Actual performance on real tasks | Medium–high, with one rubric and calibrated raters | High with human observers; lower if recorded and scored automatically | Proving on-the-job transfer; coaching priorities |
| 360 feedback | How colleagues experience someone's behaviour | Medium; depends on rater count and item wording | Medium | Leadership and manager development |
| Self-assessment | Confidence and perceived gaps | Low as a measure of proficiency | Very low | Opening a development conversation; spotting blind spots |
A few points the table can't fit:
- Knowledge tests are fair but narrow. OPM notes that applicants "often perceive job knowledge tests as being very fair". It also notes they measure what a person knows "at the time of taking the test," not whether they can apply it (OPM: job knowledge tests). A rep can score 100% on pricing policy and still discount under pressure.
- Work samples and simulations are the most realistic, and historically the most expensive. OPM says they "may be costly to develop, both in terms of time and money," and that administering them is "time consuming and expensive." It also says people see them as fair and that they relate closely to job performance (OPM: work samples and simulations). Recording real work and AI roleplay have changed the cost side of that trade-off far more than the validity side.
- Self-assessment needs a check. OPM warns that "inflation or distortion" undermines self-reported ratings and recommends telling people responses will be verified (OPM: training and experience evaluations). Self-ratings are most useful next to an observed score: the gap between the two is the coaching conversation.
Our recommendation: for any role built on conversations, such as sales, customer success, support or people management, combine one knowledge check, one simulation and one observed-work measure. That combination tells you whether a gap is about knowing, doing, or doing it under real conditions.
How to run a skills assessment in six steps
- Start from the job, not a competency library. List the five to eight tasks that most affect results in the role. For a field sales rep that might be opening a meeting, discovering needs, presenting a solution, handling a price objection and agreeing a next step.
- Write observable indicators for each task. "Builds rapport" can't be scored. "States the purpose of the meeting and checks it matches the customer's agenda" can.
- Pick a method per task, using the table above. Product knowledge gets a test. Objection handling gets a roleplay and an observed call.
- Build a rubric with anchored levels. Three or four levels, with each cell describing what you'd see or hear. There's a sample below.
- Calibrate before you score for real. Have two or three raters score the same five recordings independently, then discuss the disagreements. Where they disagree, the wording of the indicator is usually the problem. If AI does the scoring, check it the same way against human raters on a sample.
- Decide in advance what each result triggers. A low opening score triggers a specific practice scenario and a manager check-in. If a score doesn't trigger anything, ask whether you need to measure it at all.
Sample competency rubric: running a customer meeting
This rubric scores one competency, running a customer meeting, across four stages. It works for live observation, recorded calls or roleplay, which is the point: the same rubric applied to practice and to real work lets you see whether training carried over.
| Stage | 1. Developing | 2. Proficient | 3. Advanced |
|---|---|---|---|
| Opening | Jumps into the product; no agenda | States the purpose and confirms the time available | Links the purpose to something the customer said before; agrees an agenda |
| Understanding the customer | Asks mostly yes/no questions | Asks open questions about current process and pain | Quantifies the impact and confirms priorities in the customer's words |
| Identifying solutions | Presents every feature | Matches 1–2 capabilities to stated needs | Proposes a solution tied to the customer's cost or outcome, with evidence |
| Close / next step | Ends with "I'll send something over" | Agrees a dated next step | Agrees a next step with owners on both sides, and confirms who else decides |
Score each stage 1–3 and require a quote or timestamp as evidence for every score. Evidence is what makes the rubric defensible when a rep disputes a score. It's also what turns a number into a coaching conversation.
What observed-behaviour assessment reveals: a field example
Here's what happens when you assess real work instead of a sample of it. In a pilot, a European B2B field sales team (37 reps) recorded 224 real, in-person customer meetings with Shiken's meeting recorder. Every meeting was scored on a 4-stage meeting rubric similar to the one above. Before the pilot, coaching depended on the meetings a manager happened to attend.
What came out of it:
- The weakest stage across the team was the opening. The strongest was identifying solutions. A knowledge test would never have shown this, because reps knew how to open a meeting. It showed up only when real openings were scored consistently.
- The overall average stayed flat at about 47/100 across the pilot. That's the uncomfortable lesson. Measurement on its own doesn't change behaviour. The score becomes useful once managers coach against it, for example a team session on openings and a practice scenario for the lowest scorers.
- One rubric worked across languages. About 40% of the meetings were in Portuguese, Spanish or Italian. Scoring every meeting against the same rubric meant results could be compared across reps and countries. Separate local scorecards would have made that impossible.
The broader point applies even if you never record a meeting. Assess the behaviour in the real setting, use one rubric across everyone, and plan the coaching loop before you start scoring.
From scores to a skills gap analysis
A skills gap analysis is just the assessment results compared with the level each role needs. Keep it simple:
- Set a target level per role, per competency. A new account executive might need Level 2 on every stage of the rubric. A team lead needs Level 3 on the opening and the close, because they'll be coaching both.
- Look for team-wide gaps before individual ones. If most of the team scores low on the same stage, as with the openings above, fix it with team training, not individual performance plans.
- Re-assess with the same instrument. Use the same scenario, rubric and raters. If you change the test between rounds, you can't tell whether people improved or the test got easier.
- Track the trend, not the snapshot. One score is a data point. Three scores a month apart show whether coaching is working.
Where software fits
You can run all of this with spreadsheets and calendar time, and many teams should start that way to get the rubric right. Software earns its place when volume makes manual scoring impossible:
- Knowledge tests: any quiz tool or LMS that reports results at the question level. Our AI quiz generator comparison covers the trade-offs.
- Pre-hire testing: dedicated hiring-assessment platforms are a separate category built for candidate screening, not for developing existing staff.
- Simulations: AI roleplay tools let everyone practise the same scenario and be scored against the same rubric. Compare the options in our AI roleplay platform comparison.
- Observed work and skills profiles: conversation-intelligence and skills-intelligence tools score real calls against a rubric and roll the evidence into a skills profile. Shiken's Skills Intelligence is one example. It combines meeting rubric scores, roleplay grades, quiz mastery, and manager and self-assessments, and links each skill back to its evidence.
Whatever you use, tell people what is being assessed, why, and how the results will be used. Do that before the first recording or test, not after.
Frequently asked questions
What is an example of a skills assessment?
A customer-success manager is given a recorded roleplay: a customer threatens to cancel over a missed deadline. Their response is scored on a four-level rubric covering acknowledgement, diagnosis, the recovery offered and the agreed next step, with a quote supporting each score. That's a skills assessment. A 20-question quiz on the cancellation policy is a knowledge test.
What is involved in a skills assessment?
Defining the tasks that matter in the role, writing observable indicators for each, choosing a method per task (test, simulation, work sample, 360 or self-rating), scoring against a rubric, and agreeing in advance what each result triggers. Calibrating raters, or checking AI scoring against human raters, is the step most teams skip.
How often should you assess employee skills?
Assess at moments that matter: a baseline before training, a check once practice is done, and a follow-up on real work four to eight weeks later. For roles where every conversation can be recorded and scored, assessment can be continuous. The review cadence (monthly 1:1s, for example) then matters more than the test cadence.
Is a skills assessment the same as a performance review?
No. A performance review judges results over a period, such as targets hit or projects delivered. A skills assessment measures the capabilities behind those results. Keep them separate. If people believe a practice roleplay feeds directly into their rating, they stop treating it as practice.
