Defining Skill Levels That Mean Something

A self-assessment goes out across a department on a five-point scale. Almost everyone selects three. The handful who select five are, on inspection, not the strongest practitioners; several of the strongest select four, because they have a clear enough picture of the field to know what five would actually require. The resulting dataset is a study of temperament with a skills label on it.

This is not a failure of the people filling in the form. It is a failure of the scale. Beginner, intermediate, advanced and expert are adjectives, not measurements. They are empty containers that each rater fills with their own standard, which is why two managers assessing the same person routinely land two levels apart and both feel confident.

A level needs three things

A task

What can this person complete? Stated as work, not as knowledge. “Understands normalisation” is unfalsifiable. “Can design a schema for a new transactional service and defend the trade-offs in review” is something that either has or has not happened. The task descriptor is what stops a level from being a feeling.

A condition

This is the axis that does the most work and the one most often missing. The same task performed under different conditions represents very different levels of capability:

• Can do it with close supervision and a worked example to follow.
• Can do it independently on routine cases, and knows when to ask.
• Can do it independently on ambiguous cases, where the requirements are contested or incomplete.
• Sets the standard for how it is done, and is the escalation point when others are stuck.

Those four conditions are stable across almost every skill in an organization, which is what makes them useful as a backbone. The task descriptor changes; the shape of the ladder does not.

An evidence rule

What would settle a disagreement between two raters? A piece of delivered work, an assessment result, an observation against a defined checklist, a record of having handled a specific class of problem. If nothing could settle it, the level is decorative, and everybody will eventually work that out. Attaching evidence is also what makes a skill rating survive contact with a promotion committee.

Fewer levels than you think

Five-point scales usually collapse into three real distinctions under examination: cannot do it alone, can do it alone, can be relied on when it is hard. The extra levels buy a finer progression narrative that people find motivating, and they cost calibration accuracy, because the boundaries between adjacent levels stop being distinguishable in practice.

Four is the compromise worth defending: one learning state, two working states separated by whether the case is routine or ambiguous, and one state for people who set direction for others. Going beyond five is a decision to have conversations about whether someone is a three or a four, forever, with no decision resting on the answer.

One scale, or one per skill

A single organization-wide scale is comparable, cheap to teach and easy to roll up into a report. Skill-specific scales are more accurate and impossible to aggregate, which means the organization gets precision at the level of the individual and nothing at the level of the portfolio.

The compromise that survives contact with reality is a universal structure with skill-specific content: the same four conditions everywhere, with task descriptors written per skill by someone who does the work. You keep comparability, you keep meaning, and the cost is that each skill needs an hour of somebody competent writing four sentences. That is a real cost and it is the reason to keep the skill model small enough that the writing is achievable.

Levels decay, so date them

A level measured two years ago, against a version of a technology that no longer exists, is a historical claim rather than a current fact. Every rating should carry its source and the date it was established, and the interface should show the age rather than hiding it behind a number that looks equally fresh whether it is two weeks or two years old.

Decay rates differ and the policy should reflect that. Fast-moving technical skills are worth re-measuring annually. Judgment-heavy capabilities like facilitation or negotiation decay slowly and do not justify the same cadence. Re-measuring everything on the same schedule is how a programme collapses under its own administrative weight in the second year, taking the reskilling work that depends on it with it.

Calibration is the actual work

Write the descriptors well and raters will still disagree, because the descriptors are read by people with different standards. The fix is a calibration session: several raters independently rate the same anonymised evidence, then argue about the differences. The disagreements point exactly at the descriptors that are vague, which is information you cannot get any other way.

Run it twice a year, publish the worked examples as reference cases, and accept that this is not overhead — it is the mechanism that makes the numbers comparable across teams. A scale without calibration produces data that looks consistent and is not, which is a worse position than having no data, because people act on it.

What levels are not for

Be careful about wiring levels directly into pay. The moment a level determines money, the rating stops being a measurement and becomes a negotiation, evidence gets curated, and managers start advocating rather than assessing. Levels do their best work in development planning and staffing, where the incentive is to be accurate because an inflated rating leads directly to being handed work you cannot do.

If levels do eventually feed compensation, budget for the verification that becomes necessary: independent assessment, evidence review, and an appeals route. That is a fair trade, but it should be an explicit decision rather than something that happens gradually because a spreadsheet was convenient.