Assessment

Assessment.

In the comments on my post about joining Rostec’s management talent pool, Aleksejs Babkins asked how candidates could be assessed in such a short time. I promised him a post on the subject, since I know it both in theory, from a course at RANEPA, and in practice: I served as an expert in the It Runs in the Family competition.

If you are interested in the details, take a look through my presentation on the subject. It includes an example of how the assessment system for the Perm Krai – Territory of Culture programme evolved (the bibliography links to a more detailed account).

In brief, assessment rests on a few key ideas:

• It should have several different stages, so that candidates are assessed from multiple angles.
• We need to understand exactly what we want to assess at each stage.
• There should be a great many scores, so that one, two or three outliers cannot radically alter the result.
• Experts should rotate, so that the same project, person or team is assessed by many independent experts.

The most interesting stage is the in-person assessment. The person, team or project being assessed is placed in a simulated—and preferably stressful—situation, while an expert observes how the participants behave. Each criterion has a set of phrases or actions that indicate whether or not it is present.

For example, when assessing teamwork, an expert might count “Great job, everyone!” as a positive marker, while someone making a show of taking offence at a teammate would count as a negative one. The expert has around a dozen such markers, sometimes more, each with its own weighting. They record whether each marker is present or absent. Those observations are then multiplied by their weightings, added to the scores from other experts, averaged, normalised, and so on. In the end, all these tiny grains of evidence combine into a single overall score.

Why is the system hard to game, even when you know how it works? Because you cannot know exactly what will be assessed. This year, for example, the final of Leaders of Russia assessed one set of criteria, while the superfinal assessed the opposite qualities. Anyone who excelled in the final and carried the same approach into the superfinal lost.

So my best advice to anyone taking part in an assessment like this is: do not try to guess the criteria or tailor your behaviour to them. Just be yourself!