How detailed does your Competency framework need to be?

Increasingly and as part of a broader talent management strategy, we are helping client's analyse and develop competency frameworks. Working with large numbers of these you do begin to see patterns and consistent themes, but we do also see a great variation in the depth, breadth and structure of competency models.  If you'd like to see some examples that span this spectrum, read on and do contact us.

When developing a framework to underpin your talent management efforts, it can be difficult to strike a balance between something that will apply to most and making it too generalised to be useful. Our first step, is to ask why and how with the framework be used.  We often create 'personas' of people that intend to use the framework in applications such as recruitment, performance management, talent reviews, leadership development and so on.  With this 'lens' firmly in mind, we can begin to set the right level and continuously refer back to the 'how' and 'why' to keep the project outcome-focused.

Whilst staying grounded in the application of competences, we can't ignore the sound research that exists and has stood the test of time.  

James Meachin and Stephan Lucks (reported in the BPS’s Assessment and Development Matters, Vol. 2 No. 3, 2010) explored the optimal level of ‘granularity’ for competency frameworks when used as predictor measures and assessment criteria. Research into the effectiveness of various personality constructs to predict job performance suggests that some of the broad measures, such as the Big Five, have limited predictive validity, but that this might be improved when you correlate job performance with some of the finer-grain sub-traits, such as ‘dependability’. This would suggest that better predictions of job performance are made by fine-grain, or more specific, behavioural criteria.

Based on the literature, Meachin and Lucks hypothesised that assessment centre ratings which were based on a fine-grain competency framework would produce better correlations between conceptually-matched job performance measures (line manager ratings). In other words, they’d result in a more accurate prediction of high performance on the job. Interestingly, what they actually found is that the predictor measures showed stronger correlations with line manager performance ratings as they became broader, not narrower. Aggregating the competency scores into a general, overall measure of performance seemed to be a more reliable way of predicting high-performing individuals than picking on their performance in specific competency areas.

For those not immersed in the research environment, this tells us that in order to create robust assessment processes, which differentiate between higher and lower performing candidates, we need our competency framework to provide depth and a level of detail which explicitly includes the behaviours and competencies which are important to success.

Undoubtedly, in the arena of assessment for development purposes, the value is in the detail – in helping people understand the specific aspects of their performance or behaviour which makes them more or less effective. 

So to keep the level of detail in check, the optimal situation is to have a detailed, granular competency framework, which sets out the specific behavioural indicators across a number of competencies (no less than 6, and no more than 12). By collecting assessment data against your framework (through performance appraisal, assessment processes, or 360 degree feedback), you should then perform a 'factor 'analysis (or ask us to do this or help you with it!) on your competencies.  This will determine whether there are any coarse-grained factors within it that may result in perhaps two or three clusters of competencies.

Aggregating competency scores in line with these underlying factors and making decisions based on these broader measures is likely to improve the reliability of the framework, for it to appear more 'valid' to those that use it, and ensure that your investment is as future-proof as possible.

You may like to learn more about our work with customers to develop or shape their competency framework.  If so, please get in touch.

You may also be interested in this article.