5 Factors to Evaluate an Assessment Instrument-Assessment 103.

How do you decide on a an Assessment instrument . I would consider the following 5 factors.

1. Productivity increase and the cost of assessment
2. Competency Supply and the False Positive & False Negative Trade-off
3. Validity
4. Reliability
5. Assessment Experience

Productivity Increase and the the cost of Assessment

The basis of Assessment is to differentiate between the good and average or between average and those we should not hire. This can only be done if you have productivity comparisons between the good and average available.

• Reed Hastings at Netflix believes that the average difference between an average and the best is not 5X it is 10 X . Steve jobs used to say the difference between average and great software engineer is 100X.
• Hastings goes on to extrapolate this to differentiate between operational and creative roles. In operational roles the productive differences are not so much ,but in creative roles they can be humongous .So customer service is an operational role but content curation and creation are creative roles in a firm like NetFlix. If you have been following our blog- this is the difference between roles with high percentage of programmable decisions to high percentage of non-programmable decisions.
• In my experience for Recruitment ; the best outperform the average by at least 20 X , if not more . Unfortunately very few HR teams make career or compensation differentiation between their best and average performers in Recruitment. The best firms realise the capacity of this single function to improve the organisation. Every Rupee spent on a proper Recruitment and Selection team is an investment.
• In Insurance sales- the difference between the good and the average is often as much as 1.5 to 10 times
• Another cost consideration , especially at middle and senior management is the opportunity cost of a “bad hire”- if you add recruitment cost, any fees paid to external vendors- it can add up-to 2-5 times the annual salary
• Facors to consider while doing a financial analysis
  • Number of people to be hired in this role.
  • The difference in Productivity in Dollar terms between average and good and between average and rejects.
  • There are certain roles where the downside is more important to consider- example a compliance role in an NBFC

• Lets say the economic value created by each financial consultant in the bank is 200,000 a year ( after subtracting salary and other overheads)
• The better ones do 20% more – i.e.- Rs.40,000 more a year
• If the assessment improves average performance by 10%- 20,000 and with 500 people its going to be- 20,000 * 500= 1 Cr/ 10 million
• Suppose assessment comes to 3000 per person and we need to run through a pool of 3000 ( 1:6 hiring ratio)- we have a cost of 90 Lacs/ 9 million – hence its almost worthwhile

Competency Supply & the False Positive & False Negative trade-off

• If you are hiring for a skill set where the pipeline / Supply of the required competency is very rare- you may want to skip the test- because the false negatives will result in you losing out good folks. The other option is to reduce the test Score cut off to TC2 and find another way to remove the false positives.
• If you are doing mass recruitment for a competency easily available the focus should be to reduce false positives-even if the false negatives increase. In this case you can increase the test cut off to TC1 – While this might increase the false negatives – you do not have to bear the burden of false positives – in the organisation or even spend money on them during the selection rounds

Reliability

Reliability Reflects the degree to which a persons score is stable for the same assessment over a period of time

Validity

Whether the tests measures what it is supposed to measure 

• Face Validity- does it look as if it measures what it is measuring – some like Thematic Appreception may not have high face validity
• Content Validity- is established by showing the clear theoretical links between the information collected by the assessment and the different job related behaviour- so if you have done job analysis- does the assessment cover the job analysis output of important criterion
• Criteria Validity/ Predictive Validity/ Emperical Validity- tests the statistical correlation between assessment scores and performance- and in a way the most important criterion
• Construct Validity- are the apt attributes being measured- you want to measure assertiveness and agreeableness and it ends of measuring resilience – your construct validity is low- but still might have a positive correlation with empirical validity

Assessment Experience

Candidate Experience is vital for attracting great talent – and assessment experience often is a function of how an assessment test is administered and its face validity

Omar Farooq- Founder & CEO AceProHR

Great teams are not made by chance- reach out to us to understand how you can win the Talent war.

References

• Netflix CEO on paying sky-high salaries: ‘The best are easily 10 times better than average’
• Selection Assessment methods by SHRM
• Goodby to MBTI, The fad that wont die- by Adam Grant – Psychology today
• Have we all been duped by the Myers Briggs test by Roman Krznaric- Fortune
• Assessment Methods in Recruitment , Selection & Performance by Robert Edenborough