Evaluating data quality and understanding statistical measures

Reward professionals working at CIPD Level 5 need to do more than gather pay data — they need to judge whether that data is worth acting on. This video addresses AC 3.2 of 5HR03 by focusing on the critical evaluation of benchmarking data quality and reliability. It covers the key diagnostic questions that should be applied to any data source, including considerations around who produced the data and for what purpose, the transparency of methodology, sample size, representativeness, currency, and consistency of approach over time. These criteria apply across the range of sources relevant to reward benchmarking, from commercial salary surveys and payroll data to government statistics on earnings, vacancies, pay settlements and labour market conditions.

A substantial portion of the video is devoted to the statistical measures most commonly encountered when working with earnings and reward data. The mean and the median are both examined in the context of pay distribution, with particular attention to how the presence of high earners affects each measure differently. The video also covers percentiles and quartiles — including the lower and upper quartile positions — and explains how these distribution markers inform strategic pay positioning decisions for different role types, including specialist or hard-to-fill positions. Sample size at the role level, as distinct from overall survey participation figures, is highlighted as a critical factor when assessing the reliability of specific data points within a broader survey.

The video concludes with triangulation as a best-practice methodology for reward benchmarking. It addresses the practical steps involved in cross-referencing commercial survey data with government statistics, recruitment advertising intelligence, internal payroll analysis, and qualitative insight from recruiters and hiring managers. The video distinguishes between scenarios where multiple independent sources converge — increasing confidence in a finding — and those where they diverge, which signals a need for further investigation into methodological differences, timing gaps, or genuine market uncertainty.