Strategies for dealing with the diversity-validity dilemma in personnel selection: Where are we and where should we go?

• Autores: Britt De Soete, Celina Druart, Filip Lievens
• Localización: Revista de psicología del trabajo y de las organizaciones = Journal of work and organizational psychology, ISSN 1576-5962, Vol. 29, Nº 1, 2013, págs. 3-12
• Idioma: inglés
• DOI: 10.5093/tr2013a2
• Títulos paralelos:
  • Estrategias para manejar el dilema validez-diversidad en selección de personal: ¿Dónde estamos y hacia dónde deberíamos ir?
• Enlaces
  • Texto completo (pdf)
• Resumen
  • español

    El dilema validez-diversidad ha sido un tema dominante en la investigación y la práctica de la selección de personal. Dado que algunos de los instrumentos de selección más válidos presentan grandes diferencias étnicas en sus puntuaciones, los científicos intentan desarrollar estrategias que reduzcan las diferencias de los subgrupos en la selección mientras mantienen simultáneamente la validez de criterio. Este artículo proporciona una revisión basada en la evidencia de la efectividad de seis estrategias utilizadas para manejar el dilema validez-diversidad: (1) usar medidas de capacidad cognitiva “alternativas”, (2) emplear simulaciones, (3) utilizar procedimientos estadísticos para combinar las medidas predictoras y del criterio, (4) reducir la varianza de los predictores irrelevante para el criterio, (5) fomentar reacciones positivas en los candidatos y (6) facilitarles preparación y posibilidades de practicar. Tres estrategias (emplear evaluaciones basadas en simulaciones, desarrollar medidas alternativas de capacidad cognitiva y usar procedimientos estadísticos) son las más prometedoras para aminorar el problema. Se discuten áreas potenciales que requieren investigación en un futuro.

  • English

    The diversity-validity dilemma has been a dominant theme in personnel selection research and practice. As some of the most valid selection instruments display large ethnic performance differences, scientists attempt to develop strategies that reduce ethnic subgroup differences in selection performance, while simultaneously maintaining criterion-related validity. This paper provides an evidence-based overview of the effectiveness of six strategies for dealing with the diversity-validity dilemma: (1) using "alternative" cognitive ability measures, (2) employing simulations, (3) using statistical approaches to combine predictor and criterion measures, (4) reducing criterion-irrelevant predictor variance, (5) fostering positive candidate reactions, and (6) providing coaching and opportunity for practice to candidates. Three of these strategies (i.e., employing simulation-based assessments, developing alternative cognitive ability measures, and using statistical procedures) are identified as holding the most promise to alleviate the dilemma. Potential areas in need for future research are discussed.

• Referencias bibliográficas
  • Ackerman, P. L., & Beier, M. E. (2012). The problem is in the definition: g and intelligence in I-O psychology. Industrial and Organizational...
  • Ar thur, W., Day, E. A., McNelly, T. L., & Edens, P. S. (2003). A meta-analysis of the criterion-related validity of assessment center...
  • Ar thur, W., Edwards, B. D., & Barrett, G. V. (2002). Multiple-choice and constructed response tests of ability: Race-based subgroup performance...
  • Ar thur, W., & Villado, A. J. (2008). The importance of distinguishing between constructs and methods when comparing predictors in personnel...
  • Ar vey, R. D., Strickland, W., Drauden, G., & Martin, C. (1990). Motivational components of test taking. Personnel Psychology, 43, 695-716....
  • Ba rrett, G. V., Miguel, R. F., & Doverspike, D. (1997). Race differences on a reading comprehension test with and without passages. Journal...
  • Beatty, P. C., & Willis G. B. (2007). Research synthesis: The practice of cognitive interviewing. Public Opinion Quarterly, 71, 287-311....
  • Be cton, J. B., Feild, H. S., Giles, W. F., & Jones-Farmer, A. (2008). Racial differences in promotion candidate performance and reactions...
  • Berk, R. A. (1982). (Ed.). Handbook of methods for detecting test bias. Baltimore: John Hopkins University Press.
  • Bobko, P., & Roth, P. L. (2013). Reviewing, categorizing, and analyzing the literature on Black-White mean differences for predictors...
  • Bo bko, P., Roth, P. L., & Buster, M. A. (2005). Work sample selection tests and expected reduction in adverse impact: A cautionary note....
  • Bobko, P., Roth, P. L., & Nicewander, A. (2005). Banding selection scores in human resourcemanagement decisions: Current inaccuracies...
  • Brouwers, S. A., & Van De Vij ver, F. J. R. (2012). Intelligence 2.0 in I-O psychology: Revival or Contextualization? Industrial and Organizational...
  • Callinan, M., & Robertson, I. T. (2000). Work sample testing. International Journal of Selection and Assessment, 8, 248-260. doi: 10.1111/1468-2389.00154...
  • Campion, M. A., Outtz, J. L., Zedeck, S., Schmidt, F. L., Kehoe, J. F., Murphy, K. R., & Guion, R. M. (2001). The controversy over score...
  • Cascio, W. F., Jacobs, R., & Silva, J. (2010). Validity, utility, and adverse impact: Practical implications from 30 years of data. In...
  • Cascio, W. F., Outtz, J., Zedeck, S., & Goldstein, I. L. (1991). Statistical implications of six methods of test score use in personnel...
  • Cattell, R. B. (1971). Abilities: Their structure, growth, and action. Boston: Houghton Mifflin.
  • Chan, D., & Schmitt, N. (1997). Video-based versus paper-and-pencil method of assessment in situational judgment tests: Subgroup differences...
  • Chan, D., Schmitt, N., DeShon, R. P., Clause, C. S., & Delbridge, K. (1997). Reactions to cognitive ability tests: The relationships between...
  • Cullen, M. J., Hardison, C. M., & Sackett, P. R. (2004). Using SAT-grade and ability-job performance relationships to test predictions...
  • De Corte, W. (1999). Weighing job performance predictors to both maximize the quality of the selected workforce and control the level of adverse...
  • De Corte, W., Lievens, F., & Sackett, P. (2006). Predicting adverse impact and multistage mean criterion performance in selection. Journal...
  • De Corte, W., Lievens, F., & Sackett, P. (2007). Combining predictors to achieve optimal trade-offs between selection quality and adverse...
  • De Corte, W., Lievens, F., & Sackett, P. (2008). Validity and adverse impact potential of predictor composite formation. International...
  • De Corte, W., Sackett, P., & Lievens, F. (2011). Designing pareto-optimal selection systems: Formalizing the decisions required for selection...
  • De Meijer, L. A. L., Born, M. P., Terlouw, G., & van der Molen, H. T. (2006). Applicant and method factors related to ethnic score differences...
  • De Meijer, L. A. L., Born, M. P., Terlouw, G., & van der Molen, H. T. (2008). Criterionrelated validity of Dutch Police-Selection Measures...
  • De Soete, B., Lievens, F., Oostrom, J., & Westerveld, L. (in press). Alternative predictors for dealing with the diversity-validity dilemma...
  • Dean, M. A., Bobko, P., & Roth, P. L. (2008). Ethnic and gender subgroup differences in assessment center ratings: A meta-analysis. Journal...
  • DeShon, R. P., Smith, M. R., Ch an, D., & Schmitt, N. (1998). Can racial differences in cognitive test performance be reduced by presenting...
  • Druart, C., & De Corte, W. (2012). Designing Pareto-optimal systems for complex selection decisions. Organizational Research Methods,...
  • Edwards, B. D., & Arthur, W. (2 007). An examination of factors contributing to a reduction in subgroup differences on a constructed-response...
  • Evers, A., Te Nijenhuis, J., & Van der Flier, H. (2005). Ethnic bias and fairness in personnel selection: Evidence and consequences. In...
  • Fetzer, M. S. (2012, April). Current research in advanced assessment technologies. Symposium presented at the 27th Annual Conference of the...
  • Foldes, H. J., Duehr, E. E., & Ones, D. S. (2008). Group differences in personality: Metaanalyses comparing five US racial groups. Personnel...
  • Freedle, R., & Kostin, I. (1990). Item difficulty of 4 verbal item types and an index of differential item functioning for Black and White...
  • Gatewood, R., Feild, H., & Barr ick, M. (2011). Human Resource Selection (6th ed.). Cincinnati, OH: South-Western.
  • Gillespie, J. Z., Converse, P. D., & Kriska, S. D. (2010). Applying recommendations from the literature on stereotype threat: Two field...
  • Goldstein, H. W., Yusko, K. P., Braverman, E. P., Smith, D. B., & Chung, B. (1998). The role of cognitive ability in the subgroup differences...
  • Goldstein, H. W., Yusko, K. P., & Nicolopoulos, V. (2001). Exploring Black-White subgroup differences of managerial competencies. Personnel...
  • Grand, J. A., Ryan, A. M., Schm itt, N., & Hmurovic, J. (2011). How far does stereotype threat reach? The potential detriment of face...
  • Gudykunst, W. B., Matsumoto, Y., TingToomey, S., Nishida, T., Kim, K., & Heyman, S. (1996). The influence of cultural individualism-collectivism,...
  • Hall, E. T. (1976). Beyond culture. New York, NY: Doubleday.
  • Hanges, P. J., & Feinberg, E. G. (2009). International perspectives on adverse impact: Europe and beyond. In J. L. Outtz (Ed.), Adverse...
  • Hattrup, K., Rock, J., & Scalia, C. (1997). The effects of varying conceptualizations of job performance on adverse impact, minority hiring,...
  • Hattrup, K., Schmitt, N., & Lan dis, R. S. (1992). Equivalence of constructs measured by job-specific and commercially available aptitude-tests....
  • Hausknecht, J. P., Halpert, J. A., Di Paolo, N. T., & Gerrard, M. O. M. (2007). Retesting in selection: A meta-analysis of coaching and...
  • Helms, J. E. (1992). Why is the re no study of cultural equivalence in standardized cognitive ability testing? American Psychologist, 47,...
  • Helms, J. E. (2012). A legacy o f eugenics underlies racial-group comparisons in intelligence testing. Industrial and Organizational Psychology-Perspectives...
  • Hough, L. M., Oswald, F. L., & Ployhart, R. E. (2001). Determinants, detection and amelioration of adverse impact in personnel selection...
  • Huffcutt, A. I., Goebl, A. P., & Culbertson, S. S. (2012). The engine is important, but the driver is essential: The case for executive...
  • Imus, A., Schmitt, N., Kim, B., Oswald, F. L., Merritt, S., & Wrestring, A. F. (2011). Differential item functioning in biodata: Opportunity...
  • Jensen, A. R. (1998). The g factor: The science of mental ability. Westport, CT: Praeger.
  • Leong, F. T. L., Leung, K., & Cheun g, F. M. (2010). Integrating cross-cultural psychology research methods into ethnic minority psychology....
  • Lievens, F., Buyse, T., & Sackett, P. R. (2005). The operational validity of a video-based situational judgment test for medical college...
  • Lievens, F., & De Soete, B. (2012). Simulations. In N. Schmitt (Ed.), Handbook of Assessment and Selection: Oxford University Press.
  • Lievens, F., & Reeve, C. L. (2012). Where I-O psychology should really (re)start its investigation of intelligence constructs and their...
  • Lievens, F., Westerveld, L., & De C orte, W. (in press). Understanding the building blocks of selection procedures: Effects of response...
  • McDaniel, M. A., Psotka, J., Legree, P. J., Yost, A. P., & Weekley, J. A. (2011). Toward an understanding of situational judgment item...
  • Medley, D. M., & Quirk, T. J. (1974). Application of a factorial design to study cultural bias in general culture items on national teacher...
  • Mitchelson, J. K., Wicher, E. W., L eBreton, J. M., & Craig, S. B. (2009). Gender and ethnicity differences on the Abridged Big Five Circumplex...
  • Motowidlo, S. J., Dunnette, M. D., & Carter, G. W. (1990). An alternative selection procedure: The low-fidelity simulation. Journal of...
  • Murphy, K. R. (2010). How a broader definition of the criterion domain changes our thinking about adverse impact. In J. L. Outtz (Ed.), Adverse...
  • Murphy, K. R., & Shiarella, A. H. (1997). Implications of the multidimensional nature of job performance for the validity of selection...
  • Nápolez-Springer, A. M., Santoyo-Olsson, J., O'Brien, H., & Stewart, A. L. (2006). Using cognitive interviews to develop surveys in...
  • Nguyen, H. H. D., & Ryan, A. M. (2008). Does stereotype threat affect test performance of minorities and women? A meta-Analysis of experimental...
  • Ones, D. S., & Anderson, N. (2002). Gender and ethnic group differences on personality scales in selection: Some British data. Journal...
  • Oostrom, J. K., & Born, M. Ph. (submitted). Using cognitive pretesting to explore causes for ethnic differences on role-plays.
  • Oostrom, J. K., Born, M. Ph., Serlie, A. W., & van der Molen, H. T. (2010). Webcam testing: Validation of an innovative open-ended multimedia...
  • Oostrom, J. K., Born, M. Ph., Serlie, A. W., & van der Molen, H. T. (2011). A multimedia situational test with a constructed-response...
  • Outtz, J., Goldstein, H., & Ferreter, J. (2006, April). Testing divergent and convergent thinking: Test response format and adverse impact....
  • Paullin, C., Putka, D. J., Tsacoumis, S., & Colberg, M. (2010, April). Using a logic-based measurement approach to measure cognitive ability....
  • Ployhart, R. E., & Holtz, B. C. (2008). The diversity-validity dilemma: Strategies for reducing racioethnic and sex subgroup differences...
  • Pyburn, K. M., Ployhart, R. E., & Kravitz, D. A. (2008). The diversity-validity dilemma: Overview and legal context. Personnel Psychology,...
  • Reckase, M. D. (1996). Test construction in the 1990s: Recent approaches every psychologist should know. Psychological Assessment, 8, 354-359....
  • Roth, P. L., Bevier, C. A., Bobko, P., Switzer, F. S., & Tyler, P. (2001). Ethnic group differences in cognitive ability in employment...
  • Roth, P. L., Bobko, P., McFarland, L., & Buster, M. (2008). Work sample tests in personnel selection: A meta-analysis of black-white differences...
  • Roth, P. L., Buster, M. A., & Bobko, P. (2011). Updating the trainability tests literature on Black-White subgroup differences and reconsidering...
  • Roth, P. L., Huffcutt, A. I., & Bobko, P. (2003). Ethnic group differences in measures of job performance: A new meta-analysis. Journal...
  • Roth, P. L., Switzer, F. S., Van Iddekinge, C. H., & Oh, I. S. (2011). Toward better metaanalytic matrices: How input values can affect...
  • Roussos, L., & Stout, W. (1996). A multid imensionality-based DIF analysis paradigm. Applied Psychological Measurement, 20, 355-371. doi:...
  • Ruggs, E. N., Law, C., Cox, C. B., Roehling, M. V., Wieners, R. L., Hebl, M. R., & Barron, L. (2013). Gone fishing: I/O psychologists'...
  • Ryan, A. M. (2001). Explaining the Black-White test score gap: The role of test perceptions. Human Performance, 14, 45-75. doi: 10.1207/ S15327043HUP1401-04...
  • Ryan, A. M., & Greguras, G. J. (1998). Li fe is not multiple choice: Reactions to the alternatives. In M. D. Hakel (Ed.), Beyond Multiple...
  • Ryan, A. M., & Huth, M. (2008). Not much more than platitudes? A critical look at the utility of applicant reactions research. Human Resource...
  • Sacco, J. M., Scheu, C. R., Ryan, A. M., Schmitt, N., Schmidt, D. B., & Rogg, I. U. (2000, April). Reading level as a predictor of subgroup...
  • Sackett, P. R. (2003). Stereotype threat in applied selection settings: A commentary. Human Performance, 16, 295-309. doi: 10.1207/s15327043hup1603-6...
  • Sackett, P. R., Burris, L. R., & Ryan, A. M. (1989). Coaching and practice effects in personnel selection. In C. L. Cooper and I. T. Robertson...
  • Sackett, P. R., & Ellingson, J. E. (1997). The effects of forming multipredictor composites on group differences and adverse impact. Personnel...
  • Sackett, P. R., Hardison, C. M., & Cullen, M. J. (2004). On interpreting stereotype threat as accounting for African American-White differences...
  • Sackett, P. R., & Roth, L. (1996). Multi-stage selection strategies: A Monte Carlo investigation of effects on performance and minority...
  • Sackett, P. R., Schmitt, N., Ellingson, J. E., & Kabin, M. B. (2001). High-stakes testing in employment, credentialing, and higher education:...
  • Scherbaum, C. A., & Goldstein, H. W. (200 8). Examining the relationship between racebased differential item functioning and item difficulty....
  • Schleicher, D. J., Van Iddekinge, C. H., Morgeson, F. P., & Campion, M. A. (2010). If at first you don't succeed, try, try again:...
  • Schmit, M. J., & Ryan, A. M. (1997). Applicant withdrawal: The role of test-taking attitudes and racial differences. Personnel Psychology,...
  • Schmitt, N., & Mills, A. E. (2001). Traditional tests and job simulations: Minority and majority performance and test validities. Journal...
  • Steele, C. M. (1997). A threat in the air : How stereotypes shape intellectual identity and performance. American Psychologist, 52, 613-629....
  • Steele, C. M. (1998). Stereotyping and it s threat are real. American Psychologist, 53, 680-681. doi: 10.1037/0003-066x.53.6.680
  • Steele, C. M., & Aronson, J. A. (1995). Stereotype threat and the intellectual test performance of African-Americans. Journal of Personality...
  • Steele, C. M., & Aronson, J. A. (2004). Stereotype threat does not live by Steele and Aronson (1995) alone. American Psychologist, 59,...
  • Tam, A. P., Murphy, K. R., & Lyall, J. T. (2004). Can changes in differential dropout rates reduce adverse impact? A computer simulation...
  • Van Iddekinge, C. H., Morgeson, F. P., Schleicher, D. J., & Campion, M. A. (2011). Can I retake it? Exploring subgroup differences and...
  • Weekley, J. A., & Jones, C. (1997). Video-based situational testing. Personnel Psychology, 50, 25-49. doi: 10.1111/j.1744-6570.1997.tb00899.x...
  • Whetzel, D. L., McDaniel, M. A., & Nguyen, N. T. (2008). Subgroup differences in situational judgment test performance: A meta-analysis....
  • Whitney, D. J., & Schmitt, N. (1997). Relationship between culture and responses to biodata employment items. Journal of Applied Psychology,...
  • Willis, G. B., & Miller, K. (2011). Cross-cultural cognitive interviewing: Seeking comparability and enhancing understanding. Field Methods,...
  • Yusko, K. P., Goldstein, H. W., Scherbaum, C. A., & Hanges, P. J. (2012, April). Siena Reasoning Test: Measuring intelligence with reduced...