By Stephen Moulton
What Research Tells Us
Over the last 50 years, researchers have conducted numerous studies on the employment selection process. The studies discussed here are some of the most current. We hope this information will help you make better hiring decisions. Successful Interviewing is a Process, Not an Event.
1. Conduct a Job Analysis:
Wiesner, W. H., & Cronshaw, S. F. (1988) A meta-analytic investigation of the impact of interview format and degree of structure on the validity of the employment interview. Journal of Applied Psychology, 61,275-290.
The results showed that structured interviews were more than twice as effective as unstructured interviews. “On the basis of our findings we concluded that the best prediction achievable for structured interviews would be obtained where structured interview questions are based on a formal job analysis rather than a less systematic assessment of job requirements.”
The Wiesner and Cronshaw study reflected some interesting statistics.
1. The typical unstructured interview, conducted by a highly experienced interviewer, had a reliability of predicting job performance of about 15-30%.
2. A structured past-event interview, based on a job analysis, using rating guides, could achieve up to an 87% reliability in predicting job performance.
2. Create a Structured Interview:
Huffcutt, A. I., Roth, P. L., & McDaniel, M. A. (1996) A meta-analytic investigation of cognitive ability in employment interview evaluations: Moderating characteristics and implications for incremental validity. Journal of Applied Psychology, 81,459-473.
This review of 49 studies considered the impact of such factors as structure, type of interviews, and job complexity in assessing the extent to which employment interview evaluations reflect cognitive ability. Why do structured interviews have higher levels of validity? One theory is that they are better at assessing other factors related to success on the job.
3. Use Past Event Interview Questions:
Pulakos, E. D. & Schmidtt, N. (1995) Experience-based and situational interview questions: Studies of validity. Personnel Psychology, 48, 289-308.
In this study, two groups of 108 subjects were interviewed. One group was interviewed using experience-based questions (past event) about how they handled specific situations in the past, requiring skills and abilities necessary for effective job performance. The second group was interviewed using situational (highly structured future event or hypothetical) job related situations and asked how they would respond if they were confronted with these situations. The experienced-based interview questions resulted in higher levels of validity than the situational questions.
Campion, M. A., Campion, J. E., & Hudson, J. P. Jr. (1994) Structured Interviewing: A note on incremental validity and alternative question types. Journal of Applied Psychology, 79, 998-1002.
This study was conducted to see if a structured interview can have incremental validity in the prediction of job performance beyond a battery of cognitive ability tests. In addition, the study looked at whether the future oriented situational interview or past oriented behavioral interview would have higher validity. The validity of past interview questions (.51) was higher than the validity of future interview questions (.39). In addition, the past-oriented interview questions demonstrated incremental validity beyond future questions and the tests.
4. Use Interview Panels:
McDaniel, M. A., Whetzel, D.L., Schmidt, F.L., & Maurer, S. D. (1994) The validity of employment interviews: A comprehensive review and meta-analysis. Journal of Applied Psychology, 79, 599-616.
This investigation focused on what prior research had found on interview validity. The results were that the validity of structured interviews were higher than unstructured interviews and panel interviews were more valid than individual interviews. The study also found that interviewers who had access to test scores prior to interviewing candidates appeared to decrease validity.
So what is the big deal? The traditional interviews that most managers conduct will not keep your company ahead of the power curve. If you what to improve your ability to hire the right people for the right positions, structured panel interviews are the most effective. Panels need only be two or three members.
5. Control Biases and First Impressions:
Marlowe, C. M., Schneider, S. L., & Nelson, C. E. (1996) Gender and attractiveness biases in hiring decisions: Are more experienced managers less biased? Journal of Applied Psychology, 81, 11-21.
This study demonstrated that highly attractive male candidates were consistently rated as more suitable for hire than either marginally attractive male or female applicants. In addition, marginally attractive females were at the greatest disadvantage. Finally, men were perceived to be more suitable for hire or advancement than equally qualified women.
Bias and error by interviewers is a key reason for interviewing failure. Without some sort of scientific tools and training, interviewers will often make hiring decisions based on “gut feelings” and intuition. “Gut feelings” and intuition should be used to make probing more effective.
Dougherty, T. W., Turban, D.B., & Callender, J. C. (1994) Confirming First Impressions in the Employment Interview: A Field Study of Interviewer Behavior. Journal of Applied Psychology, 79, 659-665.
This study examined the behavioral styles of interviewers to confirm their first impressions of job applicants. In this study interviewers followed up positive first impressions by showing a high regard toward the candidate “selling” the company and giving job information. The interviewer also gathered less information on the candidate.
The candidate’s job is to make a good first impression. The first impression façade is often as unreal as the façade of a movie set, looks good, but little behind it. Interviewers that allow themselves to be sold by first impressions do little to check the qualifications of the candidate behind the façade.
Additional findings included:
1. Inexperienced candidates might be restricted in a range of potential responses to past questions. The relative flexibility of an interview probe based on past questions might improve relative effectiveness.
2. Future questions may be impacted by “fake-ability.”
“Fake-ability” relates to the candidate’s efforts to deliver responses that are not true in order to obtain a test or interview result that is favorable. Having the ability to minimize or eliminate “fake-ability” is perceived as important when screening for new employees.
6. Use a Rating Guide:
Cesare, S. J. (1996) Subjective Judgement and the Selection Interview: a methodological review. Public Personal Management, 25, 291-306.
Beyond such research considerations is the practical development of a sound interview that is psychometrically adequate, legally defensible, and cost effective. First, an effective interview must be premised on a through job analysis to outline the content, score, and determine the primary work domains of the position being examined. By extension, it is vital to align the rating dimensions contained within the job analysis to those content-based questions asked in the interview and those which correspond to segments of the performance criteria.
Additional findings included:
“ An essential component of any interview development strategy is the inclusion of training.”
The difference between an interview built on a job analysis, that links the interview questions and the rating guides, and one that doesn’t, is like the difference between day and night. Improving objectivity from subjectivity is a significant step toward the science end of the selection spectrum. Training is a key to user success.
7. Use Consensus Ratings:
Pulakos, E. D., Schmitt, N., Whitney, D., & Smith, M. (1996) Individual differences in interviewer ratings: The impact of standardization, consensus discussion, and sampling error of a structured interview. Personnel Psychology, 49, 85-103.
This research study investigated the validity of averaged versus standardized versus consensus ratings. This study sampled 515 professionals in a large Federal agency. The sample included a large cross-section of minorities. Past-oriented interviews were used to measure the quality of the rating process. “The underlying notion is that the best predictor of future performance is past performance in similar situations.” The result was that the validity of consensus ratings was significantly higher than the other approaches. The belief was that the interviewers were accountable to their peers and hence were more accurate in their ratings.
The consensus rating plays a key role in getting the right candidate. When interviewers and raters know they will have to meet and be accountable on the reasons for the fit of the candidate, they become more objective and thorough in their interviewing efforts.
Additional findings included:
1. Structured interviews, if preceded by a thorough job analysis, can be developed to tap the many skill and ability areas required for the job.
2. Experience-based interviews had little impact on, and were equally valid for, subgroups (White, Black, Hispanic, male, and female).
Past event questions were shown to be more effective than future event questions. Validated situational interviews often take weeks and cost $10,000 or more per position to create. Validated past event structured interviews can be created for 99.99% less.
Note that this study also supports the need for a thorough job analysis that identifies the skills or competencies necessary for success.
Bottom-line
Plowing through research studies like these can be dull and dry, as these summaries may be. Yet, they help us to understand what managers and interviewers need to do in order to hire employees and team members that will succeed and add to the success and bottom line of your company’s profitability.