Using Video for Behavioral Assessment

 

Steven Somers CEO The B-PAD Group, Inc.

 

The purpose of this document is to examine the fairness and effectiveness of testing with video scenarios, which measure a candidate’s behavioral competency.

 

Why is behavioral competence important to measure?  There are 3 reasons:

 

  1. Police liability research shows that poor behavioral competence is the primary factor underlying liability claims and losses—including excessive use of force.
  2. The majority of police activity involves human interaction, whether it be giving directions, issuing a ticket, resolving a dispute, handling a domestic disturbance or arresting a violent offender.  
  3. Researchers have consistently demonstrated that behaviorally competent people are able to handle stressful situations better than those who have difficulty handling interpersonal problems.

 

FAIRNESS:

 

A substantial body of research exists relative to the fairness of selection procedures.  The results of this body of knowledge are presented in one place: the Federal Uniform Employee Selection Guidelines.

 

Section 4 paragraph D presents the Adverse Impact guideline known as the “four-fifths rule”.  A selection rate for any minority less than 80% of the highest rate will be regarded as evidence of adverse impact. 

 

In a paper submitted by Dr. Kathleen Lundquist at the May, 2007 EEOC meeting, she stated that cognitive ability (written) tests have a substantial adverse impact.  These are the tests that most often result in court challenges.

 

Video based, behavioral competency tests on the other hand are the lowest in adverse impact of all test types.  The B-PAD Group, Inc., a leading behavioral video test publisher has had numerous studies show a lack of adverse impact.  Dr. Lundquist’s paper and two major studies of video based testing and adverse impact all showing no adverse impact:

 

EEOC Dr. Lundquist paper 2007 links
Florida State University, Dept of Criminology, Adverse Impact study
B-PAD - Adverse Impact on Women and Minorities

 

EFFECTIVENESS:

 

A selection procedure’s ability to predict future success on the job is called the validity coefficient.   Very low validity coefficients in the mid .20’s have been frequently reported in the oral interview literature.  

 

Implementing B-PAD methods including video stimulus to provide a “frame of reference effect” (i.e., the more the applicant is able to place him or herself in a specific work situation, while responding, the more predicatively valid the test), and a standardized scoring system have more than tripled the predictive power achieved by the situational interview. 

 

Here is the conclusion of a concurrent validation study for B-PAD:

 

Conclusion.  Hunter & Hunter, in their meta-analysis of the validity and utility of alternative predictors of job performance, determined that the average validity of ability tests is .53 and concluded that “this very high level of validity is the standard against which all alternative predictors must be judged” (p. 81).  Dr. Rand’s observed validity coefficient of .72 compares well with this high standard and supports the use of B-PAD as a valid predictor of on-the-job performance.

 

The study by K. Pearlman, 1985, titled Validity Generalizations: From Theory to application, presented at the Center for Human Resource Programs, the Institute of Industrial Relations, UC Berkeley summarized the validity coefficients:


Average Validity of Predictors
Predictor Type Validity
   
Work Sample/Performance Test .54
Ability Test     .53
Assessment Center/Job Simulation           

.50

Behavioral Consistency Rating        .49
Job Knowledge Test .48
Job Tryout .44
Structured Interview            .25
Unstructured Interview .20
Reference Check

.14

Personality Test .10

 

 

B-PAD is a “Work Sample” type, the most predictive category above.  The study by Dr. Rand showing a .72 validity coefficient presents B-PAD as a leading test instrument in predictive power.

 

The following is a paragraph taken from a large city’s police department website stating how their officers are to treat citizens:

 

“Treatment of Citizens
Members of the Police Department will treat all members of the community professionally, with respect, dignity, compassion, fairness, empathy, and courtesy, and will not be condescending, disrespectful, rude, or unapproachable during contacts, regardless of how we are treated by them.


Members of the Police Department will maintain a positive, "can do" attitude while handling complaints, calls, concerns, inquires, etc., from citizens, and will not become defensive or appear unwilling to help.


Department members who do not have the ability to assist citizens on a particular problem will make every effort to guide them to the appropriate available resource.”

 

The above values are the competencies that B-PAD behavioral testing screens.  B-PAD measures behavioral competence in handling job-relevant interpersonal challenges.   The basic tenet of “behavioral consistency” applies.  That the best predictor of future performance, is performance observed under similar circumstances. 

 

Therefore, making candidates demonstrate that they have these skills is a necessary step to ensure that the applicants will uphold the high values of the organization.