Abstract: The Austin (Texas) Police Department began using the B-PAD video assessment process to screen cadet applicants. The applicant pool of 466 candidates was reduced by 33.69%, thereby saving significant money and selecting a group of candidates who were able to demonstrate relatively strong decision making and communication skills. No adverse impact was created by the selection process.
Background: Like many other police departments, the Austin Police Department spends a lot of time and money in selecting police cadets. As we add community policing duties to the traditional law enforcement functions, it becomes more important that officers be able to problem solve and mediate all types of disputes.
The Austin Police Department has traditionally used numerous "screens" to select the best qualified cadets: entrance exam (basic reading, math, memory and problem solving skills); agility test (basic coordination, endurance and muscle tone); medical (basic health); psychological (basic emotional and mental stability, impulse control, etc.); background investigation (history of past performance) and oral board (maturity, communication skills, initiative).
Each of these hurdles is "job related" and anchored to police department norms but the connection is often indirect. For example, applicants are rejected who have less than a 10th. grade reading ability because this education level is necessary to pass the academy. Applicants who have a history of job conflicts and involuntary termination are frequently rejected because they may continue to create uproar. While rational and job related, these measures do not provide direct, reliable information about a candidate's skills in resolving interpersonal conflict--the heart of police work.
B-PAD ("Behavioral Personnel Assessment Devices" video assessment (Rand, 1987; Young, 1992; Corey & MacAlpine, 1994; Corey, Hickey & Ford, 1997) offers a significant advance in the selection of police applicants in that it enables police departments to directly evaluate candidate's skills in problem solving and verbally managing realistic conflict situations likely to be encountered by police officers.
Benefits of Using Video Assessment
There are several benefits in using video assessment:
1. Saving Money: Among the 466 applicants, 157 (33.69%) failed B-PAD. As a result, we saved over $150,000 on background investigations alone, as each applicant requires an average of 65 investigation hours. The time saved on oral board, medical exams and psychological exams are additional savings.
2. Selecting Skillful Problem Solvers: Each video vignette presents a police problem to solve. The candidate must: 1) decide what the problem is; 2) implement a strategy of doing something and 3) communicate with the characters in the video to accomplish the solution.
The eight vignettes provide an in-depth look at candidate's problem solving and communication skills. These are one-on-one situations, group problems and a mixture of ages and ethnic groups. Consequently, the applicant's reactions to different types of people are also evident. For example, some candidates are deferential to other officers but abrupt with younger citizens.
3. Selection Based on Demonstrated Skills: The applicants must demonstrate their skills rather than talk about what they "would" do in a hypothetical situation. As a result, we get to see the actual skills of the applicant instead of his/her best guess at the correct strategy. Many people can talk a good strategy but cannot demonstrate that strategy in real time.
4. No Adverse Impact: The evaluation of the 466 applicants resulted in no adverse impact (i.e., significantly lower selection rates for minority groups compared to majority groups). While the numbers of some minority groups were too small for robust statistical analysis, no minority groups were adversely affected. White applicants (N=321) had a 65.42% pass rate; Black applicants (N=37) had a 64.86% pass rate; Hispanic applicants (N=96) had a 68.75% pass rate; American Indian applicants (N=5) had a 60.00% pass rate; , Asians (N=7) had a 85.71% pass rate; males had a 66.20% pass rate; and, females had a 67.50% pass rate. Analysis of variance revealed no statistical differences among pass rates of ethnic groups [F = .67, df(4), p = .62], and a comparison of means test revealed no differences between male and female pass rates [t = .41, df (45.4), p = .69].
5. Potential Savings in Training and Cleaning Up Citizen Complaints: We anticipate that our training in all areas of people-related tasks will be easier and more successful because cadets will already possess relatively strong decision making and interpersonal skills. Traditional law enforcement cadet training emphasized a "take charge and get tough" approach whenever young officers met challenges or resistance from citizens. This approach was rationalized as necessary for the officers' personal safety but probably was also based on the trainers' lack of poise and interpersonal skills.
The B-PAD combination of requiring applicants to take charge of situations and use tactful communications is a long needed demonstration that the most effective officers solve difficult situations using a combination of decision making and tact. Strong interpersonal skills are essential to defuse hostile situations, sort out chaotic calls and comfort frightened or injured citizens.
6. Identifying Weak Applicants Who Might Pass Traditional Process: There are several types of applicants who look good on paper, perform well in a psychological interview and before an oral board, but who don't make good police officers. We are hopeful that the B-PAD process will help us identify these applicants. Several such individuals showed up in the first screening:
a) Authoritarian: These applicants are harsh and overly controlling with citizens who question their authority but defer to supervisors and older police officers. These candidates would likely be very respectful and appropriate during the psychological interview and oral board, but begin creating complaints once they were on the street.
b) Overly nice: Several applicants failed the B-PAD because they did not establish control and a plan of action. They were very courteous and respectful but they could not control/direct the actors in the vignettes. These applicants typically get very good job references and passing scores in the oral board, but cannot take charge of complex street situations.
c) Know-it-alls: There were several applicants who used police jargon and had "good" reasons for their actions but also demonstrated that they could not accept criticism nor empathize with other's problems. They had to be right. These applicants typically fail the oral board, but it's helpful to have an early detection system.
d) Low integrity: Several applicants had well-developed interpersonal skills and passed the B-PAD cut-off scores, but did not risk challenging fellow officers who were dishonest. We will follow up on these candidates during the oral review board.
Miscellaneous Practical Issues in Using B-PAD
1. Selecting a Cut-Off Score: We used a conservative cut-off score (raw score of 22) which produced a 66.31% pass rate. The advantage of this approach is that we are confident that the applicants scoring below 22 are demonstrably less skilled. We did not want to fail people who were initially nervous or tongue-tied in front of the camera or who needed a couple of vignettes to warm up to the process.
Veteran police officers who were trained/certified as B-PAD raters consistently labeled applicants in the range below 22 as "not likely to do well as officers."
2. Reliability of Cut-Off Score: The B-PAD cut-off score was a reliable predictor of the pass rate. We used 22 as a minimum score which was expected, according to national user data published by The B-PAD Group, to produce a 65% pass rate. Our actual pass rate for the 466 applicants was 66.31% (309 passed of 466 applicants).
B-PAD Score Comparisons
Austin Police Department
Group N Mean SD
MALE 426 22.89 5.09
FEMALE 40 22.40 4.39 t = .671, df(49.4), p = .51, n.s.
WHITE 321 22.86 4.93
HISPANIC 96 23.03 5.16 t = .283, df(150.4), p = .78, n.s.
WHITE 321 22.86 4.93
BLACK 37 22.24 5.21 t = .688, df(43.7), p = .50, n.s.
WHITE 321 22.86 4.93
AM. INDIAN 5 20.60 8.36 t = .604, df(4.0), p = .58, n.s.
WHITE 321 22.86 4.93
ASIAN 7 24.71 5.19 t = -.935, df(6.2), p = .38, n.s.
ANOVA Results:
B-PAD Score by Race: F = .666, df(4), p = .62, n.s.
B-PAD Score by Test Cycle: F = .088, df(3), p = .97, n.s.
All t-test statistics were computed using a separate variances formula to avoid underestimating the true group differences.
B-PAD Selection Rate Comparisons
(pass =1; fail = 2)
Group N Mean Pass % N Pass SD
MALE 426 1.34 66.20 282 .47
FEMALE 40 1.33 67.50 27 .47 t = .166, df(46.6), p = .87, n.s.
WHITE 321 1.35 65.42 210 .48
HISPANIC 96 1.31 68.75 66 .47 t = -.611, df(159.1), p = .54, n.s.
WHITE 321 1.35 65.42 210 .48
BLACK 37 1.35 64.86 24 .48 t = -.066, df(44.4), p = .95, n.s.
WHITE 321 1.35 65.42 210 .48
AM. INDIAN 5 1.40 60.00 3 .55 t = -.220, df(4.1), p = .84, n.s.
WHITE 321 1.35 65.42 210 .48
ASIAN 7 1.14 85.71 6 .38 t = 1.397, df(6.4), p = .21, n.s.
Overall pass rate = 66.31% (N=309); overall failure rate = 33.69% (N=157).
All t-test statistics were computed using a separate variances formula to avoid underestimating the true group differences.