B-PAD: A NEW TOOL FOR FIRE PERSONNEL SELECTION
By Paul Stein, from "The California Fire Service," Vol. 7, No. 1, January, 1996.
For this month's article, I am going to stray away from the promotional exam topics and discuss B-PAD. B-PAD is a new concept to increase effectiveness and efficiency when conducting entry level examinations.
Growing applicant pools, escalating costs and questions about the effectiveness of conventional methods are forcing departments to rethink the selection process. There have been many refinements to the traditional tools over the last 15 years but few innovations. A growing number of departments, however, believe they have found such an innovation.
Known as B-PAD (Behavioral Personnel Assessment Device), this new methods uses video to predict how recruits will respond to the challenges of the job. More specifically, it's a tool to measure how well recruits can be expected to work with coworkers and the public. Its proponents herald it as the "missing link" in the selection process -- capable of lowering selection costs and reducing adverse impact.
What B-PAD is and how it works
B-PAD is a series of video-based behavioral tests designed to measure interpersonal skills and common sense judgment. Alternate forms of B-PAD are now in use in over 100* agencies in the United States and Canada. B-PAD has video tests for both fire and police departments. Promotional assessments were introduced in 1995.
Unlike interviews, in which applicants state what they would do in response to situational questions, B-PAD requires candidates to demonstrate their skills in carrying out these responses. B-PAD utilizes professionally acted, job-relevant video simulations. Candidates are videotaped in action as they respond to a series of "real-world" scenarios. Trained, in-house personnel view the responses and numerically rate each applicant, utilizing B-PAD's validated scoring criteria.
The Norfolk Department of Fire and Paramedical Services pioneered this approach to testing with entry-level recruits in 1994. Said Director Ronald Wakeham, "Fire and EMS is really a people business. B-PAD provides the applicant with an idea of what they are getting into. It provides the department the opportunity to view and evaluate a person's potential for fitting into the world of Emergency Services."
Dr. Nancy Olivo of the city's Human Resources Department said the economic benefits of the test and its defensibility also influenced their decision to use B-PAD. Immediate economic benefits resulted from savings in staff hours dedicated to the oral assessment phase. Instead of conducting oral boards for the 250 qualifying candidates, B-PAD was used to reduce the pool to a manageable number for final review. After receiving a day of rater training, 14 raters (a combination of fire personnel and community members), spent the next two days scoring B-PAD responses from the 250 candidates. The raters were not required to be present for the tests' administration, done the week before.
Data collected over eight years with alternate forms reveals no adverse impact on protected classes, asserts David MacAlpine, founder of The B-PAD Group. "This is a significant benefit to departments striving to increase diversity without sacrificing selection standards. What is fascinating is that good people skills and judgment seem to transcend rate, gender and culture, as does our ability to identify these skills in others."
Interpersonal Competence vs. Cognitive Ability
A new book by Daniel Goleman, Ph.D., Emotional Intelligence: Why it Can Matter More Than IQ, provides a rationale for why B-PAD works. People skills are a key component of emotional intelligence as defined by Goleman and other researchers. In his book, Goleman advances the case that these skills are more important than IQ and academic achievement to success on the job and life in general: ". . .in the day-to-day world no intelligence is more important than interpersonal skills."
Goleman points out that interpersonal intelligence and IQ represent different abilities, "Indeed, there is a slight correlation between IQ and some aspects of emotional intelligence -- though small enough to make clear these are largely independent entities." Goleman goes on to suggest that the best way to assess these skills may involve using video: "Unlike the familiar tests for IQ, there is, as yet, no single paper-and-pencil test that yields an 'emotional intelligence score' and there may never be one. Although there is ample research on each of its components, some of the them, such as empathy, are best tested by sampling a person's actual ability at the task -- for example, by having them read a person's feelings from a video of their facial expressions."
Many of these skills, according to Goleman, have their roots in infancy, and have more to do with how we communicate (tone of voice, gesture, etc.) than what we communicate. This may partially explain, MacAlpine states, why candidates taking B-PAD more than once do not, on average, improve. "We've tracked repeat administrations, and so far we see no evidence of practice effects. We suspect this is lagely because they either have they skill or they don't." Applicants just can't fake all the nonverbals.
Background
B-PAD tests are produced by The B-PAD Group, a company specializing in public safety selection and assessment. The original test resulted from the University of San Francisco dissertation work of Dr. R. Rand, a cofounder of The B-PAD Group. The construction of B-PAD for Fire/EMS began with the collection of numerous problematic vignettes obtained from experienced firefighters and paramedics. These were rated for job relevance and importance by representatives from 26 agencies across the country. A core group of experts wrote scripts for the scenarios that were eventually selected for video production.
The procedures used for validity and reliability testing of B-PAD were conducted in accordance with the Standards for Education and Psychological Testing published jointly by the American Education Research Association, the American Psychological Association, and the National Council on Measurement in Education.
Criterion validity research to date has documented impressive correlations between B-PAD results and on-the-job performance.
MacAlpine says, "Our tests are firmly grounded in the theory of behavioral consistency (i.e., the best predictor is past behavior in similar situations) and are a response to the long-standing plea to utilize procedures grounded on this principle. Our research as does the empirical literature, shows that behavioral testing works."
B-PAD is a new alternative for fire personnel selection -- an alternative its users believe adds significant value to the process. Potential for cost reduction must be evaluated within the context of your current procedures and staff assignments. The final return on investment, however, may best be measured by what happens on the job.
* Editor's Note: By October, 1999, B-PAD was in use by over 400 agencies in the United States, Canada, and Australia.