Test Construction & Validation
The B-PAD Testing Procedure and Scoring
1. The Validity of B-PAD as a Measure of Problem-Solving Skills
2. The Validity of B-PAD as a Measure of Interpersonal Skills
3. The Validity of B-PAD as a Measure of Expected Job Performance
4. B-PAD and Adverse Impact On Women & Minorities
5. The Effect of Job Experience On B-PAD Scores
6. The Effect of Practice On B-PAD Scores
7. B-PAD's Correlation With Observed Competencies in the Police Academy
8. The Validity of B-PAD as a Measure of Supervision & Management Skills
9. The Validity of B-PAD as a Measure of Ability to Function As Part of a Team
10. The Effect of Fire/EMS Job Experience on B-PAD Scores
The Validity of B-PAD As a Measure of Ability to Function as Part of a Team
Purpose of the Study The purpose of this study was to assess the validity of the B-PAD behavioral response test as a measure of observed ability to function as part of a team. The study utilized a criterion validation strategy, contrasting the B-PAD Scale 3 (Overall Effectiveness) score with independent measures of job relevant interpersonal teamwork skills. Also assessed was the relationship between the B-PAD Scale 3 score and a validated written test of cognitive abilities (see ÒThe Validation of ESV Entrance Fire Service Written TestsÓ, McCann Associates, Inc., Langhorne, Pennsylvania, 1995). It was hypothesized that there would be no relationship between these scores because they presume to measure independent constructs.
Methodology and Subjects This study involved the administration of B-PAD for Fire/EMS to 29 incumbent firefighter/paramedics from a large metropolitan fire department in the Virginia. Subject B-PAD responses were scored by two independent rater panels from a midsize municipal fire department. Each panel consisted of three raters, comprised of captains and division chiefs. The raters had completed formal training in the B-PAD scoring procedures and had not previously scored outside of the training forum.
Criterion raters consisted of Medical Zone Supervisors and Fire Company Officers from the employing agency. The raters knowledge of the subjectsÕ work performance was based on personal observation. The criterion measure was the sum score of two independent Likert ratings of a subject's observed ability to function as part of a team. This was defined for raters as "subordinating oneself to team goals, getting along with other department members in non-emergency situations," and the extent to which the subject acts in an "unselfish and non-disruptive fashion." The study followed a double-blind methodology: the B-PAD raters and criterion raters had no knowledge of each others scores.
Results The two independent B-PAD panels produced scores that were significantly correlated (r = .77, p < .001). Their combined scores correlated significantly with the criterion measure (r = .44, p < .02). As expected, the B-PAD scores did not correlate significantly with the McCann written test of cognitive abilities (r = .19, p >.10).
Conclusion The results support the use of B-PAD as a predictor of on-the-job interpersonal behavior contributing to effective teamwork. The data also support the conclusion that interpersonal behavior, as measured by the B-PAD behavioral response test, and cognitive abilities, as measured by the McCann written test instrument, are independent constructs.