Test Construction & Validation
The B-PAD Testing Procedure and Scoring
1. The Validity of B-PAD as a Measure of Problem-Solving Skills
2. The Validity of B-PAD as a Measure of Interpersonal Skills
3. The Validity of B-PAD as a Measure of Expected Job Performance
4. B-PAD and Adverse Impact On Women & Minorities
5. The Effect of Job Experience On B-PAD Scores
6. The Effect of Practice On B-PAD Scores
7. B-PAD's Correlation With Observed Competencies in the Police Academy
8. The Validity of B-PAD as a Measure of Supervision & Management Skills
9. The Validity of B-PAD as a Measure of Ability to Function As Part of a Team
10. The Effect of Fire/EMS Job Experience on B-PAD Scores
The Validity of B-PAD As a Measure of Supervision and Management Skills
Purpose of the Study The purpose of this study was to evaluate the validity of B-PADÕs Scale 3 (Overall Effectiveness) score as a measure of observed supervision and management skills. The study utilized a construct validation strategy, contrasting the B-PAD Scale 3 score with independent measures of supervision and management skills.
Methodology and Subjects This study involved the administration of B-PAD for Police Promotions to 26 incumbent police supervisors or managers, each of whom was competing for internal promotion, in two police agencies (one in Northern California and the other in Southern Washington). In addition to B-PAD, a minimum of three subordinate police officers reporting to each subject was given the Clark Wilson Survey of Management Practices (c.f. Wilson, 1988) The Survey of Management Practices (SMP) contains 145 statements of management behaviors comprising the six factors in WilsonÕs "managerial task cycle": clarification of goals and objectives, orderly work planning, facilitating the work of others, obtaining and providing feedback, exercising control, and recognition of good performance. The study followed a "double-blind" methodology: the employees rating their supervisorsÕ behavior had no knowledge of the B-PAD ratings, and the B-PAD rater had no knowledge of the SMP scores. Employees completing the SMP were assured of confidentiality; their completed surveys were mailed directly by the employees to the test publisher, and only aggregate data was used in subsequent analyses.
Results 4 of the 6 SMP factors evidenced statistically significant (p < .01) Pearson product moment correlations with the B-PAD Overall Effectiveness score. These managerial factors were: clarification of goals and objectives (r = .63), orderly work planning (r = .65), facilitating the work of others (r = .68), and recognition of good performance (r = .57). A fifth factor, "exercising control," consisted of four subfactors (time emphasis, r = .39; control of details, r = .20; goal pressure, r = -.45; and, delegation, r = .43). Of these, goal pressure and delegation were significantly correlated with B-PAD scores. No correlation was achieved between B-PAD scores and "obtaining and providing feedback" (r = .13).
Conclusion The results evidenced significant correlations between rater judgments of police promotional applicants' responses to job-relevant video scenarios and employee judgments of those applicants' management skills. With respect to the noncorrelation between "obtaining and providing feedback" and B-PAD scores, it is noted that the SMP survey statements for this factor refer to behaviors which may be independent of interpersonal competence (e.g., "Gives honest opinions of the work people do", ÒHonestly says what he/she thinks about the groupÕs performanceÓ, etc.). These behaviors, in other words, may be as characteristic of interpersonally competent managers as they are of interpersonally deficient managers. In summary, this study supports the use of B-PAD Overall Effectiveness scores as predictors of management performance in key areas of competence.