Introduction
Solving problems is a quintessential aspect of the role of an educational leader. In particular, building leaders, such as principals, assistant principals, and deans of students, are frequently beset by situations that are complex, unique, and open-ended. There are often many possible pathways to resolve the situations, and an astute educational leader needs to consider many factors and constituencies before determining a plan of action. The realm of problem solving might include student misconduct, personnel matters, parental complaints, school culture, instructional leadership, as well as many other aspects of educational administration. Much consideration has been given to the development of problem-solving skills for educational leaders. This study was designed to answer the following research question: “How do aspiring educational leaders’ problem solving skills, as well as perceptions of their problem-solving skills, develop during a year-long graduate course sequence focused on school-level leadership that includes the presentation of real-world scenarios?” This mixed-methods study extends research about the development of problem-solving skills conducted with acting administrators (Leithwood & Steinbach, 1992, 1995).
The Nature of Problems
Before examining how educational leaders can process and solve problems effectively, it is worth considering the nature of problems. Allison (1996) posited simply that problems are situations that require thought and/or actions. Further, there are different types of problems presented to educational leaders. First, there are well-structured problems, which can be defined as those with clear goals and relatively prescribed resolution pathways, including an easy way of determining whether goals were met (Allison, 1996).
Conversely, ill-structured problems are those with more open-ended profiles, whereby the goals, resolution pathways, or evidence of success are not necessarily clear. These types of problems could also be considered unstructured (Leithwood & Steinbach, 1995) or open-design (Allison, 1996). Many of the problems presented to educational leaders are unstructured problems. For example, a principal must decide how to discipline children who misbehave, taking into consideration their disciplinary history, rules and protocols of the school, and other contextual factors; determine how best to raise student achievement (Duke, 2014); and resolve personnel disputes among staff members. None of these problems point to singular solutions that can be identified as “right” or “wrong.” Surely there are responses that are less desirable than others (i.e. suspension or recommendation for expulsion for minor infractions), but, with justification and context, many possible solutions exist.
Problem-Solving Perspectives and Models
Various authors have shared perspectives about effective problem solving. Marzano, Waters, and McNulty (2005) outlined the “21 Responsibilities of the School Leader.” These responsibilities are highly correlated with student achievement based upon the authors’ meta- analysis of 69 studies about leadership’s effect on student achievement. The most highly correlated of the responsibilities was situational awareness, which refers to understanding the school deeply enough to anticipate what might go wrong from day-to-day, navigate the individuals and groups within the school, and recognize issues that might surface at a later time (Marzano et al., 2005). Though the authors discuss the utility of situational awareness for long- term, large-scale decision making, in order for an educational leader to effectively solve the daily problems that come her way, she must again have a sense of situational awareness, lest she make seemingly smaller-scale decisions that will lead to large-scale problems later.
Other authors have focused on problems that can be considered more aligned with the daily work of educational leaders. Considering the problem-type classification dichotomies of Allison (1996) and Leithwood and Steinbach (1995), problems that educational leaders face on a daily basis can be identified as either well-structured or unstructured. Various authors have developed problem-solving models focused on unstructured problems (Bolman & Deal, 2008; Leithwood & Steinbach, 1995; Simon, 1993), and these models will be explored next.
Simon (1993) outlined three phases of the decision-making process. The first is to find problems that need attention. Though many problems of educational leaders are presented directly to them via, for example, an adult referring a child for discipline, a parent registering a complaint about a staff member, or a staff member describing a grievance with a colleague, there is a corollary skill of identifying what problems—of the many that come across one’s desk— require immediate attention, or ultimately, any attention, at all. Second, Simon identified “designing possible courses of action” (p. 395). Finally, educational leaders must evaluate the quality of their decisions. From this point of having selected a viable and positively evaluated potential solution pathway, implementation takes place.
Bolman and Deal (2008) outlined a model of reframing problems using four different frames, through which problems of practice can be viewed. These frames provide leaders with a more complete set of perspectives than they would likely utilize on their own. The structural frame represents the procedural and systems-oriented aspects of an organization. Within this frame, a leader might ask whether there is a supervisory relationship involved in a problem, if a protocol exists to solve such a problem, or what efficiencies or logical processes can help steer a leader toward a resolution that meets organizational goals. The human resource frame refers to the needs of individuals within the organization. A leader might try to solve a problem of practice with the needs of constituents in mind, considering the development of employees and the balance between their satisfaction and intellectual stimulation and the organization’s needs. The political frame includes the often competing interests among individuals and groups within the organization, whereby alliances and negotiations are needed to navigate the potential minefield of many groups’ overlapping aims. From the political frame, a leader could consider what the interpersonal costs will be for the leader and organization among different constituent groups, based upon which alternatives are selected. Last, the symbolic frame includes elements of meaning within an organization, such as traditions, unspoken rules, and myths. A leader may need to consider this frame when proposing a solution that might interfere with a long-standing organizational tradition.
Bolman and Deal (2008) identified the political and symbolic frames as weaknesses in most leaders’ consideration of problems of practice, and the weakness in recognizing political aspects of decision making for educational leaders was corroborated by Johnson and Kruse (2009). An implication for leadership preparation is to instruct students in the considerations of these frames and promote their utility when examining problems.
Authors have noted that experts use different processes than novice problem solvers (Simon, 1993; VanLehn, 1991). An application of this would be Simon’s (1993) assertion that experts can rely on their extensive experience to remember solutions to many problems, without having to rely on an extensive analytical process. Further, they may not even consider a “problem” identified by a novice a problem, at all. With respect to educational leaders, Leithwood and Steinbach (1992, 1995) outlined a set of competencies possessed by expert principals, when compared to their typical counterparts. Expert principals were better at identifying the nature of problems; possessing a sense of priority, difficulty, how to proceed, and connectedness to prior situations; setting meaningful goals for problem solving, such as seeking goals that are student-centered and knowledge-focused; using guiding principles and long-term purposes when determining the best courses of action; seeing fewer obstacles and constraints when presented with problems; outlining detailed plans for action that include gathering extensive information to inform decisions along the plan’s pathway; and responding with confidence and calm to problem solving. Next, I will examine how problem-solving skills are developed.
Preparation for Educational Leadership Problem Solving
How can the preparation of leaders move candidates toward the competencies of expert principals? After all, leading a school has been shown to be a remarkably complex enterprise (Hallinger & McCary, 1990; Leithwood & Steinbach, 1992), especially if the school is one where student achievement is below expectations (Duke, 2014), and the framing of problems by educational leaders has been espoused as a critically important enterprise (Bolman & Deal, 2008; Dimmock, 1996; Johnson & Kruse, 2009; Leithwood & Steinbach, 1992, 1995; Myran & Sutherland, 2016). In other disciplines, such as business management, simulations and case studies are used to foster problem-solving skills for aspiring leaders (Rochford & Borchert, 2011; Salas, Wildman, & Piccolo, 2009), and attention to problem-solving skills has been identified as an essential curricular component in the training of journalism and mass communication students (Bronstein & Fitzpatrick, 2015). Could such real-world problem solving methodologies be effective in the preparation of educational leaders? In a seminal study about problem solving for educational leaders, Leithwood and Steinbach (1992, 1995) sought to determine if effective problem-solving expertise could be explicitly taught, and, if so, could teaching problem- processing expertise be helpful in moving novices toward expert competence? Over the course of four months and four separate learning sessions, participants in the control group were explicitly taught subskills within six problem-solving components: interpretation of the problem for priority, perceived difficulty, data needed for further action, and anecdotes of prior experience that can inform action; goals for solving the problem; large-scale principles that guide decision making; barriers or obstacles that need to be overcome; possible courses of action; and the confidence of the leader to solve the problem. The authors asserted that providing conditions to participants that included models of effective problem-solving, feedback, increasingly complex problem-solving demands, frequent opportunities for practice, group problem-solving, individual reflection, authentic problems, and help to stimulate metacognition and reflection would result in educational leaders improving their problem-solving skills.
The authors used two experts’ ratings of participants’ problem-solving for both process (their methods of attacking the problem) and product (their solutions) using a 0-3 scale in a pretest-posttest design. They found significant increases in some problem-solving skills (problem interpretation, goal setting, and identification of barriers or obstacles that need to be overcome) after explicit instruction (Leithwood & Steinbach, 1992, 1995). They recommended conducting more research on the preparation of educational leaders, with particular respect to approaches that would improve the aspiring leaders’ problem-solving skills.
Solving problems for practicing principals could be described as constructivist, since most principals do solve problems within a social context of other stakeholders, such as teachers, parents, and students (Leithwood & Steinbach, 1992). Thus, some authors have examined providing opportunities for novice or aspiring leaders to construct meaning from novel scenarios using the benefits of, for example, others’ point of view, expert modeling, simulations, and prior knowledge (Duke, 2014; Leithwood & Steinbach, 1992, 1995; Myran & Sutherland, 2016; Shapira-Lishchinsky, 2015). Such collaborative inquiry has been effective for teachers, as well (DeLuca, Bolden, & Chan, 2017). Such learning can be considered consistent with the ideas of other social constructivist theorists (Berger & Luckmann, 1966; Vygotsky, 1978) as well, since individuals are working together to construct meaning, and they are pushing into areas of uncertainty and lack of expertise.
Shapira-Lishchinsky (2015) added some intriguing findings and recommendations to those of Leithwood and Steinbach (1992, 1995). In this study, 50 teachers with various leadership roles in their schools were presented regularly with ethical dilemmas during their coursework. Participants either interacted with the dilemmas as members of a role play or by observing those chosen. When the role play was completed, the entire group debriefed and discussed the ethical dilemmas and role-playing participants’ treatment of the issues. This method was shown, through qualitative analysis of participants’ discussions during the simulations, to produce rich dialogue and allow for a safe and controlled treatment of difficult issues. As such, the use of simulations was presented as a viable means through which to prepare aspiring educational leaders. Further, the author suggested the use of further studies with simulation-based learning that seek to gain information about aspiring leaders’ self-efficacy and psychological empowerment. A notable example of project-based scenarios in a virtual collaboration environment to prepare educational leaders is the work of Howard, McClannon, and Wallace (2014). Shapira-Lishchinsky (2015) also recommended similar research in other developed countries to observe the utility of the approaches of simulation and social constructivism to examine them for a wider and diverse aspiring administrator candidate pool.
Further, in an extensive review of prior research studies on the subject, Hallinger and Bridges (2017) noted that Problem-Based Learning (PBL), though applied successfully in other professions and written about extensively (Hallinger & Bridges, 1993, 2017; Stentoft, 2017), was relatively unheralded in the preparation of educational leaders. According to the authors, characteristics of PBL included problems replacing theory as the organization of course content, student-led group work, creation of simulated products by students, increased student ownership over learning, and feedback along the way from professors. Their review noted that PBL had positive aspects for participants, such as increased motivation, real-world connections, and positive pressure that resulted from working with a team. However, participants also expressed concerns about time constraints, lack of structure, and interpersonal dynamics within their teams. There were positive effects found on aspiring leaders’ problem-solving skill development with PBL (Copland, 2000; Hallinger & Bridges, 2017). Though PBL is much more prescribed than the scenarios strategy described in the Methods section below, the applicability of real-world problems to the preparation of educational leaders is summarized well by Copland (2000):
[I]nstructional practices that activate prior knowledge and situate learning in contexts similar to those encountered in practice are associated with the development of students’ ability to understand and frame problems. Moreover, the incorporation of debriefing techniques that encourage students’ elaboration of knowledge and reflection on learning appear to help students solidify a way of thinking about problems. (p. 604)
Method
This study involved a one-group pretest-posttest design. No control group was assigned, as the pedagogical strategy in question—the use of real-world scenarios to build problem-solving skill for aspiring educational leaders—is integral to the school’s curriculum that prepares leaders, and, therefore, it is unethical to deny to student participants (Gay & Airasian, 2003). Thus, all participants were provided instruction with the use of real-world scenarios.
Participants. Graduate students at a regional, comprehensive public university in the Northeast obtaining a 6th-year degree (equivalent to a second master’s degree) in educational leadership and preparing for certification as educational administrators served as participants. Specifically, students in three sections of the same full-year, two-course sequence, entitled “School Leadership I and II” were invited to participate. This particular course was selected from the degree course sequence, as it deals most directly with the problem-solving nature and daily work of school administrators. Some key outcomes of the course include students using data to drive school improvement action plans, communicating effectively with a variety of stakeholders, creating a safe and caring school climate, creating and maintaining a strategic and viable school budget, articulating all the steps in a hiring process for teachers and administrators, and leading with cultural proficiency.
The three sections were taught by two different professors. The professors used real- world scenarios in at least half of their class meetings throughout the year, or in approximately 15 classes throughout the year. During these classes, students were presented with realistic situations that have occurred, or could occur, in actual public schools. Students worked with their classmates to determine potential solutions to the problems and then discussed their responses as a whole class under the direction of their professor, a master practitioner. Both professors were active school administrators, with more than 25 years combined educational leadership experience in public schools. It should be noted that the scenario presentation and discussions took place during the class sessions, only. These were not presented for homework or in online forums.
Of the 44 students in these three sections, 37 volunteered to participate at some point in the data collection sequence, but not all students in the pretest session attended the posttest session months later and vice versa. As a result, only 20 students’ data were used for the matched pairs analysis. All 37 participants were certified professional educators in public schools in Connecticut. The participants’ professional roles varied and included classroom teachers, instructional coaches, related service personnel, unified arts teachers, as well as other non- administrative educational roles. Characteristics of participants in the overall and matched pairs groups can be found in Table 1.
Table 1
Participant Characteristics
|
Total Participants (n = 37) |
Matched Pairs Participants (n = 20) |
|||
|
Gender |
Count |
Percent |
Count |
Percent |
|
Female |
28 |
75.7 |
16 |
80.0 |
|
Male |
9 |
24.3 |
4 |
20.0 |
|
Race/Ethnicity |
||||
|
Black |
1 |
2.7 |
0 |
0.0 |
|
Hispanic |
2 |
5.4 |
2 |
10.0 |
|
Undefined |
1 |
2.7 |
0 |
0.0 |
|
White |
33 |
89.2 |
18 |
90.0 |
|
Teaching Experience (years) |
||||
|
0-5 |
5 |
13.5 |
2 |
10.0 |
|
6-10 |
13 |
35.1 |
6 |
30.0 |
|
11-15 |
11 |
29.7 |
6 |
30.0 |
|
16-20 |
4 |
10.8 |
3 |
15.0 |
|
More than 20 |
4 |
10.8 |
3 |
15.0 |
Procedure. Participants’ data were compared between a fall of 2016 baseline data collection period and a spring of 2017 posttest data collection period. During the fall data collection period, participants were randomly assigned one of two versions of a Google Forms survey. After items about participant characteristics, the survey consisted of 11 items designed to elicit quantitative and qualitative data about participants’ perceptions of their problem-solving abilities, as well as their ability to address real-world problems faced by educational leaders. The participants were asked to rate their perception of their situational awareness, flexibility, and problem solving ability on a 10-point (1-10) Likert scale, following operational definitions of the terms (Marzano, Waters, & McNulty, 2005; Winter, 1982). They were asked, for each construct, to write open-ended responses to justify their numerical rating. They were then asked to write what they perceived they still needed to improve their problem-solving skills. The final four items included two real-world, unstructured, problem-based scenarios for which participants were asked to create plans of action. They were also asked to rate their problem-solving confidence with respect to their proposed action plans for each scenario on a 4-point (0-3) Likert scale.
During the spring data collection period, participants accessed the opposite version of the Google Forms survey from the one they completed in the fall. All items were identical on the two survey versions, except the scenarios, which were different on each survey version. The use of two versions was to ensure that any differences in perceived or actual difficulty among the four scenarios provided would not alter results based upon the timing of participant access (Leithwood & Steinbach, 1995). In order to link participants’ fall and spring data in a confidential manner, participants created a unique, six-digit alphanumeric code.
A focus group interview followed each spring data collection session. The interviews were recorded to allow for accurate transcription. The list of standard interview questions can be found in Table 2. This interview protocol was designed to elicit qualitative data with respect to aspiring educational leaders’ perceptions about their developing problem-solving abilities.
Table 2
Focus Group Interview Questions
___________________________________________________________________________________________
Please describe the development of your problem-solving skills as an aspiring educational leader over the course of this school year. In what ways have you improved your skills? Be as specific as you can.
What has been helpful to you (i.e. coursework, readings, experiences, etc.) in this development of your problem-solving skills? Why?
What do you believe you still need for the development in your problem-solving skills as an aspiring educational leader?
Discuss your perception of your ability to problem solve as an aspiring educational leader. How has this changed from the beginning of this school year? Why?
Please add anything else you perceive is relevant to this conversation about the development of your problem-solving skills as an aspiring educational leader.
___________________________________________________________________________________________
Data Analysis.
Quantitative data. Data were obtained from participants’ responses to Likert-scale items relating to their confidence levels with respect to aspects of problem solving, as well as from the rating of participants’ responses to the given scenarios against a rubric. The educational leadership problem-solving rubric chosen (Leithwood & Steinbach, 1995) was used with permission, and it reflects the authors’ work with explicitly teaching practicing educational leaders components of problem solving. The adapted rubric can be found in Figure 1. Through the use of this rubric, each individual response by a participant to a presented scenario was assigned a score from 0-15. It should be noted that affect data (representing the final 3 possible points on the 18-point rubric) were obtained via participants’ self-reporting their confidence with respect to their proposed plans of action. To align with the rubric, participants self-assessed their confidence through this item with a 0-3 scale.
|
Component/Subskill |
0 |
1 |
2 |
3 |
|
Problem Interpretation: a principal’s understanding of specifically what is the nature of the problem, often in situations where multiple problems may be identified.
|
||||
|
Goals: the relatively immediate purposes that the principal is attempting to achieve in response to his or her interpretation of the problem |
||||
|
Principles/Values: the relatively long-term purposes, operating principles, fundamental laws, doctrines, values, and assumptions guiding the principal’s thinking |
||||
|
Constraints: barriers or obstacles which must be overcome if an acceptable solution to the problem is to be found |
||||
|
Solution Processes: what the principal does to solve a problem in light of his or her interpretation of the problem, principles, and goals to be achieved and constraints to be accommodated |
||||
|
Affect: the feelings, mood, and sense of self-confidence the principal experiences when involved in problem solving |
||||
|
Total Score for Problem-Solving Process: (Add points from each row.) |
||||
0 = No Use of the Subskill
1 = There is Some Indication of Use of the Subskill
2 = The Subskill is Present to Some Degree
3 = The Subskill is Present to a Marked Degree; This is a Fine Example of this Subskill
Figure 1. Problem-solving model for unstructured problems. Adapted from “Expert Problem Solving: Evidence from School and District Leaders,” by K. Leithwood and R. Steinbach, pp. 284-285. Copyright 1995 by the State University of New York Press.
I compared Likert-scale items and rubric scores via descriptive statistics and rubric scores also via a paired sample t-test and Cohen’s d, all using the software program IBM SPSS. I did not compare the Likert-scale items about situational awareness, flexibility, and problem solving ability with t-tests or Cohen’s d, since these items did not represent a validated instrument. They were only single items based upon participants’ ratings compared to literature-based definitions. However, the value of the comparison of means from fall to spring was triangulated with qualitative results to provide meaning. For example, to say that participants’ self-assessment ratings for perceived problem-solving abilities increased, I examined both the mean difference for items from fall to spring and what participants shared throughout the qualitative survey items and focus group interviews.
Prior to scoring participants’ responses to the scenarios using the rubric, and in an effort to maximize the content validity of the rubric scores, I calibrated my use of the rubric with two experts from the field. Two celebrated principals, representing more than 45 combined years of experience in school-level administration, collaboratively and comparatively scored participant responses. Prior to scoring, the team worked collaboratively to construct appropriate and comprehensive exemplar responses to the four problem-solving scenarios. Then the team blindly scored fall pretest scenario responses using the Leithwood and Steinbach (1995) rubric, and upon comparing scores, the interrater reliability correlation coefficient was .941, indicating a high degree of agreement throughout the team.
Qualitative data. These data were obtained from open-ended items on the survey, including participants’ responses to the given scenarios, as well as the focus group interview transcripts. I analyzed qualitative data consistent with the grounded theory principles of Strauss and Corbin (1998) and the constant comparative methods of Glaser (1965), including a period of open coding of results, leading to axial coding to determine the codes’ dimensions and relationships between categories and their subcategories, and selective coding to arrive at themes. Throughout the entire data analysis process, I repeatedly returned to raw data to determine the applicability of emergent codes to previously analyzed data. Some categorical codes based upon the review of literature were included in the initial coding process. These codes were derived from the existing theoretical problem-solving models of Bolman and Deal (2008) and Leithwood and Steinbach (1995). These codes included modeling, relationships, and best for kids. Open codes that emerged from the participants’ responses included experience, personality traits, current job/role, and team. Axial coding revealed, for example, that current jobs or roles cited, intuitively, provided both sufficient building-wide perspective and situational memory (i.e. for special education teachers and school counselors) and insufficient experiences (i.e. for classroom teachers) to solve the given problems with confidence. From such understandings of the codes, categories, and their dimensions, themes were developed.
Results
Quantitative Results. First, participants’ overall, aggregate responses (not matched pairs) were compared from the fall to spring, descriptively. These findings are outlined in Table 3. As is seen in the table, each item saw a modest increase over the course of the year. Participant perceptions of their problem-solving abilities across the three constructs presented (situational awareness, flexibility, and problem solving) did increase over the course of the year, as did the average group score for the problem-solving scenarios. However, due to participant differences in the two data collection periods, these aggregate averages do not represent a matched-pair dataset.
Table 3
Fall to Spring Comparison of Likert-Scale and Rubric-Scored Items
|
Non-Paired Samples |
Paired Samples |
|||||||
|
Fall (n = 37) |
Spring (n = 30) |
Fall (n = 20) |
Spring (n = 20) |
|||||
|
M |
SD |
M |
SD |
M |
SD |
M |
SD |
|
|
Perception ofa |
||||||||
|
Situational Awarenessa |
7.16 |
1.40 |
8.06 |
1.20 |
7.05 |
1.61 |
8.30 |
1.26 |
|
Flexibilitya |
7.70 |
1.86 |
8.00 |
1.14 |
7.65 |
2.11 |
8.20 |
1.01 |
|
Problem Solvinga |
7.65 |
1.63 |
8.10 |
1.27 |
7.80 |
1.77 |
8.05 |
1.40 |
|
Average Scenario Problem-Solving Rubric Scoresb |
8.54 |
2.41 |
9.06 |
2.84 |
9.15 |
2.05 |
9.25 |
2.31 |
a These problem-solving dimensions from literature were rated by participants on a scale from 1-10.
b Participants received a rubric score for each scenario between 0-18. Participants’ two scenario scores for each data collection period (fall, spring) were averaged to arrive at the scores represented here.
In order to determine the statistical significance of the increase in participants’ problem- solving rubric scores, a paired-samples t-test was applied to the fall (M = 9.15; SD = 2.1) and spring (M = 9.25; SD = 2.3) averages. Recall that 20 participants had valid surveys for both the fall and spring. The t-test (t = -.153; df = 19; p = .880) revealed no statistically significant change from fall to spring, despite the minor increase (0.10). I applied Cohen’s d to calculate the effect size. The small sample size (n = 20) for the paired-sample t-test may have contributed to the lack of statistical significance. However, standard deviations were also relatively small, so the question of effect size was of particular importance. Cohen’s d was 0.05, which is also very small, indicating that little change—really no improvement, from a statistical standpoint—in participants’ ability to create viable action plans to solve real-world problems occurred throughout the year. However, the participants’ perceptions of their problem-solving abilities did increase, as evidenced by the increases in the paired-samples perception means shown in Table 3, though these data were only examined descriptively (from a quantitative perspective) due to the fact that these questions were individual items that are not part of a validated instrument.
Qualitative Results. Participant responses to open-ended items on the questionnaire, responses to the scenarios, and oral responses to focus group interview questions served as sources of qualitative data. Since the responses to the scenarios were focused on participant competence with problem solving, as measured by the aforementioned rubric (Leithwood & Steinbach, 1995), these data were examined separately from data collected from the other two sources.
Responses to scenarios. As noted, participants’ rubric ratings for the scenarios did not display a statistically significant increase from fall to spring. As such, this outline will not focus upon changes in responses from fall to spring. Rather, I examined the responses, overall, through the lens of the Leithwood and Steinbach (1995) problem-solving framework indicators against which they were rated. Participants typically had outlined reasonable, appropriate, and logical solution processes. For example, in a potential bullying case scenario, two different participants offered, “I would speak to the other [students] individually if they have said or done anything mean to other student [sic] and be clear that it is not tolerable and will result in major consequences” and “I would initiate an investigation into the situation beginning with [an] interview with the four girls.” These responses reflect actions that the consulted experts anticipated from participants and deemed as logical and needed interventions. However, these two participants omitted other needed steps, such as addressing the bullied student’s mental health needs, based upon her mother’s report of suicidal ideations. Accordingly, participants earned points for reasonable and logical responses very consistently, yet, few full-credit responses were observed.
Problem interpretation scores were much more varied. For this indicator, some participants were able to identify many, if not all, the major issues in the scenarios that needed attention. For example, for a scenario where two teachers were not interacting professionally toward each other, many participants correctly identified that this particular scenario could include elements of sexual harassment, professionalism, teaching competence, and personality conflict. However, many other participants missed at least two of these key elements of the problem, leaving their solution processes incomplete. The categories of (a) goals and (b) principles and values also displayed a similarly wide distribution of response ratings.
One category, constraints, presented consistent difficulty for the participants. Ratings were routinely 0 and 1. Participants could not consistently report what barriers or obstacles would need addressing prior to success with their proposed solutions. To be clear, it was not a matter of participants listing invalid or unrealistic barriers or obstacles; rather, the participants were typically omitting constraints altogether from their responses. For example, for a scenario involving staff members arriving late and unprepared to data team meetings, many participants did not identify that a school culture of not valuing data-driven decision making or lack of norms for data team work could be constraints that the principal could likely face prior to reaching a successful resolution.
Responses to open-ended items. When asked for rationale regarding their ratings for situational awareness, flexibility, and problem solving, participants provided open-ended responses. These responses revealed patterns worth considering, and, again, this discussion will consider, in aggregate, responses made in both the pre- and post- data collection periods, again due to the similarities in responses between the two data collection periods. The most frequently observed code (112 incidences) was experience. Closely related were the codes current job/role (50 incidences). Together, these codes typically represented a theme that participants were linking their confidence with respect to problem solving with their exposure (or lack thereof) in their professional work. For example, a participant reported, “As a school counselor, I have a lot of contact with many stakeholders in the school -admin [sic], parents, teachers, staff, etc. I feel that I have a pretty good handle on the systemic issues.” This example is one of many where individuals working in counseling, instructional coaching, special education, and other support roles expressed their advanced levels of perspective based upon their regular contact with many stakeholders, including administrators. Thus, they felt they had more prior knowledge and situational memory about problems in their schools.
However, this category of codes also included those, mostly classroom or unified arts teachers, who expressed that their relative lack of experiences outside their own classrooms limited their perspective for larger-scale problem solving. One teacher succinctly summarized this sentiment, “I have limited experience in being part of situations outside of my classroom.” Another focused on the general problem solving skill in her classroom not necessarily translating to confidence with problem solving at the school level: “I feel that I have a high situational awareness as a teacher in the classroom, but as I move through these leadership programs I find that I struggle to take the perspective of a leader.” These experiences were presented in opposition to their book learning or university training. There were a number of instances (65 combined) of references to the value of readings, class discussions, group work, scenarios presented, research, and coursework in the spring survey. When asked what the participants need more, again, experience was referenced often. One participant summarized this concept, “I think that I, personally, need more experience in the day-to-day . . . setting.” Another specifically separated experiences from scenario work, “[T]here is [sic] some things you can not [sic] learn from merely discussing a ‘what if” scenario. A seasoned administrator learns problem solving skills on the job.”
Another frequently cited code was personality traits (63 incidences), which involved participants linking elements of their own personalities to their perceived abilities to process problems, almost exclusively from an assets perspective. Examples of traits identified by participants as potentially helpful in problem solving included: open-mindedness, affinity for working with others, not being judgmental, approachability, listening skills, and flexibility. One teacher exemplified this general approach by indicating, “I feel that I am a good listener in regards to inviting opinions. I enjoy learning through cooperation and am always willing to adapt my teaching to fit needs of the learners.” However, rare statements of personality traits interfering with problem solving included, “I find it hard to trust others [sic] abilities” and “my personal thoughts and biases.”
Another important category of the participant responses involved connections with others. First, there were many references to relationships (27 incidences), mostly from the perspective that building positive relationships leads to greater problem-solving ability, as the aspiring leader knows stakeholders better and can rely on them due to the history of positive interactions. One participant framed this idea from a deficit perspective, “Not knowing all the outlying relationships among staff members makes situational awareness difficult.” Another identified that established positive relationships are already helpful to an aspiring leader, “I have strong rapport with fellow staff members and administrators in my building.” In a related way, many instances of the code team were identified (29). These references overwhelmingly identified that solving problems within a team context is helpful. One participant stated, “I often team with people to discuss possible solutions,” while another elaborated,
I recognize that sometimes problems may arise for which I am not the most qualified or may not have the best answer. I realize that I may need to rely on others or seek out help/opinions to ensure that I make the appropriate decision.
Overall, participants recognized that problem-solving for leaders does not typically occur in a vacuum.
Responses to focus group interview questions. As with the open-ended responses, patterns were evident in the interview responses, and many of these findings were supportive of the aforementioned themes. First, participants frequently referenced the power of group work to help build their understanding about problems and possible solutions. One participant stated, “hearing other people talk and realizing other concerns that you may not have thought of . . . even as a teacher sometimes, you look at it this way, and someone else says to see it this way.” Another added, “seeing it from a variety of persons [sic] point of views. How one person was looking at it, and how another person was looking at it was really helpful.” Also, the participants noted the quality of the discussion was a direct result of “professors who have had real-life experience” as practicing educational leaders, so they could add more realistic feedback and insight to the discussions.
Perhaps most notable in the participant responses during the focus groups was the emphasis on the value of real-world scenarios for the students. These were referenced, without prompting, in all three focus groups by many participants. Answers to the question about what has been most helpful in the development of their problem-solving skills included, “I think the real-world application we are doing,” “I think being presented with all the scenarios,” and “[the professor] brought a lot of real situations.”
With respect to what participants believed they still needed to become better and more confident problem solvers, two patterns emerged. First, students recognized that they have much more to learn, especially with respect to policy and law. It is noteworthy that, with few exceptions, these students had not taken the policy or law courses in the program, and they had not yet completed their administrative internships. Some students actually reported rating themselves as less capable problem solvers in the spring because they now understood more clearly what they lacked in knowledge. One student exemplified this sentiment, “I might have graded myself higher in the fall than I did now . . . [I now can] self identify areas I could improve in that I was not as aware of.” Less confidence in the spring was a minority opinion, however. In a more typical response, another participant stated, “I feel much more prepared for that than I did at the beginning of the year.”
Overall, the most frequently discussed future need identified was experience, either through the administrative internship or work as a formal school administrator. Several students summarized this idea, “That real-world experience to have to deal with it without being able to talk to 8 other people before having to deal with it . . . until you are the person . . . you don’t know” and “They tell you all they want. You don’t know it until you are in it.” Overall, most participants perceived themselves to have grown as problem solvers, but they overwhelmingly recognized that they needed more learning and experience to become confident and effective problem solvers.
Discussion
This study continues a research pathway about the development of problem-solving skills for administrators by focusing on their preparation. The participants did not see a significant increase in their problem-solving skills over the year-long course in educational leadership.
Whereas, this finding is not consistent with the findings of others who focused on the development of problem-solving skills for school leaders (Leithwood & Steinbach, 1995; Shapira-Lishchinsky, 2015), nor is it consistent with PBL research about the benefits of that approach for aspiring educational leaders (Copland, 2000; Hallinger & Bridges, 2017), it is important to note that the participants in this study were at a different point in their careers. First, they were aspirants, as opposed to practicing leaders. Also, the studied intervention (scenarios) was not the same or nearly as comprehensive as the prescriptive PBL approach. Further, unlike the participants in either the practicing leader or PBL studies, because these individuals had not yet had their internship experiences, they had no practical work as educational leaders. This theme of lacking practical experience was observed in both open-ended responses and focus group interviews, with participants pointing to their upcoming internship experiences, or even their eventual work as administrators, as a key missing piece of their preparation.
Despite the participants’ lack of real gains across the year of preparation in their problem- solving scores, the participants did, generally, report an increase in their confidence in problem solving, which they attributed to a number of factors. The first was the theme of real-world context. This finding was consistent with others who have advocated for teaching problem solving through real-world scenarios (Duke, 2014; Leithwood & Steinbach, 1992, 1995; Myran & Sutherland, 2016; Shapira-Lishchinsky, 2015). This study further adds to this conversation, not only a corroboration of the importance of this method (at least in aspiring leaders’ minds), but also that participants specifically recognized their professors’ experiences as school administrators as important for providing examples, context, and credibility to the work in the classroom.
In addition to the scenario approach, the participants also recognized the importance of learning from one another. In addition to the experiences of their practitioner-professors, many participants espoused the value of hearing the diverse perspectives of other students. The use of peer discussion was also an element of instruction in the referenced studies (Leithwood & Steinbach, 1995; Shapira-Lishchinsky, 2015), corroborating the power of aspiring leaders learning from one another and supporting existing literature about the social nature of problem solving (Berger & Luckmann, 1966; Leithwood & Steinbach, 1992; Vygotsky, 1978).
Finally, the ultimate theme identified through this study is the need for real-world experience in the field as an administrator or intern. It is simply not enough to learn about problem solving or learn the background knowledge needed to solve problems, even when the problems presented are real-world in nature. Scenarios are not enough for aspiring leaders to perceive their problem-solving abilities to be adequate or for their actual problem-solving abilities to improve. They need to be, as some of the participants reasoned, in positions of actual responsibility, where the weight of their decisions will have tangible impacts on stakeholders, including students.
The study of participants’ responses to the scenarios connected to the Four Frames model of Bolman and Deal (2008). The element for which participants received the consistently highest scores was identifying solution processes. This area might most logically be connected to the structural and human resource frames, as solutions typically involve working to meet individuals’ needs, as is necessary in the human resource frame, and attending to protocols and procedures, which is the essence of the structural frame. As identified above, the political and symbolic frames have been cited by the authors as the most underdeveloped by educational leaders, and this assertion is corroborated by the finding in this study that participants struggled the most with identifying constraints, which can sometimes arise from an understanding of the competing personal interests in an organization (political frame) and the underlying meaning behind aspects of an organization (symbolic frame), such as unspoken rules and traditions. The lack of success identifying constraints is also consistent with participants’ statements that they needed actual experiences in leadership roles, during which they would likely encounter, firsthand, the types of constraints they were unable to articulate for the given scenarios. Simply, they had not yet “lived” these types of obstacles.
The study includes several notable limitations. First, the study’s size is limited, particularly with only 20 participants’ data available for the matched pairs analysis. Further, this study was conducted at one university, within one particular certification program, and over three sections of one course, which represented about one-half of the time students spend in the program. It is likely that more gains in problem-solving ability and confidence would have been observed if this study was continued through the internship year. Also, the study did not include a control group. The lack of an experimental design limits the power of conclusions about causality. However, this limitation is mitigated by two factors. First, the results did not indicate a statistically significant improvement, so there is not a need to attribute a gain score to a particular variable (i.e. use of scenarios), anyway, and, second, the qualitative results did reveal the perceived value for participants in the use of scenarios, without any prompting of the researcher. Finally, the participant pool was not particularly diverse, though this fact is not particularly unusual for the selected university, in general, representing a contemporary challenge the university’s state is facing to educate its increasingly diverse student population, with a teaching and administrative workforce that is predominantly White.
The findings in this study invite further research. In addressing some of the limitations identified here, expanding this study to include aspiring administrators across other institutions representing different areas of the United States and other developed countries, would provide a more generalizable set of results. Further, studying the development of problem-solving skills during the administrative internship experience would also add to the work outlined here by considering the practical experience of participants.
In short, this study illustrates for those who prepare educational leaders the value of using scenarios in increasing aspiring leaders’ confidence and knowledge. However, intuitively, scenarios alone are not enough to engender significant change in their actual problem-solving abilities. Whereas, real-world context is important to the development of aspiring educational leaders’ problem-solving skills, the best context is likely to be the real work of administration.
References
Allison, D. J. (1996). Problem finding, classification and interpretation: In search of a theory of administrative problem processing. In K. Leithwood, J. Chapman, D. Corson, P. Hallinger, & A. Hart (Eds.), International handbook of educational leadership and administration (pp. 477–549). Norwell, MA: Kluwer Academic.
Berger, P. L., & Luckmann, T. (1966). The social construction of reality. Garden City, NJ: Doubleday.
Bolman, L. G., & Deal, T. E. (2008). Re-framing organizations: Artistry, choice and leadership (4th ed.). San Francisco: Jossey Bass.
Bronstein, C., & Fitzpatrick, K. R. (2015). Preparing tomorrow’s leaders: Integrating leadership development in journalism and mass communication education. Journalism & Mass Communication Educator, 70(1), 75–88. https://doi.org/10.1177/1077695814566199
Copland, M. A. (2000). Problem-based learning and prospective principals’ problem-framing ability. Educational Administration Quarterly, 36, 585–607.
Deluca, C., Bolden, B., & Chan, J. (2017). Systemic professional learning through collaborative inquiry: Examining teachers’ perspectives. Teaching and Teacher Education, 67, 67–78. https://doi.org/10.1016/j.tate.2017.05.014
Dimmock, C. (1996). Dilemmas for school leaders and administrators in restructuring. In K. Leithwood, J. Chapman, D. Corson, P. Hallinger, & A. Hart (Eds.), International Handbook of Educational Leadership and Administration (pp. 135–170). Norwell, MA: Kluwer Academic.
Duke, D. L. (2014). A bold approach to developing leaders for low-performing schools. Management in Education, 28(3), 80–85. https://doi.org/10.1177/0892020614537665
Gay, L. R., & Airasian, P. (2003). Educational research: Competancies for analysis and applications (7th ed.). Upper Saddle River, NJ: Pearson Education.
Glaser, B. G. (1965). The constant comparative method of qualitative analysis. Social Problems, 12(4), 436-445.
Hallinger, P., & Bridges, E. (1993). Problem-based learning in medical and managerial education. In P. Hallinger, K. Leithwood, & J. Murphy (Eds.), Cognitive perspectives on educational leadership (pp. 253–267). New York: Teachers’ College Press.
Hallinger, P., & Bridges, E. M. (2017). A systematic review of research on the use of problem- based learning in the preparation and development of school leaders. Educational Administration Quarterly. 53(2), 255-288. https://doi.org/10.1177/0013161X16659347
Hallinger, P., & McCary, C. E. (1990). Developing the strategic thinking of instructional leaders. Elementary School Journal, 91(2), 89–108.
Howard, B. B., McClannon, T. W., & Wallace, P. R. (2014). Collaboration through role play among graduate students in educational leadership in distance learning. American Journal of Distance Education, 28(1), 51–61. https://doi.org/10.1080/08923647.2014.868665
Johnson, B. L., & Kruse, S. D. (2009). Decision making for educational leaders: Underexamined dimensions and issues. Albany, NY: State University of New York Press.
Leithwood, K., & Steinbach, R. (1992). Improving the problem-solving expertise of school administrators: Theory and practice. Education and Urban Society, 24(3), 317–345. Retrieved from https://journals.sagepub.com.ccsu.idm.oclc.org/doi/pdf/10.1177/0013124592024003003
Leithwood, K., & Steinbach, R. (1995). Expert problem solving: Evidence from school and district leaders. Albany, NY: State University of New York Press.
Marzano, R. J., Waters, T., & McNulty, B. A. (2005). School leadership that works: From research to results. Denver, CO: ASCD.
Myran, S., & Sutherland, I. (2016). Problem posing in leadership education: Using case study to foster more effective problem solving. Journal of Cases in Educational Leadership, 19(4), 57–71. https://doi.org/10.1177/1555458916664763
Rochford, L., & Borchert, P. S. (2011). Assessing higher level learning: Developing rubrics for case analysis. Journal of Education for Business, 86, 258–265. https://doi.org/10.1080/08832323.2010.512319
Salas, E., Wildman, J. L., & Piccolo, R. F. (2009). Using simulation based training to enhance management education. Academy of Management Learning & Education, 8(4), 559–573. https://doi.org/10.5465/AMLE.2009.47785474
Shapira-Lishchinsky, O. (2015). Simulation-based constructivist approach for education leaders. Educational Management Administration & Leadership, 43(6), 972–988. https://doi.org/10.1177/1741143214543203
Simon, H. A. (1993). Decision making: Rational, nonrational, and irrational. Educational Administration Quarterly, 29(3), 392–411. https://doi.org/10.1177/0013161X93029003009
Stentoft, D. (2017). From saying to doing interdisciplinary learning: Is problem-based learning the answer? Active Learning in Higher Education, 18(1), 51–61. https://doi.org/10.1177/1469787417693510
Strauss, A., & Corbin, J. (1998). Basics of qualitative research (2nd ed.). Thousand Oaks, CA: Sage.
VanLehn, K. (1991). Rule acquisition events in the discovery of problem-solving strategies. Cognitive Science, 15(1), 1–47. https://doi.org/10.1016/0364-0213(91)80012-T
Vygotsky, L. S. (1978). Mind in society. Cambridge, MA: Harvard University Press.
Winter, R. (1982). Dilemma analysis: A contribution to methodology for action research. Cambridge Journal of Education, 12(3), 166-173.
Author Biography
Dr. Jeremy Visone is an Assistant Professor of Educational Leadership, Policy, & Instructional Technology. Until 2016, he worked as an administrator at both the elementary and secondary levels, most recently at Anna Reynolds Elementary School, a National Blue Ribbon School in 2016. Dr. Visone can be reached at visone@ccsu.edu.