Introduction
Little is known about the instructional and assessment strategy choices of leadership educators in higher education outside of recently published quantitative studies (Jenkins 2012, 2013, 2016, 2018) and a review of the sources of learning in undergraduate leadership programs (Allen & Hartman, 2009; Eich, 2008; Zimmerman-Oster & Burkhardt, 1999). As the number of leadership programs surpasses 2,000 globally (ILA Program Directory, 2020), so does the need for scholars to study the pedagogical habits of leadership educators. For the more that is known about how leadership is taught, the more possibilities emerge for targeted and relevant leadership educator professional development programs and resulting student learning.
Additionally, the literature related to the experiences of leadership educators, particularly through qualitative and mixed methods research designs, is scant. Only recently have scholars explored leadership educators’ journeys becoming and being leadership educators (Jenkins, 2019; Priest & Jenkins, 2019a) and the factors that shape their professional identities (Guthrie & Jenkins, 2018; Jenkins, 2019; Priest & Jenkins, 2019b; Priest & Seemiller, 2018; Seemiller & Priest, 2015, 2017). This study was proposed to examine specifically leadership educators’ experiences “doing” leadership education and further understand how they approach their craft.
Objectives of the Study. The objectives of the study were to:
- Identify the instructional and assessment strategies leadership educators use most, and
- Explore in-depth leadership educators’ pedagogical habits and decision-making processes.
Limitations. Limitations of this study include:
- The quantitative phase of the study tapped into a select population of leadership studies instructors from specific professional association databases, listservs, and directories. The researcher cannot say with confidence the sample will be representative of the population.
- Equally, due to the funding available to the researcher, the qualitative phase of the study was limited to 13 leadership educators from four states and 11 universities, nine of which were associated with institutions in Ohio or Illinois. The researcher cannot say with confidence the sample will be representative of the population.
- The participants in the quantitative phase of this study were derived from an international sample where the participants in the qualitative phase of this study were from a U.S.-based sample only.
- While a snowball sampling technique (Creswell & Poth, 2018) was used to identify potential participants for this study, none of the participants who were recommended to the researcher were people of color. As a result, the diversity of participants included in the qualitative phase of this study do not fairly or accurately represent the diversity of leadership educators, nor is this group of participants in line with the diversity of the participants in the quantitative phase of this study.
- The focus of this research was on the instructional strategy use of leadership educators in face-to-face modalities only.
- This research included only leadership instructors who reported teaching an academic credit-bearing course within the last two years. This population does not include the thousands of student affairs professional who facilitates leadership learning in co-curricular contexts.
- Due to the nature of qualitative research, the data obtained in the second phase of the study may be subject to different interpretations by different readers. Moreover, because of the interpretative nature of the qualitative research, the investigator may introduce his bias into the analysis of the findings.
Methods
Study Design. A follow-up explanations variant of the sequential Explanatory mixed methods design (Creswell & Plano Clark, 2018) including a (a) survey of leadership educators to identify the frequency of use of instructional and assessment strategies, and (b) qualitative interviews with recommended “exemplary” leadership educators exploring their experiences teaching leadership were employed. Integration (Creswell & Plano Clark, 2018) occurred during the interview protocol development process which was informed by the results from the initial quantitative phase with the aim of investigating survey results in more depth. The results of both phases were integrated to develop a more robust and meaningful snapshot of the pedagogical practices of leadership educators (see Table 1 for detailed information as well as the order of phases for the mixed methods sequential explanatory design procedures in the study). Correspondingly, this paper is organized in the following order per Creswell and Plano Clark (2018): (a) Quantitative Phase—data collection and analysis; (b) Qualitative Phase—data collection and analysis; (c) Results—Quantitative and Qualitative Phases; (d) Discussion; (e) Implications and Recommendations; and (f) a Conclusion.
Table 1.
Order of Phases: Mixed Methods Sequential Explanatory Design Procedures
Phase | Procedure | Product |
1. QUANTITATIVE Data Collection | · Web-based Questionnaire (N = 836)
|
· Numeric data |
2. Quantitative Data Analysis | · Data screening
· Frequencies · Means scores · SPSS quan. Software v. 24
|
· Descriptive statistics, missing data, linearity |
3. Snowball Sampling; Interview Protocol Development | · Participant selection based on snowball sampling method
· Developing interview questions
|
·
· Interview protocol |
4. Qualitative Data Collection | · Individual in-depth face-to-face interviews with 13 participants
|
· Interview transcripts
· Short written questionnaires |
5. Qualitative Data Analysis | · Coding and thematic analysis
· Within-case and across-case theme development · Cross-thematic analysis · NVIVO qualitative software
|
· Codes and themes
· Similar and different themes and categories · Cross-thematic matrix |
6. Integration of Quantitative and Qualitative Results | · Interpretation and explanation of the quantitative and qualitative results | · Discussion
· Implications · Future research |
Quantitative Phase
Data Collection. For the first, quantitative phase, a web-based survey was used. The survey questionnaire was modeled after the approach used by Jenkins (2012, 2013) to collect data identifying the most frequently used instructional and assessment strategies for teaching undergraduate face-to-face leadership studies courses. Here, the survey was used to identify the most frequently used pedagogies for teaching leadership studies courses and profile study participants.
The analyzed survey data was collected from a web-based questionnaire through an international study that targeted thousands of leadership studies instructors through three primary sources from March 31, 2013, through May 3, 2013. The first source was the organizational memberships or databases of the following professional associations/organizations or their respective member interest groups: (a) the ILA; (b) the Association of Leadership Educators (ALE); (c) NASPA, Student Leadership Programs Knowledge Community (NASPA SLPKC); and (d) the National Clearinghouse for Leadership Programs (NCLP). The second source was the attendee list of the 2012 Leadership Educators Institute (LEI). The third source was a random sample of instructors drawn from the ILA Directory of Leadership Programs.
While the first and second sources were more so “shotgun approaches,” they were also more likely to include ideal participants. While the ILA member database, ILA Directory of Leadership Programs, and LEI Attendee list provided access to members or attendees respectively, the researcher did not have access to the individual e-mails for the NASPA SLPKC, ALE, and NCLP groups. And, while the listserv managers did send out invitation e-mails to participate in this study’s survey to their respective listservs, return rates are not available due to the undisclosed number of recipients. Nonetheless, the return rates for the ILA member directory (12.57%), ILA Directory of Leadership programs (11.25%) and LEI (25.08%) were promising. Overall, these data collection procedures provided the researcher with the best possible sources to generalize to the population.
Participants. Survey respondents were 836—390 graduate-level (GL) and 446 undergraduate-level (UL)—instructors who self-reported teaching an academic, credit-bearing face-to-face leadership studies course in the previous two years. This is the largest reported study of these populations to date. After participants’ eligibility was confirmed (i.e., they had taught a course within the previous two years), they identified one specific corresponding GL or UL academic credit-bearing course and were asked to use that course as a reference point when completing the survey.
Data Analysis. Descriptive statistics were used to analyze the means and standard deviations of the item responses indicating frequency of instructional strategy use as well as the overall percentages of assessment strategy use. Participants were asked to describe their frequency of use of the instructional strategies listed in Table 2. Frequency of use of each instructional strategy was ranked using the following scale: (a) 1 = Never; (b) 2 = Rarely; (c) 3 = Occasionally; (d) 4 = Frequently; and (e) 5 = Almost Always / Always. The rating scale for assessment strategy use included the strategies listed in Table 3 and was designed to capture the overall weight instructors placed on each strategy with respect to students’ overall grades in their courses. Accordingly, participants reported the level of weight toward a student’s final grade each assessment strategy was given in their courses using the following scale: (a) 1 = 0%, I do not use this type of assessment in my course; (b) 2 = 1-10%; (c) 3 = 11-20%; (d) 4 = 21-30%; (e) 5 = 31-40%; (f) 6 = 41-50%; and (g) 51% or more. For a complete list of the definitions of instructional and assessment strategies provided to survey participants, see Tables 2 and 3 below. Also, for a comparison of the differences between undergraduate and graduate leadership educators, please see Jenkins (2018).
Table 2.
Face-to-Face Instructional Strategies: Survey-Item Descriptions
Instructional Strategy | Description |
Case Studies | Students examine written or oral stories or vignettes that highlight a case of effective or ineffective leadership. |
Class Discussion | Instructor facilitates sustained conversation and/or question and answer segment with the entire class. |
Debates | Student teams argue for or against a position using course concepts, evidence, logic, etc. |
Games | Students engage in interactions in a prescribed setting and are constrained by a set of rules and procedures. (e.g., Jeopardy, Who Wants to be a Millionaire, Family Feud, etc.) |
Guest Speaker | Students listen to a guest speaker/lecturer discuss their personal leadership experiences. |
Icebreakers | Students engage in a series of relationship-building activities to get to know one another. |
In-Class Short Writing | Students complete ungraded writing activities such as reflective journals or responses to instructor prompts designed to enhance learning of course content. |
Interactive Lecture/Discussion | Instructor presents information in 10-20-minute time blocks with period of structured interaction/discussion in-between mini-lectures. |
Lecture | Students listen to instructor presentations lasting most of the class session. |
Media Clips | Students learn about leadership theory/topics through film, television, or other media clips (e.g., YouTube, Hulu). |
Problem-based Learning | Students learn about leadership through the experience of problem solving in specific situations. |
Role Play Activities | Students engage in an activity where they act out a set of defined role behaviors or positions with a view to acquire desired experiences. |
Self-Assessments & Instruments | Students complete questionnaires or other instruments designed to enhance their self-awareness in a variety of areas (e.g., learning style, personality type, leadership style, etc.). |
Service Learning | Students participate in a service learning or philanthropic project. |
Simulation | Students engage in an activity that simulates complex problems or issues and requires decision-making. |
Small Group Discussion | Students take part in small group discussions on course topics. |
Stories or Storytelling | Students listen to a story highlighting some aspect of leadership; often given by an individual with a novel experience. |
Student Peer Teaching | Students, in pairs or groups, teach designated course content or skills to fellow students. |
Teambuilding | Students engage in group activities that emphasize working together in a spirit of cooperation (e.g., setting team goals/priorities, delegating work, examining group relationships/dynamics, etc.). |
Table 3
Face-to-Face Assessment Strategies: Survey-Item Descriptions
Assessment Strategy | Description |
Class Participation/Attendance | Students are given points for active participation in course activities. |
Exams | Students complete tests or exams that last the majority of the class period intended to assess subject matter mastery. |
Group Projects/Presentations | Students work on a prescribed project or presentation in a small group. |
Individual Leadership Development Plans | Students develop specific goals and vision statements for individual leadership development. |
Major Writing Project/ Term Paper | Students write a significant paper exploring course content or research (such as a literature review) as a major course assignment. |
Observation/Interview of a Leader | Students observe or interview an individual leading others effectively or ineffectively and report their findings to the instructor/class. |
Portfolio or evidence collection | Students document their own learning through the creation of a course portfolio. |
Quizzes | Student complete short graded quizzes intended to assess subject matter mastery. |
Read and Respond | Students develop written reflections on their experiences or understandings of lessons learned about course content. |
Reflective Journals | Students are graded on their responses to questions generated by the instructor or from the end of the text chapter for the purpose of allowing students to explore specific ideas or statements in depth and breadth. |
Research Projects/Presentations | Students actively research a leadership theory or topic and present findings in oral or written format. |
Self-evaluation | Students respond in writing to criteria set for evaluating their learning. |
Short Papers | Students author one or more short papers (ten pages or less in length) exploring course content. |
Skill Demonstration | Students physically represent learning through problem solving ability in relevant contexts. |
Student Peer Assessment | Students critique other students’ work using previously described criteria and provide specific suggestions for improvement. |
Video Creation | Students create short video presentations to be shown in class. |
Qualitative Phase
Qualitative Research Design.
Participant selection. A snowball sample of 13 leadership educators—eight men and five women—from four states and 11 universities, participated in the qualitative phase of this study. According to Creswell and Poth (2018), a snowball sample requires that the researcher identifies cases of interest from people who know people who know that cases are information rich. Thus, during the fall of 2014, the researcher contacted 25 preeminent leadership educators in their network and asked them to “recommend three exemplary leadership educators to participate in interviews” who had “taught an undergraduate- or graduate-level academic credit-bearing leadership studies course in the last two years.” 15 of the 25 individuals contacted provided at least two potential participants. The researcher then invited potential participants who were either (a) recommended more than once; and/or (b) lived within a reasonable proximity of certain metro areas that the researcher could access with available research funds. Next, the researcher contacted 19 potential participants via e-mail, citing the individual who had referred them, and asked for their voluntary participation in the study. Four of the 19 potential participants opted out and three others could not be accommodated by their own or the researcher’s schedule. Interviews took place between January and April of 2015 at each participant’s university and most often, in their workspace. Interviews ranged in length from 76 to 131 minutes and the average interview lasted 86 minutes. The same researcher conducted all 13 interviews.
Data collection. The researcher obtained informed consent from each participant based on the guidelines required by the researcher’s IRB and asked each participant to also complete a short questionnaire of 15 questions related to their demographics, education, and teaching experience (see Table 4). No incentives were provided. Then, each participant was asked to verbally respond to the semi-structured interview protocol questions, guided by open-ended questions that lead into topical areas including assessment procedures, course design, learning outcomes, and intentionality. Accordingly, the following questions and sub questions related specifically to participants’ pedagogical use and are reported in this study: (a) What instructional strategies do you find that you use more than others? Why? (b) What instructional strategies do you tend to avoid? Why so? (c) Describe one of the most effective activities you’ve ever facilitated in a leadership course. What contributed to its effectiveness? What did the students report learning? (d) Describe one of the least effective activities you’ve ever facilitated in a leadership course. What contributed to its ineffectiveness? What did the students report learning? € Describe one of the most effective assignments you’ve ever designed for your leadership students. What contributed to its effectiveness? What did the students report learning? (f) Describe one of the least effective assignments you’ve ever designed for your leadership students. What contributed to its ineffectiveness? What did the students report? (g) How do you feel the subject matter, e.g., ethics, organizational theory, social change, etc., affects the type of instructional or assessment strategies you use? And (h) Do you find that you prefer particular instruction/assessment strategies for particular subjects? Does instructional/assessment strategy use vary significantly by class type, e.g., Intro, Capstone, Teams, etc.?
Table 4.
Qualitative Phase: Participant Demographics
Participant
Code |
Gender | State | Degree & Discipline | Current Position | Yrs. in current position | Yrs. teaching leadership | Yrs. in higher education |
M1 | M | IL | Ph.D.; Comm. Studies / OC&L | Assoc. Dir.; Clinical Assoc. Prof.; Engineering | 7 | 11 | 8 |
G1 | M | IL | Ph.D.; Public Affairs | Dir.; Clinical Assoc. Prof.; Engineering | 8 | 28 | 28 |
D1 | M | IL | Ph.D.; Counseling | Assoc. Prof.; HE | 8 | 15 | 15 |
R1 | M | IL | Ph.D.; HE. | Asst. Prof. | 4 | 11 | 17 |
A1 | M | OH | Ph.D.; Leadership & Change | Assoc. Prof.; Management | 7 | 15 | 8 |
S1 | M | MD | Ed.D.; Ed. Leadership | Sr. Lecturer; Engineering | .5 | 7 | 33 |
L1 | F | OH | Ph.D.; HE Admin | Dir. of non-profit; Adj. | 2.5 | 25 | 25 |
H1 | F | OH | Doctorate, Org. Leadership | Asst. Prof.; Counseling and HE | 3.5 | 9 | 20 |
R2 | F | MD | M.A.; HE & SA | Ph.D. student; Coordinator; Adj.; Ed. | 2 | 8 | 9 |
M2 | M | DE | Ph.D.; Ed. Psych. | Assoc. Prof.; OC&L | 10 | 15 | 15 |
B1 | F | OH | Ed.D.; HE Policy, Planning, & Leadership | Senior Lecturer; HE & SA | 3.5 | 10 | 17.5 |
L2 | F | OH | Ph.D.; Ed. Leadership | Chair & Prof.; LS | 16 | 14 | 40 |
O1 | M | MD | Ph.D.; CSP | Asst. Dir.; Adj. | 1 | 12 | 22 |
Note: HE = Higher Education; SA = Student Affairs; Adj. = Adjunct; CSP = College Student Personnel; LS = Leadership Studies; OC&L = Organizational Communication & Leadership
Qualitative analysis. The process of qualitative data collection and analysis occurred iteratively (Creswell & Poth, 2018). In this method, the researcher—and one graduate assistant who was recruited to assist in analyzing the interview transcripts for themes—read the transcripts while simultaneously listening to participants’ corresponding audio interviews to obtain an overall feeling for them. Then, building on the data from the interview questions, the researcher followed the steps of horizontalization (Moustakas, 1994), went through the interview transcripts using NVivo qualitative data analysis software and highlighted “significant statements,” sentences, and quotes that provided understanding of leadership educators’ pedagogical habits and developed clusters of meaning from these significant statements into themes (Creswell & Poth, 2018).
Results
Quantitative Phase.
Demographic information. Demographic information was collected from participants in the survey to better understand the educational and preparatory experiences of leadership educators. Additionally, questions related to participants’ home institution, program, and department offerings were also included (see Table 5).
Table 5.
Quantitative Phase: Demographic and Educational Majority Survey Data
Question | Graduate Instructors | Undergraduate Instructors |
Gender | 56.5% “Male” | 51.2% “Female” |
Race/Ethnicity | 79.1% “White/Caucasian” | 83.9% “White/Caucasian” |
Age | 35.2% “55 to 64” | 23.8% “55 to 64” |
Location of Institution | 77.1% “USA” (5.7% in both
“Canada” and the “UK”) |
88.9% “USA” (4.4% “Canada”) |
Institution Type | 54.6% “4-year Private University” | 58.4% “4-year Public University” |
College where Leadership course was located | 32.3% “Business or
Management” (25.0% “Education”) |
18.8% “Business or
Management” (12.2% “Academic Affairs, College-wide, General Education, or no affiliated college”) |
Academic Department where Leadership course was located | 27.5% “Leadership,
Organizational Leadership, or Leadership Studies” (25.0% “Management”) |
19.6% “Leadership,
Organizational Leadership, or Leadership Studies” (7.3% “Management”) |
Course Type | 12.1 % “Special/Multiple Topics”
(6.9% “Organizational/Groups/Teams”) |
14.8% “Introductory
Leadership” (11.4% “Special/Multiple Topics”) |
Leadership Degree Offered | 45.9% “Master’s” (20.3% “M.B.A.”) | 36.8% “Minor” (33.2%
“Baccalaureate”) |
Primary Activity at Institution | 58.1% “Full-time faculty” (15.8% “Part-
time faculty or adjunct”) |
45.5% “Full-time faculty”
(23.2% “Full-time staff/administration) |
Years in Current Position | 32.4% “more than 10 years” | 35.6% “1-3 years” |
Average Class Size of Course Indicated | 58.5% “15 – 29 students” | 59.5% “15 – 29 students” |
Terminal Degree | 84.2% “Doctorate” | 60.7% “Doctorate” |
Degree Area | 12.7% “Leadership” (12.0%
“Management”) |
10.5% “College Student
Affairs, Development, or Personnel” (9.0% “Higher Education”) |
Descriptive statistics. As discussed above, descriptive statistics were used to analyze the means and standard deviations of the item responses indicating frequency of instructional strategy use as well as the overall percentages of assessment strategy use (see Tables 6 and 7).
Table 6.
Instructional Strategy Use of Leadership Educators
Instructional Strategy | M | SD |
Class Discussion | 4.73 | 0.48 |
Interactive Lecture/Discussion | 4.19 | 0.87 |
Small Group Discussion | 4.00 | 0.93 |
Self-Assessments & Instruments | 3.50 | 1.13 |
Media Clips | 3.43 | 0.92 |
Case Studies | 3.42 | 1.02 |
Problem-based Learning | 3.25 | 1.09 |
Teambuilding | 3.24 | 1.23 |
Lecture | 2.97 | 1.18 |
Guest Speaker | 2.84 | 1.02 |
Icebreakers | 2.81 | 1.14 |
Stories or Storytelling | 2.77 | 1.23 |
Student Peer Teaching | 2.76 | 1.34 |
Role Play Activities | 2.67 | 1.14 |
Debates | 2.56 | 1.13 |
Games | 2.53 | 1.11 |
In-Class Short Writing | 2.48 | 1.14 |
Simulation | 2.39 | 1.17 |
Service Learning | 2.33 | 1.40 |
Note: Of the 836 survey participants, only N = 622 (Graduate: n = 272; Undergraduate: n = 350) progressed through the survey to the questions represented in Table 6.
Table 7.
Assessment Strategy Use of Leadership Educators
Instructional Strategy | M | SD |
Research Projects/Presentations | 3.33 | 1.68 |
Short Papers | 3.25 | 1.94 |
Major Writing Project/ Term Paper | 2.94 | 1.57 |
Student Peer Assessment | 2.69 | 1.87 |
Group Projects/Presentations | 2.56 | 1.89 |
Skill Demonstration | 2.48 | 1.57 |
Self-evaluation | 2.36 | 1.67 |
Class Participation/Attendance | 2.33 | 1.59 |
Read and Respond | 2.07 | 1.49 |
Individual Leadership Development Plans | 1.86 | 1.32 |
Observation/Interview of a Leader | 1.80 | 1.33 |
Portfolio or evidence collection | 1.76 | 1.40 |
Video Creation | 1.71 | 1.22 |
Reflective Journals | 1.66 | 1.36 |
Exams | 1.51 | 1.01 |
Quizzes | 1.43 | 1.12 |
Note: Of the 836 survey participants, only N = 606 (Graduate: n = 263; Undergraduate: n = 343) progressed through the survey to the questions represented in Table 7.
Qualitative Phase. The analysis of the 13 participant interviews yielded rich data related to instructional and assessment strategy use in specific pedagogical groups as well as five themes related to participants’ experiences teaching pedagogical decision-making. Descriptions of the pedagogical groups and five themes follows.
Instructional and assessment strategy use.
Most frequently used instructional strategies. Interview participants echoed much of the results from the quantitative phase, underlining discussion-based instructional strategies, group work, and reflection. And, while case studies, self-assessments & instruments, and media clips did come up in conversation, either the instructional strategy was embedded in another theme (e.g., case study as part of group work) or the frequency or depth provided by participants were not rich enough to generate a standalone theme.
Discussion. Participants described discussion as the “centerpiece of every class I teach” (D1) and offered multiple ways in which they engaged students in discussion. Discussion strategies ranged from interactive discussion that included brief five- or ten-minutes lectures concurrently with prompting questions or provocation statements as well as mixing up the size of the discussion groups (e.g., pairs, small groups, or the entire class) and duration of the discussion. The constructive and engagement factors of discussion seemed to be tantamount to choosing this instructional strategy:
H1: I’m a big fan of the interactive discussion. Because I think talking is generative. I think you get ideas from stating them
M1: So, lots of interactive discussion. I would say that’s probably the majority here’s a five-minute lecture, 10-minute lecture and now I’ll throw out some prompting questions. Once you discuss this with your neighbor for 5 minutes and then let’s talk about what you came up with. I tend to do that a lot.
R1: Can inviting students to share both their perspectives about the curriculum and how they see the curriculum playing out in their life? It is founded upon the idea that if they can’t translate what we are talking about in the class to what their life is like they are never going to be able to implement it and talking helps people.
Group work. Participants emphasized various strategies of group work in their courses including think-pair-share, impromptu small group work such as creating a short presentation or skit around content, peer feedback, case studies, and teambuilding and team-based challenges or competitive learning activities. Participants stressed the social aspects of pairing and grouping students as well as skill development around communication and feedback:
M2: It forces them to engage in the material from multiple directions and it also provides context for them to learn from each other and learn how to work together.
B1: …so I am trying to mediate and mitigate that by having the small groups that are having di-ads or they are processing through so everyone has to talk and participate, and they don’t all have to choose to report out but at least and I walk around and I kind of listen in on the different conversations that are happening. So very frequently it is just sticking some discussion words up on the screen, … having them chat through those and then coming back as a large group and comparing the different conversations that occurred.
L1: And I think generally that’s been pretty effective hearing from a peer like I don’t get it, it’s a lot more powerful than me saying I don’t get it you know, or I sort of get where you’re going but I want you to explain it better. Whereas a peer maybe doesn’t get where you’re going, and you really do have to explain it better.
Reflection. Participants shared a variety of reflection-based teaching strategies ranging from journals, short papers, reactions to self-assessments and instruments, daily blogs or observations, and semester-long self-reflections. Participants stressed the connection between reflection and developing students’ personal narratives, critical self-reflection, and meaning making:
R2: …it’s just as important for them to reflect on how their understanding of leadership is changing throughout the semester… as much as it is for them to learn the material
D1: I think sometimes that’s a deep reflection exercise and that’s really about critical self-reflection and then sometimes it’s more an analytical process. I would say narrative becomes huge, so, I try and use narrative … and bring that into the classroom… because constructing and creating a space for narrative feels pedagogical if people don’t do that naturally, but the actual use of narrative feels like the content.
Least used instructional strategies. Here as well, interview participants echoed the results from the quantitative phases, highlighting their avoidance of tests and quizzes, simulation, role-play, and games, and lecture-based methods. Traditional assessment (i.e., tests and quizzes) and lecture were cited for being irrelevant to the type of learning leadership education was intended to deliver and simulation, role-play, and games were avoided due to the often overly complicated preparation and/or facilitation required to use them.
Tests and quizzes. Participants explained how traditional assessment such as tests, quizzes, and exams were inappropriate for types of learning outcomes they had established for their courses. Further, they often avoided rote memorization in favor of applied or “relevant” learning outcomes and activities.
M1: I almost never quiz them … I almost never test them on knowledge for the sake of, like knowing material just for the sake of knowing it. They always have to write it in a way that how would you apply this.
B1: Because I think that is one way to just memorize something and forget it. I mean a lot of what I read about learning is not sort of indicative or assessed well from the test especially the kind of content that I think we are teaching in leadership. I don’t really need them to memorize these different aspects of the situational leadership model, I need them to learn how to apply it. And so if I was going to give a test it would be very like here is a case, analyze it using a leadership theory and then we do a lot of that in class so I know they can do it.
M2: I don’t think exams effectively measured the kinds of things that I’m trying to accomplish.
L1: I don’t tend to like high stakes assessments, but I do tend to try to have an array of assessments across the term that give a gauge on progress, so that I’m not stuck, and the students not stuck here being so much weight on one moment in time. I don’t particularly think that that is reflective of life.
Lecture. Participants shared several reasons why they avoided lecture and its media counterpart PowerPoint. Most often cited were passive nature of lecture, its non-experiential roots, and how it seemed impractical for teaching leadership.
O1: I tend to avoid the chalk and talk. I almost never do the power point of content, never almost never; almost never do I do the power point of content. Almost everything is going to be experiential or practical based. I’ll do content but it’s almost to get them up to speed, to be able to get to the experiential practical piece.
D1: I hate lecture—I just hate it.
H1: I really try to avoid PowerPoint or any of that Prezi, it’s very helpful when preparing for class it’s not that helpful I find to deliver–its boring,
S1: I tried to avoid straight lecturing; behind the podium I don’t think I hardly ever do that anymore… I just have to make sure I know the material and I’m out in front of things and I’m working the class and we are talking. I’m not saying that I never do it, but for the most part it’s really very rare when I talk from that kind of a thing vantage point.
Role-play, simulation, and games. Participants denoted that highly experiential learning activities such as role-play, simulation, and games were usually hit or miss, costly, complicated to prepare for and facilitate, and sometimes did not invoke long-term learning.
G1: … we were taking them on the proverbial ropes course and they would come back and say, ‘That was great, I had a lot of fun, I learned one or two things’, I mean they really were happy with the experience, it was not a dissatisfier to them and then you would ask them a week or two later what did you learn from that experience and there wouldn’t be a whole lot of traction
L1: I’m not real keen on games, I will say. I have some colleagues who are really good with games. Games aren’t my thing; they aren’t my jam.
Effective and ineffective activities and assignments. The following themes emerged from participants’ descriptions of learning activities they deemed most effective or ineffective as well as the experiences (e.g., emotional response) and factors (e.g., student learning, positive or negative feedback) that led to their reported outcomes.
Intentional, relevant, and meaningfully connected to experience. Participants recounted exemplar learning activities that sparked dialogue among students around individual and shared perspectives, ways of being, context, and identity, and provided opportunities for students to connect content to past experiences (i.e., reflection) or apply it to and create new experiences (e.g., an applied project). In both cases, the ensued dialogue was the vehicle for making the content relevant for students.
D1: I think trying to create experiential opportunity that situates someone’s lived experience in the content… there’s this awesome connection between academic learning and emotional learning… and then it gives them an inroad to say, it’s okay if I resist this content or I am not my past experiences, or I can like something despite having bad experiences in the past… like it troubles all of that. Like it’s almost like saying it’s gonna be okay that this is gonna be a contested terrain that we’re exploring all semester.
L2: I try to design an activity that inherently represents what it is we’re talking about. So, if we’re talking about power then I’m going to try to create a set of activities where people actually exercise power within the activity, in the case of choice the same thing.
R2: especially working with undergrads… the theory … the concept … which is so heavy for them, that they need something tangible as an example to kind of point to and say oh I get it now, like that’s how this concept shows up in real life
S1: …coming up with a big problem on campus and then having them get a small team and help develop some strategies as leader… leading social change. So, they’re developing business plans and presentations for this competition and we do all kinds of skill building, again it’s very intense and intensive and, but the end product is if they win the grant then they need to implement this and so, part of their planning is how to sustain it and where they’re going to get support once the class ends and sometimes that’s from me, but in, many other times it’s from other members of the community…
B1: …anything ethics, I want them to feel an ethical dilemma… how you reconcile a real dilemma that means something to you.… if you make it personal then they can really dig into it… if you don’t design something that they can relate to then it is not going to be as meaningful.
M2: …it’s a matter of figuring out the most effective way to hook them into it such that they are going to engage it more deeply… how can I get you to take this seriously and think about, where this leadership theory is, or this created problems of the process or this particular skill. So, I think it matters in the sense of you have to contextualize it individually in terms of the content.
R2: I want them to understand that what we’re talking about is applicable to their lives after they leave … and that’s rooted in why I became a leadership educator in the first place… the more ways that I can get them to apply the theory and the models to their own lives right now the more likely they are to then see the connections even after they leave my class… from a student development standpoint … teaching students how and analyze and synthesize information and apply it to various situations, that’s part of my duty… It’s… why… the college experience exists … applying the leadership content is important… From a practical standpoint I think them learning how to work with different people that have different perspectives from them that approach projects differently than they do… that’s important… the really very meaningful connections that they made between their formal leadership learning and how they had seen that unfold throughout their college experience.
Skill development. Participants reported long lasting and high levels of student learning from skills-based learning activities. Learning outcomes ranged from various “how to’s” to coaching, assessment, conflict resolution, and communication strategies.
R1: That process needs to be practiced the same way that football players can sit in a lecture hall talking about blocking schemes but until you get out on the football field, it doesn’t matter, or it does matter, you don’t know if it is going to work. So, practice… in class like its class practice, leadership practice is these experiential activities
M1: They learn how to speak up, they learn how to resolve conflict… I’m going to really ding them on it then they’d rather really work out their differences they can’t just give something that pleases everybody regardless of how ridiculous the answer is so, that forces them to… resolve their difference… effectively as a team they have to do things to get everybody’s input.
Creativity. Participants described meaningful outcomes from their experience and the feedback received from students around learning activities that synthesized content into creative activity, particularly when integrated with media and art.
B1: … ‘Life of You’ paper… about their life and where they are headed, and they have to come up with some creative way to present the content… I have had students sing songs … do spoken word and … tap into your creativity and show me the content of your paper
R1: … act out a two act play that demonstrates their assigned Tuckman stage. At the end of Act 1 what does the leader do… someone needs to step up to get the group to start transitioning from your assigned stage to the next stage. … we bring bags of popcorn, … you play up all that stuff… I love that activity. Some of the students hate it, very few of them hate it at the end…
Ineffective activities and assignments.
Challenges around team-based learning. While participants seemed to value team-based learning (TBL), they shared myriad challenges stemming from students’ past experiences in TBL, group conflict, and having to provide group processes for students to follow.
D1: like everyone I’ve had bad group project experiences, so I didn’t… as much in a leadership class which is asinine because you have to do group projects… so how do you structure those in a way that is meaningful… and provide opportunity for feedback
O1: …we’re talking about these skills in class but you’re not applying them … so then I’m like well I can’t not do a group project, so I try to put these mechanisms in place…
Structure and expectations. Participants reported challenges resulting from the overall structure (e.g., timing, limitations of class meetings or term length) of highly experiential learning activities, particularly with respect to students’ developmental readiness (e.g., students’ ability to mentor, coach, facilitate professional relationships with community partners) to participate and whether or not realistic expectations had been set.
S1: It was all logistical stuff and poor planning, great idea poor execution.
L1: I expected the students to journal and they had really superficial contributions… and I remember trying to make mid-course corrections and none of them worked. …whether it was giving them strict prompts, whether it was giving it open ended just write what you feel like writing. I think because the expectations weren’t clearly stated upfront, it didn’t work.
R2: I either expected too much or expected too little of my students…
L1: I tried to get them to create a training, like a skill training workshop. And I think they couldn’t do it because they didn’t know enough, and I thought they did.
Discussion
The purpose of this mixed methods sequential explanatory study was to identify the best practices of leadership educators’ instructional and assessment strategy use. In the quantitative phase, discussion-based pedagogies, case studies, and group projects/presentations were found to be used most frequently and that instructors avoided tests, quizzes, role-play, simulation, and games. The qualitative follow up revealed explanatory data related to instructors’ reasoning for choosing particular instructional or assessment strategies over others and offered evidence from their own experiences of factors that contributed to exemplar as well as chaotic learning activities. Notably, the mixed methods approach has been seldom used to study leadership education.
Factors Related to Instructional and Assessment Strategy Use. The primary findings from this study was that the data from the qualitative phase confirmed the results from the quantitative phase. In particular, the instructional and assessment strategies emphasized in the quantitative phase we confirmed and expounded upon in the qualitative phase. This is especially confirmatory since the interview protocol questions did not identify specific instructional or assessment strategies as benchmarks.
The integration of results from the quantitative and qualitative phases of this study suggest that the “best” leadership educators are very intentional in their instructional and assessment strategy choices. For example, discussion-based pedagogies were a clear favorite in both phases of the study and participants in the qualitative phase had no trouble sharing illustrative examples of when they used discussion, what forms of discussion they used (e.g., small group, interactive) and why they used it. Similarly, group work was integrative in that it was used to both provide multiple perspectives for learning and for applying course content—oftentimes through project- or problem-based learning or creative activity—as well as an opportunity to build leadership skills such as working with others toward a common goal. Nonetheless, participants did note the myriad challenges associated with TBL and sought out strategies to proactively address them. Comparably, reflection was more metacognitive in nature, whether students were jotting down reactions to self-assessments or refining personal vision statements, participants stressed the constructivist, pragmatic, experiential, and meaning making components of these learning activities.
Equally, study participants were intentional in avoiding particular instructional and assessment strategies. For example, while none flat out rejected the criticality of leadership theories, models, and concepts, they did feel strongly that tests and quizzes were not the right assessment strategies to measure students’ learning of such material. Consequently, applied and experiential methods for learning critical content were preferred. Likewise, participants avoided lecture and passive media to deliver course content for the same reasons.
Interestingly to this researcher was the confirmation of avoidance of the highly experiential instructional strategies role-play, simulation, and games. While their value is chronicled across the college teaching (Cherney, 2008; Stevens, 2015; Svinicki & McKeachie, 2014) and leadership education literature (Guthrie & Jenkins, 2018; Jenkins & Cutchens, 2012), instructors remain stubborn in their perceptions of either the cost or challenges associated with preparing for and facilitating these learning activities. Relatedly, the logistics of many learning activities in these realms leave much to factors outside the instructors’ control such as students’ motivation to participate, attendance, cost of materials, and developmental readiness. Accordingly, this researcher echoes Jenkins’ (2012) and Jenkins and Cutchens’ (2011) declarations that there ought to be more workshops on best practices in leadership education, particularly those related he use of critical reflection and experiential learning.
Implications and Recommendations
Perhaps the most noteworthy quantitative finding was that discussion emerged as the signature pedagogy for both UL and GL leadership educators (Jenkins, 2012, 2018). And while discussion-based pedagogies, case studies, and group projects/presentations—among others—were used most frequently by leadership educators and test and quizzes, simulations, and games were used least, the quantitative findings left the researcher craving additional explanation. Accordingly, the qualitative analysis focused on leadership educators’ motivations for using one or more pedagogical strategy more or less as well as the decision processes they utilized. Arguably, the most effective leadership educators were intentional in their pedagogical choices and focused on deep-level and relevant—aligning with students’ contexts or developing specific skills—learning activities and outcomes. Moreover, these leadership educators were constructivist in their pedagogical approaches, providing opportunities through creative and reflective learning activities for students to develop and make meaning of their experiences.
Leadership educators can benefit by understanding the process of intentional leadership program design (see Jenkins & Allen, 2017) and understanding that effective leadership education is a complex and dynamic process, which goes beyond merely talking about leadership or assessing students’ content knowledge recall. Program administrators and future research should consider the factors related to leadership educators’ professional development and academic credentialing. Successful programs may include learning outcomes related to teaching and learning, course and curriculum design, assessment, and facilitation skills.
Future Research. Still, there is limited research on the instructional strategy choices of leadership educators. Future research in this area should look beyond face-to-face instruction and include both blended and online methods of teaching and learning. Additionally, while the quantitative phase of this study included an international sample, the qualitative phase was limited to this study was 13 leadership educators from four states and 11 universities, nine of which were associated with institutions in Ohio or Illinois. Future research in this area should look to expand across the U.S. and abroad and include a more diverse group of participants in the qualitative phase. Additionally, future research may include the teaching habits of non-academic leadership educators, that is, those individuals in the student affairs professions who also facilitate leadership programs, albeit non-credit bearing.
Conclusion
This study provided only two perspective of leadership educators’ instructional and assessment strategy choices—that of the survey participants who opted to participate in quantitative phase and the interviewees in the qualitative phase who were recommended by their peers as exemplary. Being the only research on leadership educators’ instructional and assessment strategy choices, this study leaves some unanswered questions and opens a door for future research of leadership education program design. In addition to the recommendations above, perhaps in-depth exploration and/or observation of leadership educators in their instructional spaces (e.g., classrooms, service learning) may provide even richer data. The results would be productive for leadership educators and program administrators.
References
Allen, S. J., & Hartman, N. S. (2009). Sources of learning in student leadership development programming. Journal of Leadership Studies, 3(3), 6-16. https://doi.org/10.1002/jls.20119
Cherney, I. D. (2008). The effects of active learning on students’ memories for course content. Active Learning in Higher Education, 9(2), 152–171. https://doi.org/10.1177/1469787408090841
Creswell, J. W., & Plano Clark, V. L. (2018). Designing and conducting mixed methods research. (3rd ed.). Thousand Oaks, CA: SAGE Publications, Inc.
Creswell, J. W., & Poth, C. N. (2018). Quality inquiry & research design: Choosing among five approaches. Thousand Oaks, CA: Sage Publications, Inc.
Eich, D. (2008). A grounded theory of high-quality leadership programs: Perspectives from student leadership development programs in higher education. Journal of Leadership & Organizational Studies, 15(2), 176-187. https://doi.org/10.1177/1548051808324099
Guthrie, K. L., & Jenkins, D. M. (2018). The role of leadership educators: Transforming learning. Charlotte, NC: Information Age Publishing.
International Leadership Association. (2019). Directory of leadership programs. Retrieved from http://www.ilanet.org/Resources/LPD/index.htm
Jenkins, D. M. (2018). Comparing instructional and assessment strategy use in graduate- and undergraduate-level leadership studies: A global study. Journal of Leadership Education, 17(1), 75-92. Retrieved from http://journalofleadershiped.org/attachments/article/532/0647_jenkins.pdf
Jenkins, D. M. (2019). Exploring the lived experiences of becoming and being a leadership educator: A phenomenological inquiry. Journal of Leadership Education, 18(3), 141-157. Retrieved from https://journalofleadershiped.org/wp-content/uploads/2019/06/18_3_Jenkins.pdf
Jenkins, D. M. (2016). Teaching leadership online: An exploratory study of instructional and assessment strategy use. Journal of Leadership Education, 15(2), 129-149. Retrieved from http://journalofleadershiped.org/attachments/article/431/v15i2jenkins0462.pdf
Jenkins, D. M. (2013). Exploring instructional strategies in student leadership development programming. Journal of Leadership Studies, 6(4), 48-62.
Jenkins, D. M. (2012). Exploring signature pedagogies in undergraduate leadership education. Journal of Leadership Education, 11(1), 1-27. Retrieved from http://www.journalofleadershiped.org/attachments/article/109/Jenkins.pdf
Jenkins, D. M., & Allen, S. A. (2017). It’s all about intentionality: Aligning instructional strategies with learning outcomes and leadership competencies. New Directions for Student Leadership, 153, 43-58. https://doi.org/10.1002/yd.20270
Jenkins, D. M., & Cutchens, A. B. (2012). From theory to practice: Facilitating innovative experiential leadership theory-based role play activities. Proceedings of the 2012 Association of Leadership Educators Conference, 256-262. Retrieved from http://leadershipeducators.org/resources/Documents/2012%20ALE%20Conference%20Proceedings.pdf
Jenkins, D. M., & Cutchens, A. B. (2011). Leading critically: A grounded theory of applied critical thinking in leadership studies. Journal of Leadership Education, 10(2), 1-21. Retrieved from http://www.journalofleadershiped.org/attachments/article/119/Jenkins%20and%20Cutchens.pdf
Moustakas, C. (1994). Phenomenological research methods. Thousand Oaks, CA: Sage Publications, Inc.
Priest, K. L., & Seemiller, C. (2018). Past experiences, present beliefs, future practices: Using narratives to re(present) leadership educator identity. Journal of Leadership Education, 17(1), 93-113. Retrieved from http://journalofleadershiped.org/attachments/article/533/0650_priest.pdf
Seemiller, C., & Priest, K. L. (2017). Leadership educator journeys: Expanding a model of leadership educator professional identity development. Journal of Leadership Education, 16(2), 1-22. Retrieved from http://journalofleadershiped.org/attachments/article/493/0583_priest.pdf
Seemiller, C., & Priest, K. L. (2015). The hidden ‘who’ in leadership education: Conceptualizing leadership educator professional identity development. Journal of Leadership Education, 14(3), 132-151. Retrieved from http://journalofleadershiped.org/attachments/article/390/2015_0334_priest.pdf
Stevens, R. (2015). Role-play and student engagement: Reflections from the classroom. Teaching in Higher Education, 20(5), 481–492. https://doi.org/10.1080/13562517.2015.1020778
Svinicki, M. D., & McKeachie, W. J. (2014). McKeachie’s teaching tips: Strategies, research, and theory for college and university teachers (14th ed.). Belmont, CA: Wadsworth Cengage Learning.
Zimmerman-Oster, K., & Burkhardt, J. C. (1999). Leadership in the making: Impact and insights from leadership development programs in U. S. colleges and universities. Battle Creek, MI: W. K. Kellogg Foundation.