Ira H. Martin, Laurel Goulet, Jenny Mae K. Martin, Jake Owens DOI: 10.12806/V14/I4/A1
The Use of A Formative Assessment In Progressive Leader Development
Ira H. Martin
Captain John A. Henriques Leadership Scholar Institute for Leadership
US Coast Guard Academy New London, CT
Professor, Department of Management US Coast Guard Academy
New London, CT
JennyMae K. Martin
Student Department of Psychology
University of New Hampshire Durham, NH
Student Department of Mathematics University of Massachusetts
As institutions continue to place value on developing leaders, it becomes increasingly important to effectively assess students’ leadership skills. The development and subsequent use of a formative competency based leader assessment was used with (N=124) sophomore students at a small military college in the Northeast United States with a mission toward leadership. Results highlight the effectiveness of a formative assessment to develop leader skills with an undergraduate population. Relevant application for institutions of higher learning will be discussed.
Educational institutions place high value on developing undergraduate student leaders for a range of future occupational endeavors (Pearce & Conger, 2002; Posner, 2004; Smart, Ethington, Riggs, & Thompson, 2002). With many colleges and universities expressing a commitment to the development of student leaders, mission and vision statements have begun to reference students practicing, contributing, or enhancing skills as leaders in one form or another.
Further, many institutions have created training and development programs in and out of the classroom to develop leader skills (Densten & Gray, 2001; Komives, Dugan, Owen, Slack, & Wagner, 2011; Scroggs, Sattler, & McMillian, 2009; Smart, et al., 2002).
Leader development has become further tied to institutions’ mission and vision development for their students, however the rigorous and systematic assessment of student leader development has trailed behind (Lindsay, Foster, Jackson, & Hassan, 2009; Rosch & Schwartz, 2009). Higher education leadership programs are under greater examination (Blackwell, Cummins, Townsend, & Cummings, 2007), and increased need to legitimize themselves (Ritch, 2013). Leadership educators need to heed the public calls for more accountability and continuous improvement in higher education – including leadership development programs and leadership education (Goertzen, 2013). With this increased public scrutiny and calls for higher education accountability, it is more vital than ever that leadership development programs are effective and impactful for students (Goertzen, 2009). Yet, in this new age of accountability, there is a dearth of research on assessment of leadership learning in academic settings (Goertzen, 2013).
In the current study, the use of formative assessment, where student learning is assessed (meeting general accountability needs), and feedback is given to aid in student development is proposed (Evans, Zeun, & Stanier, 2014). Over the course of the manuscript a unique way to use leader competency behaviors in conjunction with a formative assessment is presented. Results of a unique approach to student development through formative assessment are discussed with recommendations for future use.
A Progressive Approach to Student Development. A progressive educational experience emphasizes an avenue of creativity, a hands-on approach to learning that incorporates student led discovery, self-awareness, and personalized education for each student’s individual goals of learning (Hayes, 2008; Little, 2013). Traditional educational experiences emphasize extrinsic motivators such as grades, testing, and comparison to a standard or benchmark, where direct instruction, lecture, and seatwork are the cornerstone of this type of learning. It has been found that students who are engaged in progressive learning strategies are more intrinsically motivated to learn, while those experiencing traditional learning approaches are less likely to engage in critical thinking and self-reflection (Hayes, 2008; Mintz, 2012).
In a truly progressive approach to learning, the educator acts as a learning partner, inspirer, and mentor, while students expand their perspectives, transforming from passive participants to performers and co-learners (Horng, Hong, ChanLin, Chang, & Chu, 2005). With an emphasis on lifelong learning, progressive strategies incorporate skills that will be important not only in the present, but also in the future. Evolving into a leader is certainly a present and future oriented task, with a distinct emphasis on lifelong learning and perpetual development (Little, 2013; Stewart, 2015).
A leader development curriculum incorporating innovative new methods of learning has the potential to be advantageous to student development (Scott, Whiddon, Brown, & Weeks, 2015; Williams & McClure, 2010). Research indicates that it is best to combine learning and practice (performance) for meaningful leadership development to occur (e.g., Lindsay, et al.,
2009). Conger (2013) warns that the impact of leadership education is limited when educators don’t include opportunities for students to apply the material learned, in a realistic fashion.
In many ways, this focus on application of leadership learning leads to consideration of leadership competencies and behaviors. Leadership competencies are important in the leadership development context because unlike innate leadership traits, competencies can be developed through practice (Seemiller & Murray, 2013). Although discussing leadership training, Allio (2005) makes an interesting division between leadership literacy and leadership competency. In order for leadership development to occur, students must practice leadership by “performing deliberate acts of leadership” (p. 1071). These deliberate acts of leadership are an integral part of the progressive educational experience. By using educators as facilitators, rather than lecturers, students can participate freely in their self-development, for the freedom to enhance competencies is granted (Horng, et al., 2005).
Reflection. One progressive technique used to transition students from a passive participant in the classroom to an active learner, is the use of structured reflection. Reflection teaches individuals to look at issues for various, unique perspectives. The process of critically thinking about behaviors, attitudes, and beliefs, while integrating theory with practice stimulates self-discovery is an important part of learning (Jefferson, Martin, & Owens, 2014; Mezirow, 1998; Roberts, 2008). This kind of progressive educational pursuit guides an emerging leader to own their values and beliefs, as well as commit to a personal stance on critical issues (Avolio & Gardner, 2005; Scott et al., 2015). Self-discovery must be developed by any great leader; through self-awareness, acceptance, and critical thinking (Gardner, Avolio, Luthans, May, & Walumbwa, 2005; Scott, et al., 2015).
A leader development program that fosters personal integration of theory and practice over time has shown to be effective (Connaughton, Lawrence, & Reuben, 2003; Doh, 2003; Kayes, 2002; Townsend, Linder, & Williams, 2005). The capacity to reflect relates to how effectively individuals can learn from their personal experiences and reflection provides a meaningful way for leaders to gain genuine understanding.
By using a reflective assessment to develop leaders in and out of the classroom, institutions and educators may find success in pushing their students’ learning forward.
Leadership Assessment. While the main focus of leadership educators is on leadership development (and rightly so), it behooves us to attend to the calls for greater accountability and to assess the impact of courses and programs on leadership development. Assessment of academic leadership programs happens at one of three levels: assignment, course, and program (Goertzen, 2013). It is important to determine not only where leadership assessment occurs (assignment, course, or program), but also what this leadership assessment measures.
Determining which leadership outcomes are assessed is of paramount importance (assessing learning only, leadership development, individual leadership performance, or some combination of the three) (Lindsay, et al., 2009, Rosch & Schwartz, 2009). While some scholars lament the lack of universal leadership outcomes, Goertzen (2009) argues that each institution or leadership program should develop mission-driven leadership outcomes. He advocates that each program should maintain not only its unique mission, but also its own unique characteristics such as
experiences and curricular content. Researchers advocate for assessing application of leadership learning, specifically in terms of leadership behaviors (e.g., Lindsay, et al., 2009; Rosch & Schwartz, 2009).
There are various methods of leadership assessment, including standardized and locally developed tests, portfolios, and focus groups (Goertzen, 2009). Historically, direct measures (such as tests) are considered better than indirect measures (e.g., self-report of learning) of leadership development. However, self-report of behavioral outcomes are valuable for leadership programs that seek to develop leadership behaviors (Goertzen, 2009). Many of these self-report measures are criticized for using proxies of leadership such as volunteerism or voting.
Formative Assessment. One way to mitigate the problems of leadership development assessment and adhere to the tenets of progressive educational experience is to employ formative assessment. Generally, formative assessment provides feedback to students on their performance, and this feedback aids the students in attaining higher levels of learning (Evans, et al., 2014; Yorke, 2003). Formative assessment differs from summative assessment; summative assessment focuses merely on whether students have accomplished the learning outcomes, but provides no further learning opportunity for students to improve (Yorke, 2003). It is important to note that well-constructed assessments may be both summative and formative, in that they provide assessment information to assessors, while simultaneously providing learning feedback to students (Yorke, 2003). Considered essential to good teaching practice, formative assessment has been found to significantly improve student learning (Hollingworth, 2012).
Because formative assessment delivers data to students upon which they can act (Blitz, Salisbury, & Kelley, 2014), it is well suited to leadership development programs. The benefits of formative assessment for such programs are multidimensional. Formative assessment provides: 1) assessment data for both external examiners as well as for program improvement, 2) feedback to students on their leadership performance, and 3) an opportunity for students to reflect on that feedback in order to improve leadership performance.
Participants. Participants were 124 of the 130 sophomore students enrolled in an Organizational Behavior and Leadership (OB&L) course at a small military college in the Northeast United States. OB&L is a core leadership course across the academic curriculum intended to help students lead themselves and others, where students enrolled across three instructors were asked to voluntarily participate. Of the 124 students, (n=20) were enrolled with instructor one, (n=22) with instructor two, and (n=82) with instructor three. Of the sample size, (n=78) or 63% were male, while (n=46) or 37% were female.
Sophomore/Role Modeler Leader Development Feedback
The primary focus of a sophomore student is to be a role model for the freshman students while internalizing key followership and leadership principles of the institution/organization
Please rate the frequency to which each question below corresponds with student leader performance: 1=Never, 2=Rarely, 3=Regularly, 4=Frequently, 5=Always
Accountability & Responsibility
Recognizes the impact of personal behavior and accepts responsibility for personal performance and the performance of others.
1 2 3 4 5 NA
Aligns personal values with class core values to model for others.
1 2 3 4 5 NA
Looks to leadership for guidance and feedback through active listening and asking questions when appropriate.
1 2 3 4 5 NA
Health & Well Being
Maintains personal health and well-being and assists others in maintaining a healthy lifestyle.
1 2 3 4 5 NA
Self Awareness & Learning
Recognizes own strengths and weaknesses and pursues self-development.
1 2 3 4 5 NA
Recognizes the impact of personal behavior and conduct on others.
1 2 3 4 5 NA
Knows classroom and personal roles in fulfilling mission tasks for self and others.
1 2 3 4 5 NA
Applies basic conventions of cadet communication with self and others both orally and in writing.
1 2 3 4 5 NA
Is open to views of others; works in collaborative, inclusive, outcome oriented manner with supervisors and others.
1 2 3 4 5 NA
Models a respectful understanding of others’ positions, while gaining the ability to achieve cooperation by giving cooperation.
1 2 3 4 5 NA
Recognizes own strengths and weaknesses and uses mentors for development.
1 2 3 4 5 NA
Respect for Others & Diversity Management
Treats all individuals fairly in compliance with class policies.
1 2 3 4 5 NA
Taking Care of People
Takes appropriate action to safeguard the welfare of others.
1 2 3 4 5 NA
Figure 1. Leader Development Feedback Assessment (LDFA).
Measure. Leader Development Feedback Assessment (LDFA). Student leader
development performance scores were assessed on a 5-point Likert scale (1 = never to 5= always). The LDFA consists of 13-questions with respondents rating the degree to which items correspond to their leader performance. The scale assesses the thirteen factors associated with the institution’s leader development framework of leading self and others (Commandant Instruction, 2006). Leading self corresponds to the competencies of Accountability & Responsibility, Aligning Values, Followership, Health & Well Being, Self Awareness & Learning, Personal Conduct, and Technical proficiency. Leading others corresponds to the competencies of Effective Communication, Team Building, Influencing Others, Mentoring, Respect for Others & Diversity Management, and Taking Care of People. All 13 competencies correspond with a one sentence written question for students to self-assess their current leader development performance with regard to their given role as a freshman (follower), sophomore (role-modeler), junior (mentor), or senior (leader). An example item for followership reads as such: (looks to leadership for guidance and feedback through active listening andasking questions when appropriate.) For further information on assessment See Figure 1.
Procedure. With Institutional Review Board Approval, students were recruited in their core leadership course (Organizational Behavior and Leadership). After enrollment, participants were asked to complete a LDFA during the first day of their OB&L class and during their last day of their OB&L class. It was made clear that participation was voluntary and that participants could withdraw at any time. Participation took 10-15 minutes.
To understand the nature of leadership competency development with this sample of students, descriptive analyses and a paired sample t-test was conducted on both pre and post responses (See Table 1). Mean scores for the LDFA at Time 1 range from a mean score of 3.41 (regularly performing) on the competency of mentoring, to a mean score of 4.08 (frequently performing) on the competency of taking care of people. Lower scoring competencies at Time 1 were Effective Communications (3.41) and Followership (3.50). Higher scores at Time 1 were Taking Care of People (4.08), Respect for Others (4.07), and Health & Well Being (3.89). In regard to mean Time 2 scores for the LDFA, a range of 3.94 (frequently performing) to 4.48 (between frequently and always performing) was present. Low means at Time 2 were Technical Proficiency (3.94) and Effective Communications (3.96), while higher mean scores corresponded with Respect for Others (4.48) and Taking Care of People (4.43).
Leader Development Feedback Assessment with Student Descriptives and Instructor ANOVA
Instructors with the same letter are not significantly different.
A paired sample t-test was implemented to assess statistical differences between Time 1 and Time 2 mean scores. The t-statistic showed significance at the p <0.01 level between all Time 1 and Time 2 competency scores. Regarding mean differences between Time 1 and Time 2, Mentoring (0.67), Followership (0.62), and Influencing Others (0.59) saw the greatest improvement. The competencies that saw the least improvement in mean scores were Team Building (0.29), Taking Care of People (0.35), and Health & Well Being (0.36). Although several competencies at Time 1 (Taking Care of People, Respect for Others) had the highest mean scores, they also had some of the lowest mean differences (0.35), highlighting a smaller overall change. Similarly, Technical Proficiency at Time 1 (3.56) and Time 2 (3.94) showed relatively low mean scores across highlighting one of the lowest mean differences (0.38)
A one-way ANOVA was conducted to determine if mean differences exist across Time 1 and Time 2 scores with regard to three distinct instructors. Given the unequal sample sizes across the three instructors, a post-hoc Scheffe test was conducted. For the Time 2 scores, no significant difference among instructor mean scores was found. However, statistical significance among instructor mean scores occurred in select Time 1 scores. Aligning Values showed a mean difference between instructor 3 (3.7) and instructor 1 (3.1) in student scores F(2, 120)=4.90, p<0.05. Followership showed a significant difference between instructor 3 (3.63) and instructor 1 (2.9) in student scores F(2, 121)=4.36, p<0.05. Additionally, instructor 2 (3.55) and 1 (2.9) saw a significant difference in Followership. Lastly, Personal Conduct showed a mean difference between instructor 2 (4.04) and instructor 1 (3.3) in student scores F(2, 121)=2.92, p<0.05.
A progressive educational experience has the power to accentuate student led discovery, self-awareness, and development of learning (Hayes, 2008). In this investigation, understanding of student leader development using a formative assessment over the course of a semester was deepened. Specifically, results of this study showed the powerful effects of a formative assessment with undergraduate students. Competency score improvement over the course of a semester was found across various instructors. With regard to the institution’s core competencies, students’ strengths lie in community-oriented skills, while areas for improvement contain personal-oriented skills. In this study, all thirteen of the institutions core leadership competencies increased across all pre semester to post semester scores. Such statistical significance supports the notion of a progressive classroom experience with emerging leaders.
A progressive approach to learning that includes real-time feedback on leader behaviors within the context of a leadership emphasizes the power of this type of application in an educational setting. From a broader pedagogical lens, this study highlights the notion that variances between instructors did not effect student leadership development. Pre semester scores showed marginal differences among instructor scores, while no significant post semester difference among instructors was found. This finding may be of particular note in the academic setting where teaching practices are often quite distinct between instructors. The use of a formative assessment, where teachers were implementing an innovative learning style, and allowing the student to participate freely in their self-development, may have outweighed any instructor differences.
Lastly, this study suggests the continued development of particular competencies across this sample. Students’ strengths resided in community-oriented skills, where the highest scoring competencies for both pre and post semester were taking care of others, respect for others and diversity management, and health and well being. Such skills in an academic and military environment are often discussed, practiced, and experienced, and thus may allow a student initial strength in the given discipline. Conversely, areas for improvement were consistent with personal-oriented skills, where the lowest scoring competencies for pre and post semester corresponded with mentoring, effective communication, followership, and personal conduct.
Given this sample of sophomore level students, this finding may not be surprising given their potentially limited development and experiences with leadership.
Further, such personal-oriented skills showed the most upward change over the course of the semester. This finding solidifies the idea that student development in the personal skill domain of leadership, although trending upward, should still be considered in this population of students. In any academic or corporate setting where leadership is found to be imperative for operational effectiveness, it may be important to remember that success at both the individual and collective level is imperative. As leadership is a people-oriented skill, understanding oneself while understanding those around you should be a critical part of any intentional leader development program.
Reflection of Practitioner
Overall, this study shows promise for the use of a formative assessment in an undergraduate class, pre and post semester, for student leader development. An intentional leader development approach across various instructors within a classroom setting may be advantageous across other educational settings and should be applied. Allowing students to have an active role in their learning may be intrinsically motivating and constructive, and has demonstrated gains in leadership competencies and behaviors. Additionally, findings offer direction for leadership development training programs in the corporate sector. Those programs that incorporate progressive educational principles and the structured use of formative assessment may show greater advances in leadership development.
This study examined only the sophomore class of an Organizational Behavior and Leadership course. This combination of progressive educational experiences teamed with formative assessment would be useful in courses dedicated to the study of leadership, as well as other courses, which are not necessarily perpetuating leadership. In order for any assessment to target organizational leadership within an institution it is important to be aware of the core leadership competencies and or mission implemented at the institution (Goertzen, 2009). A pre/post study done over the course of more than a sole semester may be beneficial. Pre/post assessment gives an ample position for growth among students, yet, implementing a more diverse demographic across leadership programs and class ranks may be advantageous. Finally, further empirical research should be dedicated to the study and utilization of progressive educational experiences mixed with formative assessment in other settings. This study was conducted at an undergraduate military academy; civilian institutions or corporate organizations may find use in similar approaches to their leader development strategies.
Avolio, B. J., & Gardner, W. L. (2005). Authentic leadership development: Getting to the root of positive forms of leadership. The Leadership Quarterly, 16(3), 315-338.
Blackwell, C., Cummins, R., Townsend, C.D., & Cummings, S. (2007). Assessing perceived student leadership skill development in an academic leadership development program. Journal of Leadership Education, 6(1), 39-58.
Blitz, M.H., Salisbury, J., & Kelley, C. (2014). The role of cognitive validity testing in the development of CALL, the comprehensive assessment of leadership for learning. Journal of Educational Administration, 52(3), 358-378.
Conger, J. (2013). Mind the gaps: What limits the impact of leadership education. Journal of Leadership Studies, 6(4), 77-83.
Connaughton, S., Lawrence, F., & Ruben B. (2003). Leadership development as a systematic and multidisciplinary enterprise. Journal of Education for Business, 79(1), 46-52.
Densten, I. L., & Gray, J. H. (2001). Leadership development and reflection: what is the connection?. International Journal of Educational Management, 15(3), 119-124.
Doh, J. (2003). Can leadership be taught? Perspectives from management educators. Academy of Management Learning and Education, 2(1), 54-67.
Evans, D.J.R., Zeun, P., & Stanier, R.A. (2014). Motivating student learning using a formative assessment journey. Journal of Anatomy, 224(3), 296-303.
Gardner, W. L., Avolio, B. J., Luthans, F., May, D. R., & Walumbwa, F. (2005). “Can you see the real me?” A self-based model of authentic leader and follower development. The Leadership Quarterly, 16(3), 343-372.
Goertzen, B.J. (2013). Assessment adrift: Review of the current state of assessment of academically based leadership education programs. Journal of Leadership Studies, 6(3), 55-60.
Goertzen, B.J. (2009). Assessment in academic based leadership education programs. Journal of Leadership education, 8(1), 148-162.
Hayes, W. (2008). The future of progressive education. Educational Horizons, 86(3).
Hollingworth, L. (2012). Why leadership matters: Empowering teachers to implement formative assessment. Journal of Educational Administration, 50(3), 365-379.
Horng, J., Hong, J., ChanLin, L., Chang, S., Chu, H. (2005). Creative teachers and
creative teaching strategies. International Journal of Consumer Studies, 29(4), 352-358.
Jefferson, J., Martin, I., Owens, J. (2014). Leader development through reading and reflection. Journal of Leadership Studies, 8(2).
Kayes, D. (2002). Experiential learning and its critics: Preserving the role of experience in management learning and education. Academy of Management Learning and Education, 1(2), 137-149.
Komives, S. R., Dugan, J. P., Owen, J. E., Slack, C., Wagner, W., & Associates. (2011). The handbook for student leadership development. Jossey-Bass: National Clearinghouse for Leadership Programs, 2.
Lindsay, D.R., Foster, C.A., Jackson, R.J., & Hassan, A.M. (2009). Leadership education and assessment: A developmental approach. Journal of Leadership Education, 8(1), 163-176.
Little, T. (2013). 21st century learning and progressive education: An intersection.
International Journal of Progressive Education, 9(1).
Mezirow, J. (1998). On critical reflection. Adult Education Quarterly, 48(3), 185-199.
Mintz, A. I. (2012). The happy and suffering student? Rousseau’s emile and the path not taken in progressive educational thought. Educational Theory, 62(3).
Pearce, C. L., & Conger, J. A. (2002). Shared leadership: Reframing the hows and whys of leadership. Thousand Oaks, CA: Sage.
Posner, B. Z. (2004). A leadership development instrument for students: Updated. Journal of College Student Development, 45(4), 443-456.
Ritch, S. (2013). Formalized program review: An evolution. Journal of Leadership Studies, 6(3), 61-66.
Roberts, C. (2008). Developing Future Leaders: The Role of Reflection in the Classrom. Journal of Leadership Education, 7(1), 116-130.
Rosch, D.M. & Schwartz, L.M. (2009). Potential issues and pitfalls in outcomes assessment in leadership education. Journal of Leadership Education, 8(1), 177-194.
Scott, M., Whiddon, A., Brown, N., Weeks, P. (2015) The journey to authenticity: An
analysis of undergraduate personal development. Journal of Leadership Education, 14(2).
Scroggs, L. E., Sattler, J. L., & McMillan, M. B. (2009). The Undergraduate Leadership Mosaic: A Challenge of Shared Purpose, 8(1), 48-58.
Seemiller, C. & Murray, T. (2013). The common language of leadership. Journal of Leadership Studies, 7(1), 33-45.
Smart, J. C., Ethington, C. A., Riggs, R. O., & Thompson, M. D. (2002). Influences of institutional expenditure patterns on the development of students’ leadership competencies. Research in higher education, 43(1), 115-132.
Thousand Oaks, CA: Sage.
Stewart, P. (2015). The progressive teaching initiative: A cohesive model for school transformation. Educational Digest: Essential Readings Condensed for Quick Review, 78(2).
Townsend, C., Linder, J., & Williams, J. (2005). Teaching leadership: Do students remember and utilize the concepts that we teach? Journal of Leadership Education, 4(1), 62-74.
US Coast Guard Leadership Framework, COMDTINST 5351.1.
Williams, J. R., & McClure, M. (2010). The effects of teaching methods in leadership knowledge retention: An experimental design of lecture, experiential, and public pedagogy. Journal of Leadership Education, 9(2), 86-100.
Yorke, M. (2003). Formative assessment in higher education: Moves toward theory and the enhancement of pedagogic practice. Higher Education, 45(4), 477-501.