Does Providing Feedback to Student Reflections Impact the Development of Their Leadership Competence?
Nicole L.P. Stedman NStedman@aged.tamu.edu
Tracy A. Rutherford TRutherford@aged.tamu.edu
T. Grady Roberts GRoberts@aged.tamu.edu
107 Scoates Hall
TAMU 2116
Texas A&M University College Station, TX 77843-2116
P: 979-458-1213
Abstract
Internship experience is a valuable component of an undergraduate degree. This is especially true in leadership education programs, where leadership development may take place in a variety of contexts. Theory purports reflection enhances a learners’ experience through a linkage of education, work, and personal development (Kolb, 1984). It is not clear, however, if reflection guided by feedback will enhance or diminish the learning and development. This study utilized a pretest-posttest experimental design in an attempt to determine if providing feedback to weekly internship reflections would make a difference in leadership skill development. Thirty-six undergraduate students were randomly assigned to treatment and control groups following enrollment in a 10-week summer internship course. All participants submitted weekly reflections. The control group received no feedback and the treatment group was provided feedback by a research team member to invoke deeper reflection and development of leadership skills. Leadership skill development was measured using the Leadership Skills Inventory-Self©. Results of the study did not yield statistically significant differences between the two groups, but did demonstrate observable
differences in the mean scores. Replication of this study is recommended utilizing quantitative and qualitative measures to further understand this phenomenon.
Introduction and Theoretical Framework
The professional internship experience provides leadership students a glimpse of their future and provides them an opportunity to practice their new leadership knowledge. Internships have shown to be an effective means for bridging academic courses with future careers by allowing students to explore competencies required by employers. Densten and Gray (2001) explained leadership draws from numerous fields and real life sources. This creates a need for the integration of knowledge with experience and through internships; the practical nature of leadership is encouraged and developed by providing students with real hands-on experience.
In leadership education programs, it becomes important for instructors to assist in steering students in positive career directions. Professional internships provide a means for exposing students to a wide variety of career interests (Morgan, Rudd, & Kaufman, 2004). For this reason, the professional internship becomes an even more integral part of the educational experience. Morgan et al. (2004) found agricultural leadership faculty supported professional internships as part of an agricultural leadership curriculum. Schumacher and Swan (1993, p.8) recommended to faculty in Colleges of Agriculture to, “overtly encourage advisees to participate in activities foster leadership skill development.” The Hart Leadership Program (HLP) at Duke purports, “The best way to learn is to combine academic study with direct experience” (Hart Leadership Program, 2006,
p. 5). Furthermore, the HLP encourages student in the area of critical reflection, enhancing the student’s ability to, “make sense of their experiences…” (Hart Leadership Program, 2006, p. 10).
Accordingly, faculty at a southern land-grant university designed a professional internship to provide a practical, hands-on experience fostering the development of leadership skills in students seeking leadership degrees. The purpose of this experience was to increase students’ interaction with leadership competencies reflective of their coursework in an undergraduate leadership program.
There were eight independent competencies identified. These competencies included communication, diversity, envisioning, mentoring, professionalism and ethics, problem solving and critical thinking, teamwork, and working with change. These competencies were operationalized by defining specific measurable skills. The first competence identified was communication and included active listening, oral communication, presentation skills, technology, and
writing skills. Diversity included cultural appreciation and cultural competence. Envisioning included creativity, team development, strategic planning and risk taking. Mentoring included career development skills, coaching skills, self- awareness, and staff development. Professionalism and ethics included personal and professional ethics and personal responsibility. Problem solving included critical thinking, creativity, consensus building, and personal judgment.
Teamwork included collaboration, committee leadership, conflict management and resolution, delegation and evaluation. Lastly, working with change included commitment, creativity, motivation and stress management. The eight competencies were adopted from a Delphi study completed by members of the Association of Leadership Educators (2000). The intent of the Delphi was to identify core leadership competencies for organizations and communities.
Because the purpose of the leadership internship was to develop students’ leadership abilities in organizations, the internship coordinator deemed these eight competencies an appropriate fit.
A supporting theoretical model provided a means for selecting appropriate instrumentation. The model, Transforming Leadership by Anderson (1998), contained 56 skills in six competency areas. These were self-management; interpersonal communication; coaching, counseling and problem management; consulting; and versatility and organizational development. Figure 1 depicts the analogous relationship between Anderson’s Transforming Leadership and the leadership competencies. Synthesis of the Transforming Leadership competencies and skills revealed the comparableness to the leadership competencies.
Figure 1. Transformational Leadership and Leadership Curriculum Comparable Competencies
Anderson’s Transformational Leadership Curriculum
Leadership
Interpersonal Communication Communication
Self-Management Envisioning
Coaching and Counseling Mentoring
Problem Management Problem Solving
Consulting Teamwork
Versatility and
Organizational Development Working with Change
Transforming Leadership Principles
Diversity, Professionalism, and Ethics
The following definitions were those provided by the Consulting Resource Group (CRG) in the LSI-I (CRG, 2005). The communication competency identified in the leadership internship correlates to interpersonal communication in the Anderson model. In the LSI-I CRG defines interpersonal communication as the, vehicles through which all interactions between you and other people are made more clear and effective” (p.6). Self-management was likened to envisioning due to the definition provided by CRG. CRG defined self-management as the section which will, “assist you to gain a more in-depth understanding of the skills you will need to have greater positive impact in the process of developing yourself and others” (p. 5). Mentoring was associated with coaching and counseling.
However, the coaching and counseling section also included problem management, but the researchers concluded this competence more closely aligning with the competencies of problem management. With that, CRG defined coaching and counseling as, “process skills required by Transformational Leaders wanting to implement and support change in others. They are critical in moving individuals toward greater self-understanding, self-responsibility and performance” (p.7). There were specific questions identified by the researchers which were more closely aligned with problem solving; namely, problem exploration, problem specification, and problem ownership (p. 7). The competency of consulting identified by CRG included, “skills and processes requiring sophisticated responses and abilities for team group environments.
These skills are required to successfully develop teams and groups” (p. 8). These were seen as aligning with teamwork for those reasons. CRG operationalized versatility and organizational development as, “skills required for complex adaptation to rapid change in the social and technical environments” (p. 9).
Researchers saw this very closely relating to the competence of working with change. Lastly, CRG used an overarching competence skill set of transforming leadership principles. Evaluation of the questions in this section confirmed it represented two of the internship competencies, diversity, and professionalism and ethics.
Competency development was determined through the completion of 13 set criteria, or expectations. Criteria were designed to enhance the overall internship experience. Each student was expected to provide a statement of personal learning goals; personal vision/mission statement; statement of goals, responsibilities, and roles; weekly reflections on the internship experience; representative work samples; reflections on the eight core leadership competencies; special internship project; honors and recognitions received; evaluations of work; job descriptions; copy of trade journal; statement of work style; philosophy, and personal goals; and an analysis of the internship learning experience. In order to further support students in their internship experiences, faculty chose to focus on a model of
experiential learning with emphasis on the weekly reflections of the internship experience.
Expanding and revising on the work of other experiential education theorists, Roberts (2006) proposed in the most simplest of terms, experiential education contains an element of experience, reflection and generalization. From this first iteration, a student may elect to continue the cycle with additional iterations of the experience. Experiential learning, as described by Kolb (1984), is a process linking education, work, and personal development. Dewey (1938) postulated learning occurs through a cycle of action and reflection. The experiences provided through academics enhance students’ understanding and understanding precedes action (Eyler & Giles, 1999). Kolb’s model, derived from Dewey (1938), Lewin (1958), and Piaget (1970), is a cyclical model with four key points. The initial point is the concrete experience followed by reflective observations, abstract conceptualizations and active experimentation (Kolb, 1984). Each point in the cycle provides a unique feature to the learner’s experience.
In this study, the researchers directly link the personal internship with the “experience” phase of the cycle, the reflective observations with the “reflection” phase and then conclude the cycle with the link between identified leadership knowledge and “generalizations.” Specifically, the interns sought to continue the cyclical nature of the experience by returning to the internship experience, submitting weekly reflections, and utilizing their textbook knowledge of leadership competencies. The literature supports the internship as a means for creating a concrete experience in applying the principles of an academic program (Morgan et al., 2004), but little emphasis has been placed on the role of reflection in leadership (Ollila, 2000). Ollila stated, “reflection is important for project leaders since it helps them make sense of uncertainties, understand the behaviour of others, and improve as leaders” (p.197).
Authors consistently highlight the role of the instructor in the literature when discussing methods and strategies for enhancing student reflection (Boud, Keogh, & Walker, 1985; Knapp, 2001). Kolb (1984) indicated learners must have the opportunity to reflect on and observe experiences from many perspectives and instructors are a source of perspective available to students. Boud et al. (1985) encouraged instructors to use six strategies: a) ask for a detailed account of what happened; b) draw out events may have gone unnoticed; c) draw attention to overlooked interpretations; d) avoid offering their own interpretations; e) encourage learners to be aware of feelings and f) give free and undivided attention to learners. Knapp (2001) expanded this by promoting the use of carefully planned and guided reflections. Densten and Gray (2001, p.119) inferred from
Boud et al., “reflection provides a meaningful way for leaders to gain genuine understanding.” Densten and Gray (2001) explained further the role of reflection in leadership development emphasizes the deliberate examination of thoughts and experiences providing students with a deeper understanding of the, “leadership and learning episodes they experience.” However, in the current literature there is little empirical evidence documenting the impact of various reflection methodologies.
Purpose
Students enrolled in a southern land-grant university had the opportunity to enroll in a 10-week summer internship to practice and develop their leadership skills and competence. These students were admitted to an internship program pending enrollment requirements. One stipulation of the internship experience was students submit a weekly reflection of their experiences. The purpose of this study was to determine if providing feedback to weekly reflections affected leadership competence and skill development. One null hypothesis guided this inquiry.
Ho1: Providing feedback to weekly reflections makes no difference in leadership competence and skill development.
Method
The researchers used a pretest-posttest control group experimental design (Gall, Gall, & Borg, 2003) to test the null hypothesis. As noted by Gall et al. (2003), this design controls for all eight threats to internal validity (history, maturation, testing, instrumentation, statistical regression, differential selection, experimental mortality, and selection-maturation interaction) presented by Campbell and Stanley (1963). However, a potential threat to external validity in this design was the interaction between the pretest and experimental treatment. The research team hypothesized because the participants were already familiar with leadership principles exposure to questions regarding leadership on the pretest had no effect on the experimental treatment.
The population was all leadership interns at a southern land grant university. The accessible sample was those students who enrolled in internship credit during the 2005 summer semester (n = 36). The internship coordinator required pre-approval of students’ site selection for their internships and required each student to submit documentation as to the location of their internship, proposed goals, and internship responsibilities with their supervisor signing-off on the documents.
This ensured consistency in the types of responsibilities assigned to interns,
although the internship coordinator placed no stipulations on to the type of organization the student chose. Students were randomly assigned into either the control group with no feedback (n = 18) or treatment group with feedback (n = 18).
The variance in type of organization interns selected is a limitation. The researchers, nor internship coordinator, attempted to control for the specific organization or type of industry selected by the students included in the study.
One component of the internship requirement was weekly reflections. Participants were required to submit weekly reflections to the internship coordinator. Email was the preferred submission method, but researchers provided students without Internet access postage-paid envelopes to submit their reflections. Reflections from students in the treatment group were forwarded to the research team. To balance the workload of the research team and ensure rich interaction with students in the treatment group, the treatment group (n = 18) were further randomly assigned to three sub-groups, with one research team member interacting with each group (n = 6). The respective research team member responded to each weekly reflection submitted by participants in the treatment group, with the goal of invoking deeper reflection and further development of leadership competencies.
The researchers responding to leadership intern reflections were purposive in their responses. Frequent contact between all three of the researchers ensured responses were uniform, engaging, and challenging. Researcher responses included queues to the student. Researchers intended the queues to further challenge students to seek and be exposed to experiences indicative of each of the eight competencies. This assured researchers that students were aware of the role of each of the eight competencies in organizations. Researchers chose this method due to the nature of the students participating in the internship. While all students were leadership students the contexts in which they choose to apply their new leadership knowledge varies creating vast opportunities for students to develop any of the eight competencies.
As discussed earlier, Leadership Skills Inventory-Self © (LSI-S) determined the leadership competency scores of students. The LSI-S is a commercially available leadership questionnaire available from Consulting Resource Group International, Inc. (CRG). The LSI-S consists of five constructs: self-management skills (SMS); interpersonal communication skills (ICS); counseling and problem-solving skills (CCP); consulting skills (CSA); and versatility skills (VOD). Each of the above- mentioned sections contain 12 items, participants rank themselves on a scale of 1
to 10, thus creating a possible score range of 12 to 120. The scale labels were: 1 to 2 – this skill is new to me, I cannot do it; 3 to 4 – I understand this skill, but I cannot do it; 5 to 6 – I can perform this skill, but not reliably; 7 to 8 – I perform this skill well, in many situations; and 9 to 10 – I perform this skill well, and I can teach others, too. Scores for the five constructs were summed to provide a grand score.
Validity and reliability of the instrument were established by Anderson (1998). Reliability, as a measure of stability, was calculated using a test-retest procedure and yielded a coefficient of .82. Face validity was verified by initial participants. Content and construct validity were verified by a thorough examination of the literature. Approval to conduct the study was granted through the Institutional Review Board (#2005-0232).
Researchers administered the pretest, in paper form, during a mandatory face-to- face meeting prior to the beginning of the internships. The research team scored the instruments. To minimize pretest sensitization, researchers provided no feedback to participants. Researchers administered the posttest through a Web- based portal administered by CRG, Inc. at the conclusion of the internship experience. Summated reports were provided to the research team for each participant. As a component of this process, participants could elect to view tutorials explaining their results.
Results
Pretest data were collected from all participants (n = 36). Posttest data were collected from 29 participants (80.1%), 15 from the control group and 14 from the treatment group. As depicted in Table 1, pretest scores ranged from 81.80 (versatility – treatment group) to 93.17 (self management – treatment group).
Although not statistically significant, it is interesting to note the control group (447.67) had initial scores higher than the treatment group (432.28). It is also important to note the variance observed in scores among participants.
Table 1. Leadership Skills Inventory-Self (LSI-S) Scores of Participants
PreTest PostTest
Control
(n = 18)
Treatment
(n = 18)
Control
(n = 15)
Treatment
(n = 14)
|
Self Management 91.50 |
12.11 |
93.17 |
8.72 |
93.93 |
11.06 |
98.29 |
11.30 |
|
Interpersonal 92.22 |
11.19 |
88.22 |
12.45 |
94.47 |
10.86 |
98.86 |
11.78 |
|
Counseling & 92.28 |
12.63 |
86.72 |
13.11 |
94.73 |
8.03 |
96.64 |
14.71 |
|
Consulting 87.72 |
15.19 |
82.28 |
16.65 |
92.67 |
11.15 |
93.29 |
15.93 |
|
Versatility 83.94 |
18.18 |
81.89 |
15.40 |
90.20 |
11.69 |
95.04 |
17.31 |
|
Total 447.67 |
59.50 |
432.28 |
57.64 |
466.00 |
48.39 |
482.11 |
65.33 |
Construct M SD M SD M SD M SD
Communication Problem
Also depicted in Table 1, posttest scores ranged from 90.20 (versatility – control group) to 98.86 (interpersonal communication – treatment group). Participants in the treatment group had higher scores in all five constructs and correspondingly overall higher scores than participants in the control group (482.11 and 466.00, respectively). As with the pretest scores, large variances were observed in scores from participants on the posttest, but these were not statistically significant.
Analysis of Variance (ANOVA) was used to test the null hypothesis, “providing feedback to weekly reflections makes no difference in leadership skill development.” As illustrated in Table 2, results indicated no significant statistical differences in posttest scores based on providing feedback. Thus, the null hypothesis was not rejected.
Table 2. Analysis of Variance on PostTest Scores of the LSI Based on Feedback
Source df F p η2 Power
|
Model |
3 |
1858.38 |
.00 |
1.00 |
1.00 |
|
PreTest |
1 |
48.92 |
.00 |
.65 |
1.00 |
|
Group |
2 |
1.92 |
.17 |
.13 |
.36 |
|
Error |
26 |
||||
|
Total |
29 |
Conclusions and Discussion
Through this study, the researchers were able to conclude weekly feedback on reflections has no affect on the leadership development of leadership interns. Specifically, the results of this study did not support rejection of the null hypothesis. However, this does not dismiss the practical influence of feedback on reflection and the development of leadership skills.
One point of consideration is the practical versus significant nature of the findings. While the results were not statistically significant, the observable differences in the mean scores of the treatment group indicate a need for further research on this phenomena. The treatment group’s mean scores were lower than the control group on four of the five pretest scales. However, the treatment group’s means were higher on all five posttest scales. This change would lead the researchers to believe the feedback on the students’ reflections had some value, although we can not validate this statistically.
An area of interest is the observable difference in the mean scores, supporting the suppositions of Kolb (1984) and Dewey (1938). All of the students’ scores increased from pretest to posttest. Therefore some interaction or link of their experience and their education must have occurred. The greatest increase was in the construct of versatility; defined as the leadership competencies of commitment, creativity, motivation, and stress management. In an internship experience, employers require students to perform a variety of functions; something often discussed, but difficult to practice in a classroom environment. The internship experience may have been the platform, for which students could interface with versatility, creating the competency development. The feedback on their reflections may have made this a more obvious advancement. This supports HLP’s notion that critical reflection helps students make sense of their experiences (Hart Leadership Program, 2006, ¶10).
Meanwhile the construct of self-management had the least increase between pretest and posttest. This construct included the leadership competencies of creativity, team development, strategic planning, and risk taking. This may be a result of the intern position. An intern is rarely given the power or scope to lead a team or take risks. Part of the internship process is to provide a safe environment for hands-on experience. Neither the student nor the employer is typically willing to risk the future by overestimating the competencies of the intern.
Although the results of this study were not significant, the researchers feel the practical results can be applied in other internship programs. The researchers do
believe providing feedback on the student reflections has an impact, but future research will be necessary. However, observable differences are a starting point for continuing to ask questions regarding the role of critiqued reflection in leadership internships.
As leadership educators, there is a continual need to provide students with practical experiences which will further develop their leadership capabilities. Leadership development through experience is a must for students to remain viable candidates for career opportunities. With that, if the possibility exist that an instructor can impact student leadership development during an internship experience than it is owed to the student.
Recommendations
This study creates the opportunity for many additional studies. These studies include quantitative and qualitative methods of analyzing the existing data as well as collecting new data. This also includes analyzing the appropriateness of instrumentation for college student interns with little or no prior professional experience.
This study should be replicated with future intern groups both at the southern land-grant university and in similar programs. Additionally, further research should examine the factors during the actual internship beyond feedback on reflections effecting leadership skill development.
The actual reflections and feedback provided by the research team is a rich data source. A qualitative study examining the reflections of both the control and the treatment groups could identify the leadership constructs addressed. This information could provide greater insight into the leadership skill development of each group and the large variances observed.
Another qualitative study could evaluate the feedback provided to the students and the change, or lack of change, based on feedback. The intention of feedback is to encourage a deeper understanding of their experience and the opportunity to tie the experiential learning to classroom experiences. Kolb (1984) emphasized the role of the instructor in the reflection and observation cycle. The influences of the instructor’s comments were not measurable or identifiable in this study. This information would be a valuable contribution to the literature surrounding reflections and experiential learning.
References
Anderson, T. (1998). Transforming leadership (2nd ed.). New York: St. Lucie Press.
Association of Leadership Educators. (2000, June). Core leadership competencies for organizations and communities. Unpublished manuscript.
Boud, D., Keogh, R., and Walker, D. (1985). Reflection: Turning experience into learning. London: Kogan Page.
Campbell, D.T., & Stanley, J.C. (1963). Experimental and quasi-experimental designs for research. Chicago: Rand McNally.
Densten, I., and Gray, J. (2001). Leadership development and reflection: What is the connection? The International Journal of Educational Management, 15(3), 119-124.
Dewey, J. (1938). Experience in education. Kappa Delta Pi.
Eyler, J., and Giles D., Jr. (1999). Where’s the learning in service-learning? San Francisco: Jossey-Bass, Inc.
Gall, M.D., Gall, J.P. & Borg, W.R. (2003). Educational research: An introduction (7th ed.). Boston: Allyn and Bacon.
Hart Leadership Program Duke University. (n.d.). About HLP. Retrieved June 12, 2006 from http://www.pubpol.duke.edu/centers/hlp/about/index.html.
Knapp, C. (2001). Lasting lessons: A teacher’s guide to reflection on experience.
Charleston, WV: Clearinghouse of Rural Education and Small Schools.
Kolb, D.A. (1984). Experiential learning. Upper Saddle River, NJ: Prentice Hall, Inc.
Lewin, K. (1958). Field theory in social sciences. New York: Harper & Row.
Morgan, A.C., Rudd, R.D., and Kaufman, E. (2004, July). Elements of an undergraduate agricultural leadership program: A delphi study. Paper presented at the 2004 Association of Leadership Educators Conference, Lexington, KY.
Ollila, S. (2000). Creativity and innovativeness through reflective project leadership. Creativity and Innovation Management, 9(3), 195-200.
Piaget, J. (1970). Genetic epistemology. New York: Columbia University Press.
Roberts, T.G. (2006). A philosophical examination of experiential learning theory for agricultural educators. Journal of Agricultural Education, 47(1), 17-29.
Schumacher, L., & Swan, M. (1993). Need for formal leadership training for students in a land grant college of agriculture. Journal of Agricultural Education, 34(3), 1-12.
Biography
Dr. Nicole Stedman, Assistant Professor, is a faculty member of the Agricultural Leadership, Education, and Communications Department at Texas A&M University. She teaches Agricultural Leadership and Development courses at the Undergraduate and Graduate levels. She supervises research at the graduate level and works with freshmen in the Leadership Living Learning Community. Dr.
Stedman’s Bachelors of Science, in Human Resource Development, Masters of Science, in Leadership, and Doctorate, in Leadership Education, are from the University of Florida in Gainesville. Prior to completing her Ph.D., she spent four years working with youth through the justice system for the State Attorney’s Office in the Eighth Judicial Circuit of Florida. Her current research interests include developing strategies for the teaching of leadership emphasizing Emotional Intelligence and Critical Thinking.
Tracy A. Rutherford is an assistant professor in Agricultural Leadership, Education, and Communications department at Texas A&M University. Tracy teaches and advises undergraduate and graduate students in agricultural communications and journalism. She is a graduate of Texas A&M University (Ph.D. Agricultural Education, M.A. Speech Communication) and Cornell University (B.S. Animal Science). Her research interests include biotechnology communications, visual communications, and program development.
Dr. T. Grady Roberts is currently an Assistant Professor in the Department of Agricultural Leadership, Education, and Communications at Texas A&M University. His teaching and research focus on experiential learning and internship experiences for undergraduate students.