A Retrospective Study of Academic Leadership Skill Development, Retention and Use:
The Experience of the Food Systems Leadership Institute
Claudia S. P. Fernandez, DrPH, MS, RD, LDN
Clinical Assistant Professor, Department of Maternal and Child Health Gillings School of Global Public Health
University of North Carolina at Chapel Hill
Adjunct Assistant Professor, Department of Agricultural & Extension Education North Carolina State University Claudia_fernandez@unc.edu
Cheryl C. Noble, MSPH, MSW Department of Maternal and Child Health Gillings School of Global Public Health University of North Carolina at Chapel Hill cnoble@email.unc.edu
Elizabeth T. Jensen, MPH, PhD
Assistant Professor, Department of Epidemiology and Prevention Wake Forest School of Medicine
Adjunct Assistant Professor, Department of Medicine University of North Carolina at Chapel Hill ejensen@wakehealth.edu
Linda Martin, PhD
Associate Dean & Director, Academic Programs FSLI Site Director for OSU
College of Food, Agriculture, and Environmental Sciences The Ohio State University
Marshall Stewart, Ed.D
Special Assistant to the Dean, College Leadership & Strategy Director, APLU Food Systems Leadership Institute
NC State University, College of Agriculture and Life Sciences marshall_stewart@ncsu.edu
Abstract
The Food Systems Leadership Institute (FSLI) is a 2-year leadership development program consisting of 3 intensive in-person immersion retreats, and a robust and customizable distance-based program. Participants come primarily from land-grant and public universities and learn about personal, organizational and system leadership with a focus on food systems as an organizing theme. For this study, program graduates from FSLI Cohorts 4-6 (n=60) were asked to complete an online retrospective pre- and post-test of skill competency and skill use for 20 competencies addressed in the program, with 47 (78%) completing the survey. Data indicate participants’ ratings of skill competency increased significantly across all 20 targeted areas.
Participants further noted that they used these skills more after completing the program as compared to prior to the Fellowship training. Data suggest the FSLI model of leadership development can have a significant impact on participants’ perceived skill level in and use of important skills in both personal and organizational leadership in academic and food system settings.
Introduction
Leaders balance many responsibilities, the skills for which rarely fall into their discipline of practice, and thus a set of skills that will serve them beyond the confines of their technical area is required. For example, leaders must set the culture of the organization and teams (Fernandez, 2007a; Fernandez, 2013; Schein, 2010). Culture and leadership help drive employee engagement (Fernandez, 2007c; Hill, Brandeau, Truelove, & Lineback, 2014), which subsequently creates a broad array of impacts on relationships with partners, customers (students) and clients; relationships with colleagues; staff morale (Freshman & Rubino, 2002); and turnover (Gifford, Zamuto, & Goodman, 2002; Hill, 2002). Academic leaders in particular face the challenges of managing very large, complex, and diverse organizations. In addition, today’s academic leaders face the remarkable lingering challenges resulting from the economic recession and the constant changes to the broader public education landscape.
When resources are limited, or when changes introduce greater complexity and uncertainty, the “softer” leadership skills often become essential to creating positive morale, innovative thinking and entrepreneurialism (Hill et al., 2014). Such skills, in turn, help address tough problems while engaging and retaining valuable staff (Fernandez & Fernandez, 2014; Fernandez & Steffen, 2013). Indeed, “people skills” represent a set of skills not specific to any one discipline or area of expertise, and as such, span the boundaries that exist between professions. The types of people skills that allow individuals to work with others of diverse backgrounds and concerns are of paramount importance, as personnel issues claim particularly large portions of the academic unit budget. For example, as documented at the University of Virginia, personnel expenses consume nearly 60.5% of budgetary dollars (Financing the University, 2014). Rarely have academic leaders needed such a depth of sophisticated leadership skills as they do in the current economic, political and social environment.
Background. In response to the leadership needs discussed above, in 2002, the National Association of State Universities and Land-Grant Colleges (NASULGC, now renamed as the Association of Public and Land-Grant Universities or APLU) appointed a design team consisting of senior university, food industry, and non-profit leaders. This team was responsible for envisioning and designing a new leadership institute for food systems leaders. As such, core
competencies were identified that would be required to lead cultural change in land-grant universities intended for a future “Food Systems Leadership Institute” (FSLI) program. The senior leaders appointed to the design team reflected on personal experience, performed job analysis, and dialogued with leadership development experts. A two-year program structure was designed for the FSLI, which included three onsite week-long training retreats, distance learning activities, mentoring, and a group capstone project (Bryan, 2008).
With subsequent seed funding from the W. K. Kellogg Foundation, the FSLI (The Food Systems Leadership Institute, n.d.) was then created in 2005 to provide intensive leadership education to food system leaders from academic institutions, as well as government stakeholders, and related industries (Bryan, 2008; Food Systems Leadership Institute, n.d.; O’Sullivan, 2008). Working from the initial competencies and recognizing leadership has long been acknowledged as a skill of fundamental importance for success in a variety of fields, including public health (Fernandez & Steffen, 2013; Halverson, Mays, Kaluzny, & House, 1997; Institute of Medicine, 1988; Institute of Medicine, 2003a; Institute of Medicine, 2003b; Umble, et al. 2005; Uno & Zakariasen, 2010), medicine (Chaudry, Jain, McKenzie, & Schwartz, 2008; Cummings et al., 2010; Gifford, Zamuto, & Goodman, 2002; Horwitz et al., 2008; Lattore & Lumb, 2005; Levinson et al., 2002; Loop, 2009) and academia (Kekale, 2003; Sugden, Valania, & Wilson, 2013; Williams & Olsen, 2009), the FSLI Directors created a curricula aiming to build leadership skills that were not bounded by or limited to a specific profession. Of the 20 leadership competencies targeted in the FSLI program, (see Table 1) a majority also form the curricular core of the Maternal and Child Health Public Health Leadership Institute (Fernandez, Noble, Jensen, & Steffen, 2015; Maternal and Child Health Public Health Leadership Institute, n.d.), nearly half serve as the curricular foundation for the American College of Obstetricians and Gynecologists (ACOG) National Leadership Institute (ACOG Leadership Institute, n.d.; Fernandez, Noble, Jensen, & Chapin, in-press), and are also highly similar to the Maternal and Child Health Leadership Competencies (MCH Leadership Competencies, 2012), published by the Federal Maternal and Child Health Bureau.
|
Table 1 Competency Definitions of Leadership Skills Targeted in the Food Systems Leadership Institute (FSLI) |
|
Core Leadership Skills: |
|
|
Organizational and Institutional Leadership Skills |
|
holistic Food Systems goals; strengthen the overall organization by promoting the development and skills of team members
-
Leading Change/Change Management: Identify the need for organizational change; implement processes to bring necessary changes about in order to achieve organizational sustainability, new strategic partnerships, technology development, etc…
-
Stakeholder Analysis: Assess and analyze important players/factors that contribute to or impede individual, team, or organizational success; develop and implement strategies to align stakeholders to organizational mission and vision
-
Futuring: Assess current trends for potential future developments in Food Systems programs, concerns, political agendas, or concepts; contribute to creating the Food Systems of the future through technology, innovation, partnerships, and political influence; embrace a holistic, family/community-based concept of future Food Systems
-
Collaboration/Creative Partnerships: Recognize and reconcile emotional and rational elements in collaboration-building and strategic planning; create opportunities for individual, team, and organizational success through the development of creative partnerships internal to and external to the organization; link partnership development with positive revenue streams
-
Innovation and Performance Management: Implement systems to promote innovation, develop and/or implement performance standards and measures of performance improvements; use performance measures and standards (accountability) to facilitate innovation and entrepreneurship at your home organization; link performance measures and standards to a potential strategic plan for your organization
-
Advocacy: Influence policy, public policy, and resource allocation decisions within political, economic and social systems and institutions; create persuasive dialogue to support one’s issue or goal
-
Food Systems Thinking: Applying systems thinking to understanding food production,
processing, distribution and consumption relationships
There has been great investment in continual examination of the relevance of the competencies used in the FSLI to food system leaders. In 2006 the FSLI competencies were validated and refined through review of scholarly literature on leadership (Bryan, 2008; O’Sullivan, 2008). In 2007 the competencies were prioritized through a study using a Delphi technique to collect input from a larger sample of food system leaders and leadership experts (Bryan, 2008). In 2008, program alumni and food system experts convened to once again review and validate the curriculum, the outcome of which supported continuation of the program as executed since 2005 (O’Sullivan, 2008).
In 2014, a series of stakeholder interviews were conducted with program alumni to ascertain the relevance of the competencies and the program structure in preparation for the 10th anniversary of the program (Stewart, 2014). The targeted leadership competencies and program structure have remained stable throughout these multiple validation processes.
The FSLI program utilizes a hybrid model of executive education to focus training opportunities on developing the 20 targeted leadership skills. The program consists of three residential in-person multi-day intensives held at the partnering Universities (North Carolina State University, The Ohio State University, California Polytechnic State University at San Luis
Obispo, and formerly the University of Vermont) which provide 115 hours of continuing education through both group and individualized learning opportunities and across varying instructional media over a total of 15 days. The face-to-face instruction is supported by a multi- platform highly customizable distance-based core which includes personal executive coaching, peer coaching, online tutorials, program readings, webinars, conference calls, book clubs, mentoring, and completion of a Personal Leadership Project. Given the intensity of this two-year program, serving as an FSLI Fellow represents a serious commitment to leadership development on the part of the participant.
Similar to our previously published work in developing public health (Fernandez et al., 2015) and physician leaders (Fernandez et al., in-press), the objective of the FSLI training was not to perfect abilities in any one area, but rather to “move the needle” of skill development across the board, particularly in the Fellows’ own self-analysis, which is examined in this study. Appraising their skill development helps the academic and industry leaders attending the FSLI assess how their experience in the program impacted their own leadership trajectory and prepared them with the essential skills required. This paper presents the results and analysis of data regarding changes in participants’ (hereafter referred to as Fellows) perceived competence level in and use of the program’s 20 targeted leadership skill areas in three cohorts of FSLI Fellows.
Methods
Participants: Quantitative Study. Fellows who participated in FSLI Cohorts 4-6 (n = 60; enrolling in 2008, 2009, and 2010) were asked in early 2014 to complete a retrospective pre- and post-test of the 20 Core Competencies taught in the program. A total of 47 (78%) Fellows participated in the post-program follow up survey with between 33-36 successfully completing the entire set of pre/post questions (55-60%). Of the 46 Fellows who started the survey, 76% (n =
35) were male and 24% (n = 11) female. With regard to race and ethnicity, 15% (n = 7) of responders identified as Black/African American and 85% as white/Caucasian. None reported their ethnicity as Hispanic. The average age of respondents at the time of completing the survey was 55.0 years. Most (83%) of respondents described their current work duties as “administration”, 4% as teaching, 7% as research, 2% as product development, and 4% as “other”, which they further clarified as a mix of the previously mentioned duties. Of the 60 potential FSLI participants in this study, a majority (n = 57) were academic leaders.
Data Collection: Quantitative Study. At the start of each FSLI program, all Fellows are given a paper-based list of the 20 personal and organizational leadership competencies and their definitions (see Table 1), which are highlighted during orientation. The 20 leadership skills are divided into two domains: Core/Personal Leadership Skills, meaning they are foundational skills needed for a leader to be personally successful; and Organizational/Institutional Leadership skills, indicating their impact is primarily on the organizational and institutional level.
In the first quarter of 2014 all members of FSLI Cohorts 4-6 were contacted several times over a 3-month period via email with a request to complete an online survey using Qualtrics (2014). This 24-question survey collected demographic data as well as responses to multiple choice, open-ended, and ratings questions. Ratings were collected in a retrospective pre- and
post-test. Fellows were asked to rate their level of competency for each of the 20 key leadership skills prior to FSLI participation and currently (10 Core/Personal skills and 10 Organizational/Institutional-related skills) using a standard 5-point Likert scale, with 1 = Unskilled, 2 = Low Skills, 3 = Moderately Skilled, 4 = Good Skills and 5 = Highly Skilled.
Fellows were also asked to rate their use of these same skills prior to FSLI participation and currently, using the scale of Skill use: 1 = not at all, 2 = to a small degree, 3 = moderately, 4 = to a large extent, 5 = extensively. For analysis, the interpretation of scores that fell into mid- ranges was designated as follows: 2.0-2.74 as low skills; 2.75-2.99 = low-moderate; 3.0-3.74 = moderately skilled; 3.75-3.99 = moderate-good; 4.0-4.74 good skills; 4.75-5.0 = highly skilled.
Commonly found in educational venues, a retrospective pre- and post-test is a self- assessment tool in which a participant completes a pretest following an intervention or training program, along with a post-test (Pratt, McGuigan, & Katzev, 2000; Rockwell & Kohn, 2013; Sprangers & Hoogstraten, 1989). This approach is particularly helpful given the problem of response-shift bias (Rohs, 1999), in which learners are unable to accurately assess their development of certain skills prior to a learning or training experience which targets development of those skills. Thus, learners over-estimate their skill level, making the pre- learning self-rating an unreliable or inaccurate evaluation. By design, the retrospective pre and post-test method asks participants to reflect on their growth and rate their level of skill, knowledge, competence, etc…both before they start and after they complete a training experience, when they have a better understanding of the skills targeted for development (Lam & Bengo, 2003; Rockwell & Kohn, 2013; Rohs, 1999; Sprangers & Hoogstraten, 1989).
Educational research suggests that the retrospective pretest method often provides more accurate measures of change than the traditional pretest-posttest design (Lam & Bengo, 2003; Mezoff, 1981; Rohs, 1999), making the design useful when attempting to determine how participants feel about skill acquisition or program effectiveness (Hill & Betz, 2005). Other leadership training programs have successfully used this method in their evaluation efforts (Fernandez et al., 2015; Fernandez et al., in-press; Saleh, Williams, & Balougan, 2004).
This study followed all IRB approval processes by the University of North Carolina at Chapel Hill (UNC) (study #13-4011). Consent was obtained from all participants prior to the collection and analysis of these data and all UNC-IRB ethical guidelines were followed.
Data Analysis: Quantitative Study. Examination of the distribution in mean differences for Fellow perception of skill development and skill use indicated that the distribution of the differences violated assumptions of normality. Therefore, a Wilcoxon signed rank test (SAS, v9.3, Cary, North Carolina) of median score difference was conducted to test the hypothesis of a change in median score from pre to post retrospective assessment for each leadership competency for both skill development and use.
Stakeholder Interview: Qualitative Data Gathering and Analysis. Independent of the collection and analysis of the quantitative retrospective pre and post program data and open ended response data described above, during 2014 the FSLI Director also conducted a series of phone based interviews with program alumni representing each of the eight Cohorts who had completed FSLI training. This was not conducted as a formal research undertaking, but rather an investigation to gain enhanced understanding of alumni learning and their experience as a
Fellow. Interviews were conducted via phone with 12 Fellows by a single interviewer (Stewart, 2014). Data were hand-coded and analyzed by this individual for reporting to the FSLI Oversight Commission (The Food Systems Leadership Institute, n.d.; Stewart, 2014). Alumni were asked five questions relating to a) the most impactful components of the FSLI experience, b) skills alumni have found necessary but were not developed by the program, c) components that are “must do” for the program (including elements to keep or add), d) unique experiences in FSLI not replicated in other leadership development opportunities, and e) post graduate learning needs. The information gained from these interviews was analyzed for this assessment for possible triangulation of the quantitative analysis and online survey findings.
Results
Core/Personal Skills: Perceptions of Skill Development. Pre and post test scores are presented in Table 2 and Figure 1. For the 10 leadership competencies categorized as “Core/Personal Leadership Skills”, mean retrospective pre-test competency scores ranged from a low of 2.74+0.74 (Reflective Leadership) to a high of 3.28+70 (Communication), while mean post-test competency scores ranged from a low of 3.89+0.71 (Career Management) to a high of 4.28+0.51 (Self-Awareness). The difference between the retrospective pre- and post-test competency scores were significantly higher in all ten Personal Leadership skills (p<0.01).
|
Table 2 Pre and Posttest Scores of FSLI Fellows’ Reported Skill Level |
||||||||||
|
n |
Minimum Score |
Maximum Score |
Median |
P |
||||||
|
Pre |
Post |
Pre |
Post |
Pre |
Post |
Pre |
Post |
Diff |
||
|
Core Leadership Skills |
||||||||||
|
1. Self-Awareness |
37 |
37 |
2.0 |
3.0 |
4.0 |
5.0 |
3.0 |
4.0 |
1.0 |
<0.01 |
|
2. Communication |
37 |
37 |
2.0 |
3.0 |
4.0 |
5.0 |
3.0 |
4.0 |
1.0 |
<0.01 |
|
3. Negotiation |
37 |
37 |
1.0 |
3.0 |
5.0 |
5.0 |
3.0 |
4.0 |
1.0 |
<0.01 |
|
4. Conflict Management |
37 |
37 |
1.0 |
3.0 |
5.0 |
5.0 |
3.0 |
4.0 |
1.0 |
<0.01 |
|
5. Visioning |
36 |
36 |
1.0 |
3.0 |
5.0 |
5.0 |
3.0 |
4.0 |
1.0 |
<0.01 |
|
6. Innovation |
36 |
36 |
1.0 |
1.0 |
5.0 |
5.0 |
3.0 |
4.0 |
1.0 |
<0.01 |
|
7. Emotional Intelligence |
37 |
37 |
1.0 |
3.0 |
5.0 |
5.0 |
3.0 |
4.0 |
1.0 |
<0.01 |
|
8. Thinking Politically |
37 |
37 |
1.0 |
3.0 |
5.0 |
5.0 |
3.0 |
4.0 |
1.0 |
<0.01 |
|
9. Reflective Leadership |
36 |
36 |
1.0 |
3.0 |
4.0 |
5.0 |
3.0 |
4.0 |
1.0 |
<0.01 |
|
10. Career Management |
37 |
37 |
1.0 |
2.0 |
5.0 |
5.0 |
3.0 |
4.0 |
1.0 |
<0.01 |
|
Organizational and Institutional Leadership Skills |
||||||||||
|
1. Creating/Impacting Organizational Culture |
36 |
36 |
1.0 |
3.0 |
4.0 |
5.0 |
3.0 |
4.0 |
1.0 |
<0.01 |
|
2. Systems Thinking |
36 |
36 |
1.0 |
2.0 |
5.0 |
5.0 |
3.0 |
4.0 |
1.0 |
<0.01 |
|
3. Bench Building and Succession Planning |
36 |
36 |
1.0 |
2.0 |
5.0 |
5.0 |
3.0 |
4.0 |
1.0 |
<0.01 |
|
4. Leading Change/Change Management |
36 |
35 |
2.0 |
3.0 |
4.0 |
5.0 |
3.0 |
4.0 |
1.0 |
<0.01 |
|
5. Stakeholder Analysis |
36 |
36 |
1.0 |
3.0 |
5.0 |
5.0 |
3.0 |
4.0 |
1.0 |
<0.01 |
|
6. Futuring |
35 |
36 |
1.0 |
2.0 |
5.0 |
5.0 |
3.0 |
4.0 |
1.0 |
<0.01 |
|
7. Collaboration/Creative Partnerships |
36 |
36 |
1.0 |
3.0 |
5.0 |
5.0 |
3.0 |
4.0 |
1.0 |
<0.01 |
|
8. Innovation and Performance Management |
36 |
35 |
1.0 |
3.0 |
5.0 |
5.0 |
3.0 |
4.0 |
1.0 |
<0.01 |
|
9. Advocacy |
36 |
36 |
1.0 |
3.0 |
5.0 |
5.0 |
3.0 |
4.0 |
1.0 |
<0.01 |
|
10. Food Systems Thinking |
35 |
36 |
1.0 |
3.0 |
5.0 |
5.0 |
3.0 |
4.0 |
1.0 |
<0.01 |
|
Note: Likert scale key: 1 = Unskilled, 2 = Low Skills, 3 = Moderate Skills, 4 = Good Skills and 5 = Excellent Skills |
||||||||||
Core/Personal Skills: Perceptions of Skill Use. Pre and post test scores are presented in Table 3 and Figure 2. Fellows also reported their use of the 10 Core/Personal leadership skills, with the retrospective pre-test ranging from a low of 2.80+0.90 (Reflective Leadership) to a high of 3.72+0.85 (Communication). On the post-test of skill use, scores ranged from a low of 4.03+0.71 (Career Management) to a high of 4.44+0.61 (Communication). The difference between the retrospective pre- and post-test scores of skill use were significantly higher in all ten Core/Personal leadership skill areas (p<0.01).
|
Table 3 Pre and Posttest Scores of FSLI Fellows’ Reported Skill Usage |
||||||||||
|
n |
Minimum Score |
Maximum Score |
Median |
P |
||||||
|
Pre |
Post |
Pre |
Post |
Pre |
Post |
Pre |
Post |
Diff |
||
|
Core Leadership Skills |
||||||||||
|
1. Self-Awareness |
37 |
37 |
1.0 |
2.0 |
4.0 |
5.0 |
3.0 |
4.0 |
1.0 |
<0.01 |
|
2. Communication |
37 |
37 |
2.0 |
3.0 |
5.0 |
5.0 |
4.0 |
4.0 |
1.0 |
<0.01 |
|
3. Negotiation |
37 |
37 |
2.0 |
2.0 |
5.0 |
5.0 |
3.0 |
4.0 |
1.0 |
<0.01 |
|
4. Conflict Management |
37 |
37 |
2.0 |
2.0 |
5.0 |
5.0 |
3.0 |
4.0 |
1.0 |
<0.01 |
|
5. Visioning |
36 |
36 |
1.0 |
2.0 |
5.0 |
5.0 |
3.0 |
4.0 |
1.0 |
<0.01 |
|
6. Innovation |
36 |
36 |
1.0 |
1.0 |
5.0 |
5.0 |
3.0 |
4.0 |
1.0 |
<0.01 |
|
7. Emotional Intelligence |
37 |
37 |
1.0 |
3.0 |
5.0 |
5.0 |
3.0 |
4.0 |
1.0 |
<0.01 |
|
8. Thinking Politically |
37 |
37 |
2.0 |
3.0 |
5.0 |
5.0 |
3.0 |
4.0 |
1.0 |
<0.01 |
|
9. Reflective Leadership |
36 |
36 |
1.0 |
2.0 |
4.0 |
5.0 |
3.0 |
4.0 |
1.0 |
<0.01 |
|
10. Career Management |
36 |
36 |
1.0 |
3.0 |
5.0 |
5.0 |
3.0 |
4.0 |
1.0 |
<0.01 |
|
Organizational and Institutional Leadership Skills |
||||||||||
|
1. Creating/Impacting Organizational Culture |
36 |
35 |
1.0 |
2.0 |
4.0 |
5.0 |
3.0 |
4.0 |
1.0 |
<0.01 |
|
2. Systems Thinking |
35 |
35 |
2.0 |
2.0 |
5.0 |
5.0 |
3.0 |
4.0 |
1.0 |
<0.01 |
|
3. Bench Building and Succession Planning |
36 |
36 |
1.0 |
2.0 |
5.0 |
5.0 |
3.0 |
4.0 |
1.0 |
<0.01 |
|
4. Leading Change/Change Management |
36 |
35 |
2.0 |
2.0 |
5.0 |
5.0 |
3.0 |
4.0 |
1.0 |
<0.01 |
|
5. Stakeholder Analysis |
36 |
34 |
1.0 |
2.0 |
5.0 |
5.0 |
3.0 |
4.0 |
1.0 |
<0.01 |
|
6. Futuring |
36 |
36 |
2.0 |
1.0 |
5.0 |
5.0 |
3.0 |
4.0 |
1.0 |
<0.01 |
|
7. Collaboration/Creative Partnerships |
35 |
36 |
1.0 |
3.0 |
5.0 |
5.0 |
4.0 |
5.0 |
1.0 |
<0.01 |
|
8. Innovation and Performance Management |
35 |
36 |
1.0 |
1.0 |
5.0 |
5.0 |
3.0 |
4.0 |
1.0 |
<0.01 |
|
9. Advocacy |
36 |
34 |
1.0 |
1.0 |
5.0 |
5.0 |
3.0 |
4.0 |
1.0 |
<0.01 |
|
10. Food Systems Thinking |
36 |
35 |
1.0 |
1.0 |
5.0 |
5.0 |
3.0 |
4.0 |
1.0 |
<0.01 |
|
Note: Likert scale key: 1 = Not at all, 2 = To a small degree, 3 = Moderately, 4 = To a large extent, and 5 = Extensively |
||||||||||
Core/Personal Skills: Perceptions of “Unskilled” vs. “Highly Skilled” Abilities and Use of Skills. Competency ratings were analyzed to assess perceptions of unskilled and highly skilled abilities in the Core Skills, and the change from pre- to post-FSLI program (Table 4). In their pre-program appraisals, Fellows provided 13 ratings of “unskilled” across all Core categories and no Fellows provided unskilled ratings in any of the dimensions after the program. Conversely, 14 Fellows rated their skill level as “excellent” in any Core competency prior to the program, as compared to 76 such ratings after the program, a 542% increase. With respect to use of skills, prior to the program five Fellows indicated they did “not at all” use a skill prior to their participation, as compared to none responding after the program that they did not use a skill. In terms of using skills “extensively”, 31 Fellows gave that rating to skills listed in the Core/Personal list for before the program as compared to 139 ratings of “extensively” after the program, a 448% increase.
|
Table 4 Comparison of Low and High Perceived Skill Level and Skill Use Ratings for Personal/Core and Organizational/Institutional Leadership Skills |
||||
|
Personal/Core Leadership Skills |
||||
|
Pre |
Post |
% change |
||
|
Perceived Skill Competency |
Rating of 1 |
13 |
0 |
|
|
Rating of 5 |
14 |
76 |
+542% |
|
|
Perceived Skill Use |
Rating of 1 |
5 |
0 |
|
|
Rating of 5 |
31 |
139 |
+448% |
|
|
Organizational/Institutional Leadership Skills |
||||
|
Pre |
Post |
% change |
||
|
Perceived Skill Competency |
Rating of 1 |
18 |
0 |
|
|
Rating of 5 |
16 |
77 |
+481% |
|
|
Perceived Skill Use |
Rating of 1 |
12 |
3 |
-75% |
|
Rating of 5 |
22 |
123 |
+559% |
|
Organizational/Institutional Skills: Perceptions of Skill Development. Pre and post test scores are presented in Table 2 and Figure 1. For the 10 Organizational/Institutional Leadership skills mean retrospective pre-test scores of competency ranged from 2.69+0.80 (Creating/Impacting Organizational Culture) to a high of 3.34+0.94 (Collaboration/Creative Partnerships), while mean post-test scores of perceived competency ranged from a low of 3.71+0.62 (Advocacy) to a high of 4.21+0.69 (Leading Change/Change Management). Median post-test scores were significantly higher in all ten Organizational/Institutional Leadership skill competency areas (p < 0.01).
Organizational/Institutional Skills: Perceptions of Skill Use. Pre and post test scores are presented in Table 3 and Figure 2. Fellows reported their use of these same 10 Organizational/Institutional leadership skills as well, with the retrospective pre-test ranging from a low of from 2.86+0.97 (Food Systems Thinking) to a high of 3.26+0.98 (Stakeholder Analysis). On the post-test of skill use, scores ranged from a low of 3.73+0.84 (Advocacy) to a high of 4.43+0.70 (Collaboration/Creative Partnerships). The median difference between the retrospective pre- and post-test scores of skill use were significantly higher in all ten Organizational/Institutional leadership skill areas (p < 0.01).
Organizational/Institutional Skills: Perceptions of “Unskilled” vs. “Highly Skilled” Abilities and Perceptions of Use of Skills. The Organizational/Institutional Leadership skill competency ratings were analyzed to assess perceptions of either being unskilled or highly skilled, both before and after the program (see Table 4). In pre-program ratings, 18 Fellows rated themselves as “unskilled” in a category however, no Fellows rated themselves as unskilled in any of the dimensions for the post-program time point. Conversely, 16 Fellows rated their pre- program skills as “excellent” in an Organizational/ Institutional leadership competency, as compared to 77 such ratings after the program, a 481% increase. With respect to use of Organizational/Institutional Leadership skills, 12 Fellows noted that prior to the program they
did “not at all” use a certain skill, as compared to only 3 responding after the program that they did not use a skill in this category. In terms of using the skills “extensively”, 22 Fellows gave that rating to skills listed in the organizational/institutional list for before the program as compared to 123 ratings of “extensively” after the program, a 559% increase.
Overall Perceptions of the Usefulness of Leadership Training for Food System Leaders from Academic and Other Environments. Participants endorsed that the FSLI experience was “beneficial to your practice as a food systems leader” with 83% of respondents answering “yes, very” and 15% responding “somewhat”. No participants answered “neutral” or “no”. Ninety-two percent of Fellows responded that they had made “many” or “some” changes to their communication and leadership approaches as a result of their FSLI experience. A full 84% endorsed that since completion of the program they had received a promotion, change of job, or taken on a new leadership opportunity (most of the examples given reflect positions in upper administration in academic institutions). Interestingly, in our review of the FSLI program website (which lists all Fellows and program alumni from all cohorts) we found that 43% of all past participants (Cohorts 1-10) and 55% of the Cohorts numbered 4-6 have reported a job promotion or significant responsibilities to the FSLI team, which were lower than the Cohort 4-6 self-report on this follow up study. Of the 36 respondents who stated they had made a position change, 67% indicated that the skills learned in this program “very much so” prepared them for their new position and/or leadership opportunity, and 25% indicated “somewhat”. Only three respondents were neutral on that issue. Ninety-eight percent of respondents noted that they would recommend the course to their colleagues.
When asked for examples of how they had made changes in their communication or leadership approaches, Fellows provided 109 examples, from which several themes emerged: 32% of examples related to communication/crisis communication practices, 17% to greater self- awareness and personal skills, 20% to creating organizational culture and highly functioning teams, 11% to managing difficult conversations, and 4% to emotional intelligence. The remaining 14% of comments ranged broadly across issues such as strategic planning, leading change, evaluation, etc…
The independent interview process (Stewart, 2014) revealed that 54% of the comments made by program alumni noted the strength of the curriculum and 43% of the comments related to the structure of the leadership development program as strong assets of the program.
Comments related to program structure included those which remarked on the combined residential and customizable distance-based training, the implementation of executive coaching, the use of psychological and leadership assessments, and the online and distance-based learning options. In terms of components that are “must do” for the program (including elements to keep or add) 67% of responses related to continuing the content and structure of the program, 22% endorsed expansion of the training with more contact time, and 11% of comments related to suggesting new topics or new methodologies of teaching.
Discussion
Impact on Skill Development and Use. This study indicates that Fellows in the FSLI program perceived their ability to implement the 20 targeted leadership skills to be higher after their Fellowship than prior to attending FSLI, as can be ascertained at a glance in Figures 1 and 2 (see also Tables 2 and 3). While changes in all skill ratings proved statistically significant for all targeted competencies, more importantly the changes were all greater than 0.5 in magnitude (range of change 0.8-1.3). The industry-standard for meaningful difference, sometimes referred to as ‘clinically significant”, on 360-degree surveys is 0.5 (Musselwhite, n.d.), and hence this measure is more critical to understanding the implications of these findings than even statistical analysis.
While FSLI Fellows were far less likely to rate themselves as “unskilled” in any category post program, there was a dramatic shift in the proportion of Fellows who perceived themselves as “highly skilled” across these 20 competencies. Fellows were asked to participate in the retrospective assessment of learning and skill use between 1-3 years after graduation from FSLI, therefore it is assumed Fellows had sufficient time to utilize the skills taught in the program and to gain a more distanced and objective perspective of their experience. The participants entering this program are highly accomplished and terminal-degree holding leaders, the vast majority of whom had undergone previous leadership training (personal communication), which made designing a curriculum to effectively “move the needle” of their learning a particular challenge. In fact, FSLI is not designed to necessarily impact a particular skill, but rather to move the proverbial “needle” across a broad array of skills, thus shifting the curve of both competency and competitiveness for higher positions of leadership.
The 20 competencies focused upon in FSLI were divided into 10 skills related to personal or “core” leadership skills and a set of 10 “organizational and institutional” leadership skills.
Within the Core set of skills, Fellows rated “Reflective Leadership” as both the area of lowest skill level/competency (2.74, a “low-moderate” score) and the area of least use (2.80, a “use to a low/moderate degree” score) prior to their FSLI Fellowship. These scores for Reflective Leadership significantly increased after the FSLI program (competency: 4.03 (a score indicating “good skills”); use: 4.11 (a score indicating “use to a large extent”). Fellows reported that prior to the program Communication was both their highest scoring skill in terms of competency (3.28) and use (3.72), and it still remained their most-used skill post-program, at 4.44. In the Organizational and Institutional leadership skills set prior to the program, Fellows rated their skills in Creating/Impacting Organizational Culture as the lowest scoring skill (2.69, in the “low” range) and Food Systems Thinking as the least used skill (2.86, in the “used to a moderately- small degree” range), while their highest developed skill was Collaboration and Creative Partnerships (3.34, a “moderate” score), with Stakeholder Analysis rating as their most used skill (3.26, a “moderately used” level). After the program these shifted to Leading Change as the skill they saw as their greatest asset of the set, rated at 4.21 (“good skills”), and Collaboration and Creative Partnerships their most used skill, at 4.43 (“used to a large extent”). The range of skill scores in their pre and post views did not overlap, indicating a strong shift in the curve, which did indeed prove to be statistically significant.
While a few Fellows felt they were “unskilled” in some of the Core/Personal and Organizational/Institutional areas of competency prior to their FSLI experience, none of them felt so afterwards for any of the skills in either category. The vast majority of Fellows reported they used the skills more frequently after completing the FSLI program; in fact, only three individuals endorsed items on the Organizational/Institutional list as “not used at all” (n = 1 each for Futuring, Food Systems Thinking, and Political Systems). Overall, these findings indicate the Fellows felt more equipped with highly useful skills for impacting their home organizations and issues in the food system after participating in the FSLI program.
Appropriateness of Targeted Leadership Competencies. Another issue we wished to examine is the appropriateness of skills targeted in the FSLI program. If skills pre-program were viewed as already in the “moderate-good” range or higher, then it could be an inefficient use of resources to continue to develop such already honed skills. Additionally, if the skills targeted were those Fellows simply did not use after the program, curricular adjustment would be similarly indicated. The most highly rated skill pre-program was Collaboration/Creative Partnerships, which failed to reach the moderate-good range (at 3.34), thus supporting the concept that the skills taught in the FSLI were appropriately targeted.
It is reasonable to consider Fellows might not have developed a specific skill because that skill is simply not relevant to their needs, and is thus not called for in their day-to-day leadership life. Were a skill to be unused by the Fellows after their graduation from the program, such a finding may suggest that while the skills could be sufficiently developed, they were poorly chosen for inclusion because they had less relevance to the real-world situations faced by Fellows. All of the 10 Core/Personal Leadership skills taught in FSLI were rated at or above “to a large extent” post program, suggesting they were appropriate for inclusion in a program directed to Food System leaders, particularly to those who serve in higher education. Of the 10 Organizational/ Institutional Leadership skills rated post-program, half rated at or above “to a large extent” and four rated in the “moderately high” range (3.75-3.99). Only Advocacy rated in the moderate range, at a 3.73.
These data are encouraging in that they support the idea that leadership skill levels did indeed improve, as measured by the perspective and experience of the Fellows themselves. The fact that Fellows report they use each of the 20 leadership skills targeted in FSLI to a significantly greater extent after the program than prior to their participation suggests that when one hones a skill, one has more confidence to use that skill. Fellows endorse the idea that the skills learned in FSLI prepared them for the increased responsibilities and opportunities they have encountered as their careers have progressed. The greater skill use could also be explained by the 84% of respondents who noted they had moved to a position of higher responsibility or influence since becoming an FSLI Fellow. It is interesting to note that the FSLI website states “more than a third of the group has experienced promotions, selection to lead high-profile initiatives, election to university-wide leadership posts, and other recognitions” (Food Systems Leadership Institute, n.d.). However, our review of FSLI Fellows indicated that 36.6% of all Fellows (including currently enrolled) have “moved up”, while 44% of all graduated Fellows, and 55% of Fellows from Cohorts 4-6 who participated in this study. Such large contrasts could indicate that career trajectory may actually be more positively impacted than current public program records indicate, suggesting Fellows do not independently and proactively report career
changes back to the program. Accordingly, such data need to be continually sought out, as was done with the survey presented in this study.
It is true that organizations differ widely, even academic organizations and departments differ from institution to institution. In addition, leadership needs of different positions vary widely as well. Given that the skills targeted in the FSLI program are similar to the skills of focus in leadership training programs for non-academic and non-food system leaders, (ACOG Leadership Institute, n.d.; Maternal and Child Health Public Health Leadership Institute, n.d.) it raises the possibility the FSLI Fellows developed skills useful for facilitating collaboration across organizational silos—skills that could be seen as boundary spanning types of skills. While it is a valid assumption that academic leaders need such skills to work with those outside of their particular discipline in order to be effective in their roles, this study did not explicitly investigate how they made new connections or how they might have worked across boundaries. Such an analysis would be useful in a future assessment of program outcomes. Nevertheless, these data suggest the broad range of leadership skills targeted in the FSLI program was appropriate to include and focus upon, the program does indeed help Fellows build the targeted skills in ways they find significant, and the targeted skills are used by the Fellows as their careers continue to unfold.
One of the most striking outcomes of this study is the similarity of its findings to successful leadership development in other sectors. This analysis confirms our work in similar investigations of leadership development in public health contexts (Fernandez et al., 2015) and for physicians (Fernandez et al., in-press), both of which replicated this data for vastly different audiences, but used very similar methodology and a focus on very similar leadership skills. The replication of these findings supports the concept that leadership can be effectively taught as a strategic and over-arching subject, rather than as grounded within a discipline. Thus it can contribute not only to enhancing the skills of individuals within a field but more importantly, enhance the skills for them to become what have been described as boundary-spanning leaders (Fernandez, 2007b; Fernandez & Fernandez, 2014; Fernandez & Steffen, 2013) who reach across and outside of their fields. More data would be needed from studies such as these to examine how these leaders make connections and work across boundaries and silos using skills such as those investigated in this analysis.
While in-discipline training prepares a professional to advance in their career to the point where they are noticed for their leadership potential, leadership training itself has a different focus and must meet a need far beyond the boundaries of any particular professional discipline. The Food Systems Leadership Institute aims to address this very need. From this analysis of the data, it appears that the FSLI program is successful at helping mid- to senior-level leaders effectively build and hone skills they find highly relevant to their career needs and trajectory.
Implications and Recommendations. Given the broad interest in leadership development across sectors and the fact that academic organizations represent a wide range of specialties, it is reasonable to consider how the FSLI experience might have implications for other leadership programs. First, the program is grounded in competencies twice validated for this audience by an appointed commission of established food- and academic-system related leaders—a set of competencies which ultimately closely resembled those developed for public
health and medical leaders. These cross-discipline similarities could suggest that many of the skills needed for leadership are not necessarily specific to any particular field, but rather represent a “boundary spanning” set of skills that are actually related to dealing with complexity and dealing with people. As such, most of the competencies utilized in FSLI could be of interest to others creating or refining leadership development efforts. While a variety of methods can be used to teach any competencies, it is possible the FSLI model, which embraces a combination of onsite intensives, robust and customizable distance-based components, a combination of peer- based and executive coaching, and project-focused learning, would be useful for others to consider in planning leadership development programs. For example, as one teaching methodology, FSLI incorporated eight valid and reliable psychological/leadership assessment tools which were included to promote self-awareness and to also develop deeper ability in how to understand, motivate, communicate, lead change, and engage with others. FSLI also embraces a philosophy of “meet the Fellows where they are,” working to customize the experience of each participant to their current needs, challenges, and interests.
Lastly, one implication of this analysis is the potential usefulness of this method of post- program examination of skill development and implementation. In our experience many leadership programs assess participant post-program satisfaction but do not examine perceived development or implementation of the skills targeted for improvement. While it can be prohibitively expensive to physically visit a significant number of program alumni to observe and study their behavior post program (particularly if they are widely dispersed nationally or globally, as are FSLI alumni), an evaluation approach such as the one utilized for this analysis can help program designers glean insight into the degree to which participants feel their skills were developed as well as the degree to which they use those skills or find them relevant to their practice in ensuing years. It is relatively simple to both create and evaluate a program participants enjoy, but it is far more challenging to assess that program years later for impact, particularly when resources are limited. This study presents a model that can be implemented in a fiscally responsible manner and also provide useful information to the program evaluators.
Limitations. While this analysis suggests the FSLI approach does indeed promote the development of a wide array of leadership skills, there are some limitations to this study. The research design is not a randomized trial, which would have been impossible to do within the confines of such a program. As a post-completion retrospective analysis of an educational development program, this is a quasi-experimental study. Three Cohorts of a possible nine were included in this study. Cohorts 1-3 were not included due to the concern that assessing pre- intervention competency might not be reliable after a period of five to seven years. Cohort 7 was not included because they graduated from the program just one month prior to this study, which violated the goal of including Fellows who had had time to use and reflect upon their skill development, and Cohorts 8 and 9 were currently enrolled in the program. It is possible that including more Fellows might have indicated different results, however these results were not surprising given the findings of the independent stakeholder interview work with alumni (Stewart, 2014), which help to triangulate this data. FSLI represents a broad array of topic content coupled with a highly customizable format of learning to suit the diversity of learning styles of its participants. Yet, as a leadership program it must be taken as an integrated whole; key importance or impact cannot be assigned to any single facet of the program. FSLI, like other leadership development programs (Fernandez & Noble, 2014; Fernandez, et al., 2015) was
designed, implemented, and evaluated with a synergistic approach and the outcomes are related to “the sum of all the parts”. Finally, as is a concern with any self-report study, the method utilized for this study has the possibility of social desirability bias (Furnham, 1986). It is hoped that the use of the retrospective pretest approach minimizes such bias, as is discussed in the methods section.
Conclusion
The data presented in this analysis suggest the FSLI program does indeed significantly “move the needle” of both self-perceived competency and use of leadership skills in a variety of competencies that seem to be common across professions. In addition, the data suggests such perceived leadership development is possible to achieve in already highly trained, successful leaders in food system-related areas, particularly those working in academic settings. These findings also suggest FSLI targets a broad, appropriate and relevant set of skills for the leaders who attend – skills they perceive to use more frequently as their careers progress. The design of this program lends itself to a positive experience for the Fellows, and to which they attribute an impressive amount of subsequent support for their career development. Nearly all FSLI participants in this study noted a positive career trajectory since participating in the program.
Given that this data has replicated the findings in very similarly structured public health and physician audiences, there may be training elements of this leadership development program which represent “boundary spanning” opportunities and might hold promise for those from other disciplines as well.
References
ACOG Leadership Institute. (n.d.). Retrieved April 13, 2015, from www.acogleadershipinstitute.org
Bryan, M. F. (2008). Prioritizing core competencies for food system leadership. Unpublished manuscript, University of North Carolina, Chapel Hill, NC.
Chaudry, J., Jain, A., McKenzie, S., & Schwartz, R. W. (2008). Physician leadership: The competencies of change. Journal of Surgical Education, 65(3), 213-220. doi:10.1016/j.jsurg.2007.11.014
Cummings, G. G., MacGregor, T., Davey, M., Lee, H., Wong, C. A., Lo, E., et al. (2010).
Leadership styles and outcome patterns for the nursing workforce and work environment: A systematic review. International Journal of Nursing Studies, 47(3), 363-385. doi:10.1016/j.ijnurstu.2009.08.006
Fernandez, C. (2007a). Creating thought diversity: The antidote to group think. Journal of Public Health Management and Practice, 13(6), 670-671. doi:10.1097/01.PHH.0000296146.09918.30
Fernandez, C. (2007b). Emotional intelligence in the workplace. Journal of Public Health Management and Practice: JPHMP, 13(1), 80-82.
Fernandez, C. (2007c). Employee engagement. Journal of Public Health Management and Practice: JPHMP, 13(5), 524-526.
Fernandez, C., & Fernandez, R. (2014). It-factor leadership: Become a better leader in 13 steps.
Chapel Hill, NC: FastTrack Leadership.
Fernandez, C., Noble, C., Jensen, E., & Chapin, J. (in-press). Improving leadership skills in physicians: A 6-month retrospective study. Journal of Leadership Studies.
Fernandez, C., Noble, C., Jensen, E., & Steffen, D. (2015). Moving the needle: A retrospective pre- and post-analysis of improving perceived abilities across 20 leadership skills.
Maternal and Child Health Journal, 19(2), 343-352. doi:10.1007/s10995-014-1573-1
Fernandez, C., & Steffen, D. (2013). Leadership for public health. In L. Shi, & J. A. Johnson (Eds.), Novick and morrow’s public health administration: Principles for population based management (3rd ed., pp. 241-265). Santa Barbara, CA: Jones and Bartlett Publishers.
Financing the university 101: University of Virginia questions and answers. Retrieved September 26, 2014, from http://www.virginia.edu/finance101/answers.html
The Food Systems Leadership Institute. (n.d.). Retrieved March 18, 2015, from www.FSLI.org Freshman, B., & Rubino, L. (2002). Emotional intelligence: A core competency for health care
administrators. The Health Care Manager, 20(4), 1.
Furnham, A. (1986). Response bias, social desirability and dissimulation. Personality and Individual Differences, 7(3), 385-400. doi:10.1016/0191-8869(86)90014-0
Gifford, B. D., Zammuto, R. F., & Goodman, E. A. (2002). The relationship between hospital unit culture and nurses’ quality of work life. Journal of Healthcare Management / American College of Healthcare Executives, 47(1), 13-25.
Halverson, P. K., Mays, G., Kaluzny, A. D., & House, R. M. (1997). Developing leaders in public health: The role of executive training programs. The Journal of Health Administration Education, 15(2), 87-100.
Hill, K. (2002). The relationship between hospital unit culture and nurses’ quality of work life – practitioner application. Journal of Healthcare Management, 47(1), 25-26.
Hill, L. G., & Betz, D. L. (2005). Revisiting the retrospective pretest. American Journal of Evaluation, 26(4), 501-517. doi:10.1177/1098214005281356
Hill, L., Grandeau, G., Truelove, E., & Lineback, K. (2014). Collective genius: The art and practice of leading innovation. Cambridge, MA: Harvard Business Review Press.
Horwitz, S. K., Horwitz, I. B., Daram, P., Brandt, M. L., Brunicardi, F. C., & Awad, S. S. (2008). Transformational, transactional and passive-avoidant leadership characteristics of a surgical resident cohort – analysis using the multifactor leadership questionnaire.
Journal of Surgical Research, 144(2), 182-183. doi:10.1016/j.jss.2007.12.020
Institute of Medicine (U.S.) Committee for the Study of the Future of Public Health. (1988). The future of public health National Academy Press. Retrieved from http://books.nap.edu/openbook.php?record_id=1091&page=R2
Institute of Medicine (U.S.) Committee on Assuring the Health of the Public in the 21st Century. (2003a). The future of the public’s health in the 21st century. Washington, DC: The National Academies Press. Retrieved from http://books.nap.edu/openbook.php?record_id=10548&page=R2
Institute of Medicine (U.S.) Committee on Assuring the Health of the Public in the 21st Century. (2003b). Who will keep the public healthy?: Educating public health professionals for the 21st century. Gebbie K., Rosenstock L. and Hernandez L. (Eds.). Washington, D.C: National Academies Press. Retrieved from http://books.nap.edu/openbook.php?record_id=10542&page=R2
Kekäle, J. (2003). Academic leaders as thermostats. Tertiary Education and Management, 9(4), 281-298. doi:10.1080/13583883.2003.9967110
Lam, T., & Bengo, P. (2003). A comparison of three retrospective self-reporting methods of measuring change in instructional practice. American Journal of Evaluation, 24(1), 65- 80. doi:10.1016/S1098-2140(02)00273-4
Lattore, P., & Lumb, P. D. (2005). Professionalism and interpersonal communications: ACGME competencies and core leadership development qualities. why are they so important and how should they be taught to anesthesiology residents and fellows? Seminars in Anesthesia, Perioperative Medicine and Pain, 24(3), 134-137. doi:10.1053/j.sane.2005.07.006
Levinson, W., D’Aunno, T., Gorawara-Bhat, R., Stein, T., Reifsteck, S., Egener, B., et al. (2002).
Patient-physician communication as organizational innovation in the managed care setting. American Journal of Managed Care, 8(7), 622-630.
Loop, F. (2009). Leadership and medicine. Gulf Breeze, FL: Fire Starter Publishing.
Maternal and Child Health Public Health Leadership Institute. (n.d.). Retrieved March 18, 2015, from http://mchphli.org
MCH leadership competencies, ver. 3.0. (2012). Retrieved March 18, 2015, from http://leadership.mchtraining.net
Mezoff, B. (1981). Pre-then-post testing: A tool to improve the accuracy of management training program evaluation. Performance & Instruction, 20(8), 10-16. doi:10.1002/pfi.4150200808
Musselwhite, W. C. Discovery leadership profile: Facilitator guide (4.09th ed.). Greensboro, NC: Discovery Learning Press.
O’Sullivan, R. (April, 2008). Origin, validation, and prioritization of FSLI core competencies.
Food Systems Leadership Institute curriculum review meeting preparatory materials, Columbus, OH.
Pratt, C. C., McGuigan, W. M., & Katzev, A. R. (2000). Measuring program outcomes: Using retrospective pretest methodology. American Journal of Evaluation, 21(3), 341-349. doi:10.1016/S1098-2140(00)00089-8
Qualtrics Research Suite (2015). Qualtrics (version 61,679) [software]. Available from https://unc.az1.qualtrics.com/WRQualtricsControlPanel/?T=5IS2Hi6VxcEHjE5cNf0oyg
Rockwell, S., & Kohn, H. (1989). Post-then-pre evaluation. Journal of Extension [Internet], 27(2), Sept 13, 2013.
Rohs, F. R. (1999). Response shift bias: A problem in evaluating leadership development with self-report pretest-posttest measures. Journal of Agricultural Education, 40(4), 28-37. doi:10.5032/jae.1999.04028
Saleh, S. S., Williams, D., & Balougan, M. (2004). Evaluating the effectiveness of public health leadership training: The NEPHLI experience. American Journal of Public Health, 94(7), 1245-1249. doi:10.2105/AJPH.94.7.1245
Schein, E. (2010). Organizational culture and leadership. San Francisco, CA: Josey-Bass.
Sprangers, M., & Hoogstraten, J. (1989). Pretesting effects in retrospective pretest-posttest design. Journal of Applied Psychology, 74(2), 265.
Stewart, M. (2014). “FSLI Fellow Conversations”: A stakeholder interview report to the FSLI Oversight Commission, a study of the current FSLI structure and content on the 10th Anniversary of the FSLI program. Unpublished manuscript.
Sugden, R., Valania, M., & Wilson, J. R. (2013). Leadership and cooperation in academia : Reflecting on the roles and responsibilities of university faculty and management.
Cheltenham: Edward Elgar Publishing. doi:10.4337/9781781001820
Umble, K., Steffen, D., Porter, J., Miller, D., Hummer-McLaughlin, K., Lowman, A., et al. (2005). The national public health leadership institute: Evaluation of a team-based approach to developing collaborative public health leaders. American Journal of Public Health, 95(4), 641-644. doi:10.2105/AJPH.2004.047993
Uno, H., & Zakariasen, K. (2010). Public health leadership education in North America. Journal of Healthcare Leadership, 2010, 11-15. doi:10.2147/JHL.S9727
Williams, R. L., & Olsen, S. (2009). Leadership development in higher education. In J. C. Knapp, & D. Siegel (Eds.), The business of higher education volume 1: Leadership and culture. Santa Barbara, CA: Greenwood Publishing Group.
Wright, K., Baker, E., Rowitz, L., Merkle, A., Reid, W. M., Robinson, G., et al. (2000).
Competency development in public health leadership. American Journal of Public Health, 90(8), 1202-1207. doi:10.2105/AJPH.90.8.1202
Author Biographies
Dr. Fernandez creates and directs leadership institutes for the public sector, health care, public health and academic institutions. She is an executive coach and earned her doctorate in public health leadership from the University of North Carolina. She is one of the founding members of the Food Systems Leadership Institute.
Ms. Noble earned her MSPH and MSW from the University of North Carolina at Chapel Hill. She currently serves as the evaluator for multiple leadership development programs. She has received extensive leadership training in Maternal and Child Health, social work, and the non-profit sector.
Dr. Jensen has a PhD in Epidemiology and an MPH in Maternal and Child Health from the University of North Carolina at Chapel Hill. She has undergone extensive leadership training in Maternal and Child Health. These skills have facilitated development of her research program in reproductive, perinatal, and pediatric epidemiology.
Dr. Martin serves as OSU Site Director of FSLI and is the Associate Dean and Director of Academic Affairs in the College of Food, Agricultural, and Environmental Sciences at The Ohio State University. With more than 25 teaching awards, she is known nationally for her record of teaching excellence.
Dr. Marshall Stewart serves as Director of the Food Systems Leadership Institute as a part of his responsibilities in NC State’s College of Agriculture and Life Sciences, where he directs the College’s leadership and strategy initiatives. He has over 29 years of experience in public education, higher education and leadership development.