Preparing Leadership Scholars in PhD Programs: A Review of Research Methodology Training
Companies, government agencies, and organizations continue to seek people with proven leadership skills while looking to develop the leadership skills in their current staffs. It is said that “poor leadership can ruin good people and organizations” while “good leadership can promote employee engagement and organizational success” (Shoup, 2016, p. 177). Leadership is not constrained within traditional organizational boundaries. It is complex, dynamic and reaches across and throughout many institutions and disciplines (Winton & Palmer, 2018).
The increased need to understand the practical importance of this complex concept gave birth to leadership studies as an academic discipline. Leadership studies has been defined as “an interdisciplinary, academic and applied field of study that focuses on the fluid process and components of the interaction between leaders and followers in a particular context” (Sowcik, 2012, p. 193). Leadership studies has emerged as a discipline that is on par with other established interdisciplinary soft sciences and is believed to continue to grow and receive academic recognition (Harvey & Riggio, 2011). The creation of several academic journals and professional associations reveals that the field of leadership studies meets the “hallmark of an academic discipline” (Riggio, 2011, p. 13). Since the turn of the millennium, leadership peer-reviewed journal options have increased due to the growth of journal page counts, the number of articles published and the continued emergence of new leadership research journals (Gardner et al., 2010).
Leadership’s complexity and interdisciplinary nature, however, makes studying it extremely challenging as it is found in all different areas of business, government, as well as other nonprofit organizations (Andenoro et al., 2013). Methods used to explore these topics are becoming increasingly complex and multifaceted as leadership scholars are recommended to take advantage of more sophisticated statistical techniques (Gardner et al., 2010; Lowe & Gardner, 2000; Scandura & Williams, 2000). Leadership research methods used in leadership studies over the past decades have been categorized into various quantitative and qualitative techniques. These techniques include, but are not limited to, linear regressions, analysis of variance techniques, factor analytic techniques, structural equation modeling/path analytical techniques, multi-levels-of-analysis techniques, meta-analysis, linear techniques for categorical dependent variables, time series/event history techniques, non-parametric techniques, computer simulation techniques, case analysis, grounded theory, and content analysis techniques (Gardner et al., 2010; Scandura & Williams, 2000).
Although researchers are advised to take advantage of more sophisticated research techniques, Doctor of Philosophy (PhD) preparation for these researchers may be lacking. According to Ferguson et al., “A common theme across the literature is the determination that PhD students are not receiving enough instruction and practice in ether quantitative or qualitative methods to be qualified researchers” (2017, p. 139). With the rapid growth of leadership research and the increased complexity of research methods, preparing PhD students to do research is becoming more challenging. It is for this reason that research into the statistics, measurement, and methodology course requirements of PhD leadership studies programs is warranted. The purpose of this study is to explore how PhD leadership programs prepare students to do research in the complex area of leadership. More specifically, the study aims to explore what the current contributions of statistics, measurement, and methodology classes are to the course and curriculum requirements of PhD-level university leadership studies programs.
Literature Review
As indicated by Bernard Bass (1990) in the first chapter of their seminal book Bass and Stogdill’s Handbook of Leadership, “There are almost as many different definitions of leadership as there are persons who have attempted to define the concept” (p. 11). Bass demonstrates that leadership has been conceptualized in a variety of different capacities. According to Bass (1990), leadership has been conceived:
as the focus of group processes, as a matter of personality, as a matter of inducing compliance, as the exercise of influence, as particular behaviors, as a form of persuasion, as a power relation, as an instrument to achieve goals, as an effect of interaction, as a differentiated role, as initiation of structure, and as many combinations of these definitions. (p. 11)
Not only does leadership have a diverse array of possible perceptions, it also cuts across disciplines and industries and is embedded within each organization’s very fabric (Winton & Palmer, 2018). It follows that researchers would use a variety of research methodologies to decipher and comprehend such a complex phenomenon.
Research Methodologies Used in Leadership Studies
In a broad sense, research methodologies can be grouped into three categories: quantitative, qualitative, and mixed methods. When looking at the social sciences, Creswell & Creswell (2017) conceptualize these research designs as “types of inquiry within qualitative, quantitative, and mixed methods approaches that provide specific direction for procedures in a research study” (p. 18). With the advancement of computer technology, these various research designs have expanded as individuals have articulated new procedures for conducting social science research and as our data analysis and ability to analyze complex models has increased (Creswell & Creswell, 2017).
Quantitative Methods in Leadership Studies
At its fundamental core, quantitative research is a means for testing objective theories by examining the relationship among variables (Creswell & Creswell, 2017). Quantitative research methodology has had a persistent and customary role in leadership research. It remains the most commonly used approach among leadership researchers (Stentz et al., 2012, p. 1173). The strong grasp these quantitative methodologies have on leadership research is precipitated by the fact that leadership research was mostly born out of the fields of psychology and organizational behavior (Ospina, 2004). Historically, these fields have been dominated by a logical positivism or post positivist worldview which asserts that positive knowledge is based on natural phenomena and can be scientifically or mathematically proven (Ospina, 2004, p. 1281).
This quantitative research dominance can be seen in the research done in leadership studies over the last few decades. Research on leadership was prevalent in leading industrial and organizational psychology journals such as the Journal of Applied Psychology (JAP) and Personnel Psychology (PPsych) in the 1960s, 1970s, and 1980s (Cascio & Aguinis, 2008). As leadership studies as a discipline gained relevance and momentum in the early 1990s and 2000s, other leadership-focused outlets for leadership studies research became available (Gardner et al., 2010). For example, The Leadership Quarterly was conceived by Bernard Bass, Bob House, and Henry Tosi over dinner one night as they lamented the fact that “leadership research was scattered across so many different journals and that no one venue brought together leadership scholars from diverse disciplines” (Lowe & Gardner, 2000, p. 460). Even so, the introduction of research journals with a leadership studies focus did not sway researchers from publishing leadership research in the traditional psychology- and management-focused journals as there were no noticeable declines during this period (Bernerth et al., 2018; Cascio & Aguinis, 2008; Morrison, 2010).
In their seminal study on scholarly leadership studies, Gardner et al. (2010) completed a comprehensive review of 353 articles that appeared in The Leadership Quarterly from 2000-2009 (p. 923). This was a follow-up study to Lowe and Gardner’s (2000) study looking at 188 articles from the same publication from 1990 to 1999. Gardner et al. (2010) found that the proportion of quantitative publications relative to qualitative and mixed methods designs used in leadership research published in The Leadership Quarterly during the 1990s increased substantially in the 2000s, while the number of qualitative and mixed methods designs used decreased. The ratio of quantitative to qualitative studies increased from 2 to 1 in the 1990s to 3 to 1 in the 2000s (Gardner et al., 2010, p. 943). When looking at the type of statistical analysis used by leadership researchers in those two decades, they found that the most common forms of multivariate analysis were regression analysis, ANOVA/MANOVA, and factor analytical techniques, exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) (Gardner et al., 2010, p. 943). They also uncovered that while the number of articles that used more advanced models like structural equation modeling and multiple-levels-of analysis techniques were slightly more numerous in the second decade, these methods remained to be small proportions of the total number of methods utilized overall (Gardner et al., 2010, p. 943).
Analogous to the Gardner et al. (2010) study, Stentz et al. (2012) published a thorough review of 1,170 leadership articles published in The Leadership Quarterly between 1990 and June 2012 (p. 1178). In a more aggregate fashion, they classified the studies they found into four comprehensive categories. They observed that 65% of the articles reviewed were non-empirical, 27% were quantitative, 3% were qualitative, and 5% were potentially mixed methods studies (Stentz et al., 2012, p. 1178). This adds evidence to the claim that quantitative research methodologies largely eclipse the empirical leadership studies literature.
Qualitative Methods in Leadership Studies
Although quantitative methods seem to dominate the current leadership research, many have argued for an increased use of qualitative methods in leadership research. Rather than determining cause and effect, predicting, or describing the distribution of some attribute among a population, qualitative research is interested in uncovering the meaning of a phenomenon for those involved (Merriam, 2009, p. 5). Before the turn of the millennium, Conger (1998) saw the benefits of using qualitative research methods for leadership research. Conger (1998) advanced the idea that quantitative methodology “fails to capture the great richness of leadership phenomena” and instead leaves leadership researchers with “only sets of highly abstracted and generalized descriptors” (p. 118). The idea presented was that qualitative methods are much better suited to uncover “leadership’s many dimensions” (Conger, 1998, p. 118). In a similar fashion, Ospina (2004) proposed the idea that qualitative research methods are much more equipped to explore the high sensitivity that leadership has to context (p. 1280). Ospina presented qualitative methods as an inside approach where researchers are immersed in terms of experiential engagement, direct contact with subjects, and physical involvement in the setting, which allows the researcher to inductively build theory from observations and allows the “data to speak” (p. 1281).
In an exhaustive review of leadership research, sociologist Bryman (2004) looked at leadership studies that used qualitative research methods and were published between 1979 and 2003 in the Social Sciences Citation Index (pp. 732-747). It was noted that qualitative researchers “have made important contributions” to leadership studies during that time frame (Bryman, 2004, p. 762). Although critical of some of the methods in the study, Bryman (2004) states that qualitative methods have proven valuable in a variety of leadership areas. A few of these areas mentioned are understanding leadership in relation to the change process; how leaders manipulate symbols and meanings to achieve organizational ends; uncovering or attaching greater significance to aspects of leadership that are relatively neglected by quantitative researchers; and appreciating the relevance of context for leader behavior (Bryman, 2004, p. 762). Bryman (2004) also noted that while hardly “explosive,” there was a real and considerable upward trend in the number of leadership studies published that have adopted a qualitative approach between 1990 and 2003 (p. 749).
In similar fashion, Parry et al. (2014) looked at all research published in The Leadership Quarterly from 1990 through 2013 (p. 136). In their research, they noted that while quantitative research continues to “enjoy methodological hegemony” (p. 132) within the field of leadership studies, there is little doubt that qualitative research is beginning to make inroads in the field. Like Bryman (2004), Parry et al. (2014) noted a steady rise in the number of leadership studies published using qualitative methods from 1990 to 2013, with the period of 2008 to 2012 having the largest increase (p. 137). Klenke et al. (2015) remark that even journals “traditionally known for strong quantitative research…have opened the doors for submission of qualitative articles or commissioned special issues dealing with theoretical and methodological issues in qualitative leadership research along with the publication of empirical qualitative articles” (pp. 5-6). When carried out with the same degree of rigor and consideration for validity and quality, the distinct advantages these methods have over quantitative approaches in leadership studies has elicited this upward trend in qualitative research (Klenke et al., 2015).
Mixed Methods in Leadership Studies
The last method reviewed can be conceptualized as a mixture of both quantitative and qualitative methods. Clark and Ivankova (2016) offer a definition of mixed methods research as “a process of research in which researchers integrate quantitative and qualitative methods of data collection and analysis to best understand a research purpose” (p. 4). Mixed method has also gained support for its use in leadership studies. Bryman’s (2004) critical review of qualitative research advocated for the use of both qualitative and quantitative methods (mixed methods) to provide clearer outcomes for leadership research. Stentz et al. (2012) furthered Bryman’s notion, indicating “mixed methods can answer the complex questions of leadership in new and meaningful ways” and that there is reason to believe that these methods will be used into the future to advance our understanding of leadership (p. 1181). Indeed, leadership scholars have seen the value in using mixed methods to answer complex leadership questions in important and meaningful ways and will continue to employ this method into the future (Bryman, 2004; Gardner et al., 2010; Stentz et al., 2012).
PhD Program Research Methods Requirements
Graduate and post-graduate programs were developed by universities to increase discipline-specific mastery (Hyatt & Williams, 2011, p. 53). It is no surprise that because of leadership’s diverse and interdisciplinary nature, doctoral programs that emphasize leadership prepare students for work in a variety of capacities, including corporations, non-profit organizations, K-12 schools, higher education, and government agencies (Hyatt & Williams, 2011, p. 54). Many universities have various forms of leadership education taught in a variety of areas, including colleges of agriculture, business, engineering, military sciences, education, and liberal arts (Andenoro et al., 2013, p. 2). Though a variety of doctoral degrees are conferred each year, a Doctor of Philosophy, or PhD, is considered by many as the summit of university learning and scholarship (Mowbray & Halse, 2010). While distinctions between the different types of doctoral degrees in some contexts may be vague and imprecise (Shulman et al., 2006), the PhD has generally been viewed as the research-intensive doctoral degree (Hyatt & Williams, 2011). At a very foundational level, the ability of a researcher to accurately use and evaluate the research and the methods that should be used in each research study is tied to their doctoral preparation (Ferguson et al., 2017). It then follows that PhD degrees that are offered in leadership programs should have a research focus and include classes and curriculum that prepare doctoral students for future research responsibilities.
Because of the nature of research, it would be expected that graduates of PhD programs be well versed in the different qualitative, quantitative, and mixed methods research. However, various studies have found results contradictory to this logic. Capraro and Thompson (2008) looked at 251 doctoral programs within the college of education at 21 universities. Specifically, they examined the research methods requirements (quantitative, qualitative, and mixed methods) for both PhD and EdD programs (Capraro & Thompson, 2008). Surprisingly, they found that 25.9% of the 251 doctoral programs required neither a quantitative methods course nor a qualitative methods course, and only 44.2% required completion of courses in both areas (Capraro & Thompson, 2008, p. 250). Specific to the PhD programs, only 74% required at least one quantitative course and only 48% required at least one qualitative course, while the mean number of courses required in each area were 2.5 and 1.3 respectively (Capraro & Thompson, 2008, p. 250).
In their 2008 study of 100 different colleges and schools that were part of the American Association of Colleges for Teacher Education and offered PhD and EdD degrees, Leech and Goodwin (2008) found that quantitative research methods courses were required by 62% of the programs, with advanced quantitative courses being required by only 15% of the programs (p. 4). Furthermore, qualitative research courses were required by 62% of the programs, while only 22% required at least one mixed methods course (Leech & Goodwin, 2008, p. 4). They found a wide range of research requirements in their study, which ranged from programs with very extensive methods training provisions to programs with very little (Leech & Goodwin, 2008, p. 6). Comparatively, Leach and Haug (2015) surveyed 28 different institutions and found that of the 18 that offer PhD degrees, only 21% require a basic statistics course, while 21% require an intermediate statistics course.
Hallinger (2011) looked at methodologies used by 130 doctoral students in educational leadership and management programs in more than 85 American universities over a 30-year period (p. 280). The 130 dissertations used in the study were authored by 92 EdD and 38 PhD students (Hallinger, 2011). Interestingly, 54% used a single causal factor correlation method (two variables), 12% used a single factor correlation with controls (two variables while controlling for a third), and 24% used a multiple factor model (Hallinger, 2011, p. 289). Although the PhD dissertations tended to use a higher level of statistical methods like multiple factor and advanced modeling more often, there was no significant difference in the level of statistical methods used between the two degree programs. Overall, Hallinger (2011) suggests that the methodologies used by the doctoral students were “inadequate for the task of contributing to either the theoretical or the practical knowledge base” in the field (pp. 46-47).
While most of the previous studies are tied to educational studies and educational leadership, researchers have looked at research class requirements for doctoral programs in other disciplines. Aiken et al. (2008) surveyed PhD programs in psychology and found that quantitative training for doctoral students is lagging behind, that they have great concern about training in measurement, and that they are “profoundly troubled” (p. 48) about the area of research design. In the area of medical schools, Dawson-Sanders and colleagues (1987) found a “wide variation” (p. 2630) in the topics being covered and the number of hours devoted to instruction in biostatistics. In physiology, Weissgerber et al. (2016) found that although the ability to understand and apply statistical concepts is essential for research, biostatistics training is not always required to complete a PhD (Weissgerber et al., 2016, p. 2). In political science, Schwartz-Shea (2003) noted the dominance of quantitative over qualitative classes taught in doctoral programs is a lopsided seven to one (Schwartz-Shea, 2003, p. 382). Studies have looked at doctoral programs in management and organizational studies (Scandura & Williams, 2000), educational statistics (Curtis & Harwell, 1998) and mathematics education (Shih et al., 2016). In leadership studies, researchers have looked at instructional and assessment strategies of graduate and undergraduate programs (Jenkins, 2018), competencies of doctoral leadership faculty (Hyatt & Williams, 2011), and research course offerings in master’s degree programs (Bustamante & Combs, 2011). However, no recent studies were found that specifically look at the research methods course offerings or requirements for leadership studies PhD programs. Because of this gap, there is a need to further the research in leadership studies in this area. This exploratory study aims to extend the literature by studying the current statistics, measurement, and methodology course and curriculum requirements of PhD level university leadership studies programs.
Methods
A questionnaire asking about quantitative, qualitative, and mixed methods curriculum and course requirements was sent electronically to the faculty contacts of 95 leadership PhD programs. The programs and contacts were identified by using the International Leadership Association’s (ILA) list of PhD leadership programs. Given that the PhD degree has generally been viewed as the research-intensive doctoral degree (Hyatt & Williams, 2011), it is expected that these programs would have an explicit curruicular focus to prepare students to engage in research. Although EdD programs may include a research focus, EdD programs are likely to prioritize developing leadership practitioners and not necessarily leadership researchers (for examples, see Organizational Leadership Degrees, 2020 and Scott, 2016). In order to focus the exploration on research needs for leadership scholarship, we specifically explore PhD programs in this study. The questionnaire was deployed via email and responses were recorded by the Qualtrics Survey Software. The identified contacts were asked to answer the survey directly or to forward the questionnaire to the program faculty member who is most knowledgeable about their program’s methodology curriculum and requirements. The first emails were sent in December 2018, and 12 reminders were sent through March 2019. The survey included program-specific questions to investigate the research methods course requirements and offerings. A majority of the questions focused specifically on program demographics like the average number of PhD students and full-time faculty, which specific methodology classes were required or were electives, and the amount of coverage each program offered for each research topic or methodology. Of the original 95 programs contacted, nine indicated that their program was not a PhD program or was not focused on leadership, and these programs were removed from the survey, leaving 86 potential programs meeting the inclusion criteria. Of the programs surveyed, 27% (n = 23) responded.
Results
A variety of program demographic information was included on the questionnaire and collected from the respondents. Of the programs responding to the survey, the average enrollment size was 46 PhD students. The programs that responded also indicated that the number of full-time faculty ranged from 1 to 17 with an average of 7.5 faculty per program over the last three years. When surveyed about having a home department, 82.6% of the respondents indicated that they had a home department while the remaining 17.4% indicated they either did not have a home department or listed other colleges or departments associated with the degree. Program responses indicated that 48% of the programs were completed face-to-face, 39% used a hybrid of face-to-face and online classes, and 9% were strictly completed online. Most programs, 65%, are on a semester schedule, 18% indicated they were on a trimester system, and 4% indicating they used quarter systems. Thirteen percent responded to the schedule question using the “other” category; some of these programs indicated that they had six- or nine-week classes. Most programs indicated that the program’s primary focus was business, management or organizational behavior (43.4%), or the program had an educational focus, which included training administrators or student affairs professionals (26.0%). Leadership studies (13.0%), women in leadership (4.4%), public and nonprofit (4.4%), interdisciplinary (4.4%), and other or not given (4.4%) were also indicated as a focus for the responding programs. Just less than 30% (29.7%) of responding programs indicated that their students have published articles requiring any kind of quantitative, qualitative, or mixed methods research design in a peer-reviewed journal while enrolled in their doctoral program.
When looking more specifically at the methodology requirements of each program, respondents indicated that research design methods (94.4%), analysis of variance (ANOVA) (90.0%), multiple regression (90.0%), and introduction to qualitative methods and approaches (85.0%) were required the most by responding programs in a program core course (see Table 1). Coursework on meta-analysis (44.4%), modern missing data treatments (43.8%), intermediate or advanced mixed methods approaches (43.8%), and classical and/or item response theory (41.2%) were most often cited as not being available to PhD students in their course offerings (see Table 1).
| Table 1
Program Methodology Availability |
||||
| Methodology | Core Course | Elective | Not Available | |
| Analysis of Variance (ANOVA) | 90.00% | 10.00% | 0.00% | |
| Multiple Regression | 90.00% | 10.00% | 0.00% | |
| Multivariate Analysis (MANOVA) | 52.63% | 36.84% | 10.53% | |
| Factor Analysis | 63.16% | 31.58% | 5.26% | |
| Structural Equation Modeling (SEM) | 44.44% | 33.33% | 22.22% | |
| Multilevel Modeling (MLM) | 41.18% | 35.29% | 23.53% | |
| Longitudinal Data Analysis | 33.33% | 33.33% | 33.33% | |
| Meta-Analysis | 27.78% | 27.78% | 44.44% | |
| Categorical Data Analysis | 38.89% | 38.89% | 22.22% | |
| Modern Missing Data Treatments | 18.75% | 37.50% | 43.75% | |
| Nonparametric Statistics | 25.00% | 43.75% | 31.25% | |
| Introduction to Qualitative Methods and/or Approaches | 85.00% | 15.00% | 0.00% | |
| Intermediate or Advanced Qualitative Methods and/or Approaches | 47.37% | 47.37% | 5.26% | |
| Analyzing Qualitative Data | 61.11% | 33.33% | 5.56% | |
| Narrative-Biographical Approach | 35.29% | 52.94% | 11.76% | |
| Phenomenological Approach | 50.00% | 50.00% | 0.00% | |
| Grounded Theory Approach | 38.89% | 44.44% | 16.67% | |
| Ethnography | 44.44% | 50.00% | 5.56% | |
| Content Analysis (sometimes considered a quantitative methodology) | 41.18% | 35.29% | 23.53% | |
| Case Study Approach | 55.56% | 38.89% | 5.56% | |
| Introduction to Mixed Methods Approach | 55.56% | 27.78% | 16.67% | |
| Intermediate or Advanced Mixed Methods and/or Approaches | 12.50% | 43.75% | 43.75% | |
| Classical Test Theory | 29.41% | 41.18% | 29.41% | |
| Item Response Theory | 23.53% | 41.18% | 35.29% | |
| Classical and/or Item Response Theory | 23.53% | 35.29% | 41.18% | |
| Test Construction | 37.50% | 43.75% | 18.75% | |
| Multidimensional Scaling | 23.53% | 41.18% | 35.29% | |
| Research Design | 94.44% | 0.00% | 5.56% | |
| Quasi-Experimental Design | 82.35% | 5.88% | 11.76% | |
| Survey Research | 66.67% | 27.78% | 5.56% | |
| Program Evaluation | 47.06% | 47.06% | 5.88% | |
| Survey Sampling | 66.67% | 27.78% | 5.56% | |
When looking at the required methodology areas for each program, the respondents were asked to describe the degree to which each required topic was covered in their core courses. Responses were only available for the topics that the respondents initially indicated as “required in a core course.” Of the topics selected, programs offered a full class covering analysis of variance (ANOVA), multiple regression, and introduction to qualitative methods approaches, as well as research design methods and program evaluation (see Table 2 for full methodology course results and Table 3 for measurement/scaling and research design course results).
| Table 2
Program Methodology Coverage Per Required Topic (Core Statistics, Quantitative, Qualitative, and Mixed Methods) |
|||||||
| Methodology | Percent Responses of Content Coverage During a Course | ||||||
| Min. | Intro. | Approx 25% | Approx 50% | Approx75% | Full | ||
| Analysis of Variance (ANOVA) | 0.00% | 38.46% | 38.46% | 0.00% | 0.00% | 23.08% | |
| Multiple Regression | 0.00% | 30.77% | 30.77% | 7.69% | 0.00% | 30.77% | |
| Multivariate Analysis (MANOVA) | 0.00% | 66.67% | 33.33% | 0.00% | 0.00% | 0.00% | |
| Factor Analysis | 12.50% | 50.00% | 37.50% | 0.00% | 0.00% | 0.00% | |
| Structural Equation Modeling (SEM) | 20.00% | 60.00% | 0.00% | 20.00% | 0.00% | 0.00% | |
| Multilevel Modeling (MLM) | 25.00% | 50.00% | 25.00% | 0.00% | 0.00% | 0.00% | |
| Longitudinal Data Analysis | 75.00% | 25.00% | 0.00% | 0.00% | 0.00% | 0.00% | |
| Meta-Analysis | 50.00% | 50.00% | 0.00% | 0.00% | 0.00% | 0.00% | |
| Categorical Data Analysis | 20.00% | 40.00% | 40.00% | 0.00% | 0.00% | 0.00% | |
| Modern Missing Data Treatments | 0.00% | 0.00% | 100.00% | 0.00% | 0.00% | 0.00% | |
| Nonparametric Statistics | 0.00% | 100.00% | 0.00% | 0.00% | 0.00% | 0.00% | |
| Introduction to Qualitative Methods and/or Approaches | 0.00% | 7.69% | 30.77% | 15.38% | 0.00% | 46.15% | |
| Intermediate or Advanced Qualitative Methods and/or Approaches | 0.00% | 0.00% | 75.00% | 25.00% | 0.00% | 0.00% | |
| Narrative-Biographical Approach | 20.00% | 60.00% | 0.00% | 20.00% | 0.00% | 0.00% | |
| Phenomenological Approach | 14.29% | 71.43% | 0.00% | 14.29% | 0.00% | 0.00% | |
| Grounded Theory Approach | 20.00% | 40.00% | 20.00% | 20.00% | 0.00% | 0.00% | |
| Ethnography | 20.00% | 80.00% | 0.00% | 0.00% | 0.00% | 0.00% | |
| Content Analysis (sometimes considered a quantitative methodology) | 20.00% | 60.00% | 0.00% | 0.00% | 20.00% | 0.00% | |
| Case Study Approach | 14.29% | 71.43% | 0.00% | 14.29% | 0.00% | 0.00% | |
| Introduction to Mixed Methods Approach | 37.50% | 50.00% | 0.00% | 12.50% | 0.00% | 0.00% | |
| Intermediate or Advanced Mixed Methods and/or Approaches | 0.00% | 100.00% | 0.00% | 0.00% | 0.00% | 0.00% | |
Note. Min. = minimally covered; Intro = introductory (1 or 2 class periods); Approx = approximately; Full = full course to cover content area.
| Table 3
Program Methodology Coverage Per Required Topic (Measurement/Scaling and Research Design) |
|||||||
| Methodology | Percent Responses of Content Coverage During a Course | ||||||
| Min. | Intro. | Approx 25% | Approx 50% | Approx75% | Full | ||
| Classical Test Theory | 0.00% | 66.67% | 33.33% | 0.00% | 0.00% | 0.00% | |
| Item Response Theory | 0.00% | 100.00% | 0.00% | 0.00% | 0.00% | 0.00% | |
| Classical and/or Item Response Theory | 0.00% | 100.00% | 0.00% | 0.00% | 0.00% | 0.00% | |
| Test Construction | 20.00% | 40.00% | 40.00% | 0.00% | 0.00% | 0.00% | |
| Multidimensional Scaling | 0.00% | 100.00% | 0.00% | 0.00% | 0.00% | 0.00% | |
| Research Design | 7.69% | 30.77% | 7.69% | 15.38% | 0.00% | 38.46% | |
| Quasi-Experimental Design | 10.00% | 60.00% | 10.00% | 20.00% | 0.00% | 0.00% | |
| Survey Research | 22.22% | 55.56% | 11.11% | 11.11% | 0.00% | 0.00% | |
| Program Evaluation | 33.33% | 33.33% | 0.00% | 16.67% | 0.00% | 16.67% | |
| Survey Sampling | 25.00% | 50.00% | 12.50% | 12.50% | 0.00% | 0.00% | |
| Note. Min. = minimally covered; Intro = introductory (1 or 2 class periods); Approx = approximately; Full = full course to cover content area.
|
|||||||
When asked broadly about the number of required classes that are devoted to different areas, the survey respondents indicated that all programs required as least one quantitative methods course and 87.5% required at least one qualitative methods course. Only 31.25% of programs indicated that they required a measurement course (see Table 4). Of the responding leadership PhD programs, 86.96% indicated that they offer an introductory research methods course. Of the respondents who answered yes to offering this course, 75% of them indicated that the class is taught by a faculty member in their home department.
| Table 4
Program Required Courses |
||||||
| Topic | Number of Required Courses Covering Specific Topics | |||||
| Not a full course | 1 | 2 | 3 | 4 | 5 or more | |
| Statistics Courses (Ex: Analysis of Variance, Regression, Etc.) | 0.00% | 37.50% | 37.50% | 0.00% | 6.25% | 18.75% |
| Measurement Courses (Ex. Test Theory, Test Construction, Etc.) | 68.75% | 18.75% | 12.50% | 0.00% | 0.00% | 0.00% |
| Qualitative Methods Courses | 12.50% | 68.75% | 12.50% | 0.00% | 0.00% | 6.25% |
Programs were also surveyed to better understand what types of software were used in each program. Of the programs who participated, 83.3% said they used SPSS for quantitative methods. R was also popular with 60.0% of the programs indicating that they used the software for quantitative computations. The results for the qualitative software were more divided. NVivo (33.3%), MaxQDA (58.3%), and atlas.ti (40.0%) were the most popular qualitative software programs in use by the PhD leadership programs (see Table 5 for full results).
| Table 5
Computer Software |
||
| Software | Used | Not Used |
| Quantitative: SPSS | 83.33% | 16.67% |
| Quantitative: SAS | 40.00% | 60.00% |
| Quantitative: Stata | 27.27% | 72.73% |
| Quantitative: SYSTAT | 0.00% | 100.00% |
| Quantitative: MPlus | 30.00% | 70.00% |
| Quantitative: R | 60.00% | 40.00% |
| Quantitative: Minitab | 10.00% | 90.00% |
| Qualitative: NVivo | 33.33% | 66.67% |
| Qualitative: MaxQDA | 58.33% | 41.67% |
| Qualitative: atlas.ti | 40.00% | 60.00% |
| Qualitative: HyperRESEARCH | 40.00% | 60.00% |
| Qualitative: Dedoose | 0.00% | 100.00% |
Discussion
This study extends some of the recent literature focused on graduate education within leadership studies (e.g. Winton & Palmer, 2018) by uncovering the current statistics, measurement, and methodology course and curriculum requirements of PhD level leadership programs. Possibly the most important implication of this research is the findings offer a framework from which faculty can benchmark what they are currently doing to train PhD students in the current methodologies and explore the direction their programs can take to improve. This study provides leadership programs with objective data of the current state of PhD leadership curriculum.
Beyond this benchmark, we also offer these additional findings. First, as quantitative research methodologies have persisted as the most commonly used approaches among leadership researchers (Stentz et al., 2012), it is encouraging to see the variety of quantitative research methods offered by the responding programs. As it was found that regression analysis, ANOVA/MANOVA, and factor analysis were some of the most common quantitative approaches in leadership research (Gardner et al., 2010), a majority of the respondents to the survey indicated that these methods are either required or offered as an elective in their programs. Very few indicated that these methodologies were “not available” to their students. Additionally, all respondents indicated the requirement of at least one statistics course in their program. This is contrary to the findings of researchers in other fields of study (Capraro & Thompson, 2008; Leech & Haug, 2015; Leech & Goodwin, 2008). However, with the growing use of more advanced techniques like structural equation modeling in leadership research (Gardner et al., 2010), there seems to still be a lack of specialized statistics content available to leadership PhD students. More focus on these emerging quantitative methods may be needed to prepare leadership scholars for future research demands.
Furthermore, the value and growth of qualitative methods in leadership research is becoming increasingly clear (Bryman, 2004; Ospina, 2004; Parry et al., 2014). Qualitative research continues to be used to uncover the many dimensions and levels of complexity in the practice of leadership that may be hidden from quantitative methods (Conger, 1998). Based on the results of this study, 87.5% of the leadership programs required at least one qualitative methods course. As with quantitative methods, some of the more specialized qualitative methods (like grounded theory, content analysis, and narrative-biographical approaches) were not available in all programs. Because of the growing use of this methodology, programs may want to increase their students’ exposure to these areas in the future.
Although the results of this exploratory study are encouraging, there are still some areas of concern. As with qualitative methods, mixed methods approaches have also been utilized increasingly by leadership scholars (Bryman, 2004; Gardner et al., 2010; Stentz et al., 2012). Analogous to the relatively short history of mixed methods designs, the offerings of these were found to be less available to students in these programs, as only 55.56% require at least an introduction to the topic in a core course. It is also a point of concern to the authors that many topics in the areas of measurement and scaling, as well as research design, were also either not required or not offered in the curriculum of the responding programs. These results could underscore the need for more training to better prepare future leadership scholars during their PhD programs, and it may also serve as a call to provide ongoing professional development for scholars to refine and enhance their research skills.
Conclusion
The results of this study may inform the development of future PhD programs and the evaluation of current programs. The study provides objective information describing how PhD programs prepare future leadership researchers. However, this study does not provide much objective information about appropriate level of methods preparation required for PhD leadership students. Do the designs of course offerings by leadership programs adequately prepare future scholars to understand the approaches used to study the discipline? Although this study fills a large gap in the literature, more questions on leadership program research methodology course offerings still remain.
Future research considerations could include: (a) more in-depth comparisons to doctoral requirements in other fields, (b) comparing the impact of the programs that require breadth over depth (deeper specialization in quantitative, qualitative, or mixed methods), (c) examining the relationship between number of methods courses taken and publication expectations, and (d) examining the impact of utilizing methods experts in other departments as instructors versus in-house methods instruction.
References
Aiken, L. S., West, S. G., & Millsap, R. E. (2008). Doctoral Training in statistics, measurement, and methodology in psychology: Replication and extension of Aiken, West, Sechrest, and Reno’s (1990) survey of PhD programs in North America. American Psychologist, 63(1), 32–50. https://doi.org/10.1037/0003-066X.63.1.32
Andenoro, A. C., Allen, S. J., Haber-Curran, P., Jenkins, D. M., Sowcik, M., Dugan, J. P., & Osteen, L. (2013). National Leadership Education Research Agenda 2013-2018: Providing strategic direction for the Field of Leadership Education. Retrieved from http://leadershipeducators.org/ResearchAgenda
Bass, B. M. (1990). Bass & Stogdill’s Handbook of Leadership: Theory, Research, & Managerial Applications (3rd ed.). The Free Press.
Bernerth, J. B., Cole, M. S., Taylor, E. C., & Walker, H. J. (2018). Control variables in leadership research: A qualitative and quantitative review. Journal of Management, 44(1), 131–160. https://doi.org/10.1177/0149206317690586
Bryman, A. (2004). Qualitative research on leadership: A critical but appreciative review. The Leadership Quarterly, 15(6), 729–769. https://doi.org/10.1016/J.LEAQUA.2004.09.007
Bustamante, R. M., & Combs, J. P. (2011). Research courses in educational leadership programs: relevance in an era of accountability. International Journal of Education Policy and Leadership, 6(3), 1–11. https://doi.org/10.22230/ijepl.2011v6n3a276
Capraro, R. M., & Thompson, B. (2008). The educational researcher defined: What will future researchers be trained to do? The Journal of Educational Research, 101(4), 247–253. https://doi.org/10.3200/JOER.101.4.247-253
Cascio, W. F., & Aguinis, H. (2008). Research in industrial and organizational psychology from 1963 to 2007: Changes, choices, and trends. Journal of Applied Psychology, 93(5), 1062–1081. https://doi.org/10.1037/0021-9010.93.5.1062
Clark, V. L. P., & Ivankova, N. V. (2016). Mixed Methods Research: A Guide to the Field. Sage Publications Inc. https://doi.org/10.4135/9781483398341
Conger, J. A. (1998). Qualitative research as the cornerstone methodology for understanding leadership. Leadership Quarterly, 107-121. https://doi.org/10.1016/S1048-9843(98)90044-3
Creswell, J. W., & Creswell, J. D. (2017). Research Design: Qualitative, Quantitative, and Mixed Methods Approaches (5th ed.). Sage Publications Inc.
Curtis, D. A., & Harwell, M. (1998). Training doctoral students in educational statistics in the United States: A national survey. Journal of Statistics Education, 6(1). https://doi.org/10.1080/10691898.1998.11910604
Dawson-Saunders, B., Azen, S., Greenberg, R. S., & Reed, A. H. (1987). The instruction of biostatistics in medical schools. The American Statistician, 41(4), 263–266. https://doi.org/10.1080/00031305.1987.10475497
Ferguson, S. L., Hovey, K. A., & Henson, R. K. (2017). Quantitative preparation in doctoral dducation programs: A mixed-methods study of doctoral student perspectives on their quantitative training. International Journal of Doctoral Studies, 12, 137–156. https://doi.org/10.28945/3789
Gardner, W. L., Lowe, K. B., Moss, T. W., Mahoney, K. T., & Cogliser, C. C. (2010). Scholarly leadership of the study of leadership: A review of The Leadership Quarterly’s second decade, 2000–2009. The Leadership Quarterly, 21(6), 922–958. https://doi.org/10.1016/j.leaqua.2010.10.003
Hallinger, P. (2011). A review of three decades of doctoral studies using the principal instructional management rating scale: A lens on methodological progress in educational leadership. Educational Administration Quarterly, 47(2), 271–306. https://doi.org/10.1177/0013161X10383412
Harvey, M., & Riggio, R. E. (2011). Leadership Studies: The Dialogue of Disciplines. Edward Elgar Publishing. https://doi.org/10.4337/9780857936486
Hyatt, L., & Williams, P. E. (2011). 21st century competencies for doctoral leadership faculty. Innovative Higher Education, 36(1), 53–66. https://doi.org/10.1007/s10755-010-9157-5
Jenkins, D. (2018). Comparing instructional and assessment strategy use in graduate and undergraduate-level leadership studies: A global study. Journal of Leadership Education, 17(1), 73–92. https://doi.org/10.12806/V17/I1/R2
Klenke, K., Wallace, J. R., & Martin, S. M. (2015). Qualitative Research in the Study of Leadership. Emerald Group Publishing Limited. https://doi.org/10.1108/9781785606502
Leech, N., & Haug, C. A. (2015). Investigating graduate level research and statistics courses in schools of education. International Journal of Doctoral Studies, 10, 93–110. https://doi.org/10.28945/2111
Leech, N. L., & Goodwin, L. D. (2008). Building a methodological foundation: doctoral-level methods courses in colleges of education. Research in the Schools, 15(1), 1-8.
Lowe, K. B., & Gardner, W. L. (2000). Ten years of The Leadership Quarterly. The Leadership Quarterly, 11(4), 459–514. https://doi.org/10.1016/S1048-9843(00)00059-X
Merriam, S. B. (2009). Qualitative research: A guide to design and implementation: Revised and expanded from qualitative research and case study applications in education. John Wiley & Sons.
Morrison, E. (2010). OB in AMJ: What is hot and what is not? Academy of Manamgent, 53(5). https://doi.org/10.5465/amj.2010.54533166
Mowbray, S., & Halse, C. (2010). The purpose of the PhD: Theorising the skills acquired by students. Higher Education Research & Development, 29(6), 653–664. https://doi.org/10.1080/07294360.2010.487199
Organizational Leadership Degrees. (2020). Edd vs Phd: Which degree is right for you? https://organizationalleadershipdegrees.org/edd-vs-phd/
Ospina, S. (2004). Qualitative Research. In J. M. Burns, G. J. Sorenson, & G. R. Goethals (Eds.), Encyclopedia of leadership (pp. 1279–1284). Sage Publications Inc.
Parry, K., Mumford, M. D., Bower, I., & Watts, L. L. (2014). Qualitative and historiometric methods in leadership research: A review of the first 25 years of The Leadership Quarterly. Leadership Quarterly, 25(1), 132–151. https://doi.org/10.1016/j.leaqua.2013.11.006
Riggio, R. E. (2011). Is leadership studies a discipline? In M. Harvey & R. E. Riggio (Eds.), Leadership Studies: The Dialogue of Disciplines (pp. 9–19). Edward Elgar Publishing. https://doi.org/10.4337/9780857936486.00006
Scandura, T. A., & Williams, E. A. (2000). Research methodology in management: Current practices, trends, and implications for future research. Academy of Management Journal, 43(6), 1248–1264.
Schwartz-Shea, P. (2003). Is this the curriculum we want? Doctoral requirements and offerings in methods and methodology. PS: Political Science & Politics, 36(3), 379–386.
Scott, S. (2016, October 24). Ph.D. vs. Ed.D. Inside Higher Ed. https://www.insidehighered.com/blogs/gradhacker/phd-vs-edd
Shih, J., Reys, R. E., & Engledowl, C. (2016). Profile of research preparation of doctorates in mathematics education in the United States. Far East Journal of Mathematical Education, 16(2), 135–148. https://doi.org/10.17654/ME016020135
Shoup, J. R. (2016). Leadership, Organizational, and institutional studies: Reconciling and teaching competing perspectives. Journal of Leadership Education, 15(4), 167–182. https://doi.org/10.12806/V15/I4/T1
Shulman, L. S., Golde, C. M., Bueschel, A. C., & Garabedian, K. J. (2006). Reclaiming education’s doctorates: A critique and a proposal. Educational Researcher, 35(3), 25–32. https://doi.org/10.3102/0013189X035003025
Sowcik, M. (2012). The next phase of leadership development. Concepts and Connections: Developing a Leadership Program, National Clearinghouse for Leadership Programs, 18(2), 1-15.
Stentz, J. E., Clark, V. L. P., & Matkin, G. S. (2012). Applying mixed methods to leadership research: A review of current practices. The Leadership Quarterly, 23(6), 1173–1183. https://doi.org/10.1016/j.leaqua.2012.10.001
Weissgerber, T. L., Garovic, V. D., Milin-Lazovic, J. S., Winham, S. J., Obradovic, Z., Trzeciakowski, J. P., & Milic, N. M. (2016). Reinventing Biostatistics Education for Basic Scientists. PLOS Biology, 14(4). https://doi.org/10.1371/journal.pbio.1002430
Winton, S. L., & Palmer, S. (2018). Developing Leadership for Increasing Complexity : A Review of Online Graduate Leadership Programs. Journal of Leadership Education, 17(1), 162–176. https://doi.org/10.12806/V17/I1/A4