Introduction
The methods of learning about leadership can come through many forms. Massive Open Online Courses (MOOCs) are an educational delivery method used in the leadership field and are relevant to the transference of leadership knowledge (Karnouskos, 2017). Leadership education focuses on the purpose, mission, and context directly aligned with learning and utilizes resources that provide quality leadership development through many delivery methods (Chunoo & Osteen, 2016), including MOOCs.
There are divided schools of thought surrounding the role of MOOCs in the education system. Wood (2013) asserted that MOOCs are in their infancy in terms of development and implementation, and their impact on the future of education has yet to be realized. On the contrary, some argue that MOOCs have already reached their peak interest level and the market has become stagnant (Bannier, 2016). While both arguments could be true, MOOCs continue to emerge in leadership education and influence the way leadership is being taught to the masses. More current trends shows that MOOCs continue to hold educational value in their certificate-granting abilities and that content continues to expand (Reich & Ruipérez-Valiente, 2019).
In 2017, 81 million students registered for nearly 9,400 courses being offered by over 800 universities worldwide (Shah, 2018). In 2018, over 101 million students enrolled in over 11,400 courses offered at over 900 universities globally, with the amount of paying users increasing and Coursera (the largest MOOC platform) hitting record revenues of over $140 million (Shah, 2018). In short, this data shows an increase in MOOC education over the past few years. Content areas include general education, healthcare, and any other area where there is a topical expert willing to teach a MOOC (Hoy, 2014).
In terms of leadership education, Jenkins (2012) established a set of traditional classroom pedagogies through a nationwide survey aimed at university faculty and staff who taught leadership education courses. In a follow-up study, Jenkins (2016) presented a set of leadership pedagogies for traditional online courses taught on the undergraduate and graduate levels. Their findings highlighted the role of discussion boards, with shared instructor-student led discussion boards, instructor-led discussion boards, and group discussions being the most utilized methodologies in online leadership courses. A better understanding of MOOC instructional and assessment pedagogy can be beneficial to leadership educators who are seeking to develop an array of methodology and course strategies for expansion into the educational and professional development delivery fields.
This study examines the current state of leadership MOOCs using a quantitative content analysis approach. These MOOCs are offered by U.S. and Canadian universities and institutions and offered across MOOC platforms identified through an independent MOOC review. In order to better understand and fill the gap in the literature centered on leadership-oriented MOOCs, the pedagogy used to teach and assess leadership MOOC learners, leadership MOOC characteristics, and the range of leadership topics covered were explored. The purpose of this study is to survey leadership MOOCs by identifying a) the pedagogical strategies utilized in leadership-oriented MOOCs, b) the defining characteristics (e.g. enrollment numbers, suggested study effort, financial cost, etc.), and c) the range of leadership topics covered through their content.
The central research question and sub-questions of this study are:
- What is the current state of leadership MOOCs offered in English by U.S. and Canada universities?
- What instructional pedagogies are used in leadership-oriented MOOCs?
- What are the defining characteristics of course and curricular design of leadership MOOCs?
- What leadership development topics are covered in leadership MOOCs?
Literature Review
To better understand the development and presence of leadership education pedagogy in the world of MOOCs, we must understand massive open online courses (MOOCs). One important note is that MOOC research is varied and limited in nature. The research can show contrasting views regarding the use of and popularity of MOOCs, the intent of their purpose, and discrepancies in the instructional and conceptual information provided (Esposito, 2012).
MOOCs as Distance Education. MOOCs are the modern-day representation of distance education. Distance education has become a common fixture in the American and global education system. By moving coursework to an online format, learners gain access to information as never before. This access began through the use of shared educational database networks and library-sharing databases (Anderson, 2008, p.397- 399). This made it easier for learners to access resources and information for their success in these courses. MOOCs were the next step in the distance education continuum as they take on online learning on an even greater global scale (Butcher & Wilson-Strydom, 2013). Heralded as the “most important educational technology in 200 years” (Regalado, 2013, p. 61), MOOCs have brought great promise to the way educational courses are offered on a global scale. Figure 1 represents the key differences between MOOCs and online coursework.
Figure 1
Defining Massive Open Online Courses. Massive open online courses are generally defined as an online course aimed at unlimited participation and open access via the web (Kaplan & Haenlein, 2016). MOOCs are designed and delivered online by experts in the topical area being taught (Marshall, 2013). Marshall (2013) describes that MOOCs are designed to be taken by cohorts of students and designated into two key structures: xMOOC and the cMOOC (Daniel, 2012; Mohamed & Hammond, 2018).
xMOOCs, or eXtended MOOCs, focus on the knowledge to be learned and are based on traditional university online courses (Mohamed & Hammond, 2018). Connectivist MOOCs, or cMOOCs, focus on the student as a member of a community of learners that engages their resources and personal experiences. xMOOCs are set up much like traditional courses with the instructor providing resources and content, whereas cMOOCs operate along the assumption that course materials and content engagement are derived from instructors and students during the course (Hew & Cheung, 2014).
The MOOC structure allows for a variety of educational purposes to be met. Each distance education pedagogical design captures a worldview defined by its epistemological foundations, development models and technologies used (Aerts, et al., 1994; Anderson & Dron, 2011;). MOOCs provide a synthesis of leadership education and can expand on professional development in these formats.
MOOCs as Professional Development. Professional development can serve many functions, but one of the most common MOOC uses are courses focused leadership development and training for the individual (Moldoveanu & Narayandas, 2019). MOOCs have emerged as a popular and affordable delivery method of professional development among businesses and for those seeking professional development certification for their employees (Meister, 2013; Moldoveanu & Narayandas, 2019). Addressing the skills gap is a concern for companies who have found it difficult to find the right talent for their positions (Barton, 2012). The skills gap phenomenon is of global concern (Calonge & Shah, 2014; Lakshmi, 2013). A large number of international sampled employers (n= 103) were already using or had planned to use MOOCs for professional development for their employees (Radford, et al., 2014). Credentialing, through certificates or digital badges, was also necessary, according to surveyed employees (Egloffstein & Ifenthaler, 2017). Digital badges have also been granted to students who have completed MOOCs as a form of motivation and to promote engagement (Muilenburg & Berge, 2016).
MOOCs in Leadership Education. MOOCs have a strong foundation in content related to leadership education. A Google search by the author returned over 5,800 MOOC courses that centered on various aspects of leadership. [Note: The searches returned results for a large number of traditional online courses; therefore all 5,800 are not available to be included in the study population.] The intersection of effective delivery, cognition, and application in leadership education can impact both the instructor pedagogy and the learner development (Miller & Miller, 2001). Teaching leadership involves a complexity that may not exist in some academic disciplines, as leadership educators focus their courses on attitudes, knowledge, behavior, and application (Rosch & Anthony, 2012).
Leadership educators are encouraged to look at assignments and activities that integrate evaluation and assessment through interactive teaching methods, such as MOOCs (Jenkins, Endersby, & Guthrie, 2015). Creative and digital technologies and leadership platforms will advance the field toward more connected and collaborative learning environments (Jenkins, Endersby, & Guthrie, 2015).
Instructional strategy use in online leadership courses has been investigated in very few studies (Jenkins, Endersby, & Guthrie, 2015; Phelps, 2012). More specific research into instructional methods utilized in leadership courses include work on blogs (Gifford, 2010), service learning (Guthrie & McCracken, 2010) social media (Odom, et al., 2013), and podcasts (Robinson & Ritzko, 2009).
Conceptual Framework
Distance education pedagogical design captures a worldview defined by its epistemological foundations, development models and technologies used (Aerts, et al., 1994; Anderson & Dron, 2011). For this study, the decision was made to combine focus on Jenkins’ (2016) online leadership course pedagogy and the key characteristics of MOOCs, provided by Glance and colleagues (2013), to provide a conceptual framework for the research.
Jenkins’ (2016) research on online courses sought to explore the types of teaching pedagogies associated with leadership education. Within the bounds of online leadership courses, they used a quantitative survey (N=118) to gain the perspectives of instructional methods and assessments used by undergraduate and graduate instructors in leadership courses in the U.S and Canada. Their selection of online strategies were informed by the empirical studies of Djajalaksana (2011; Bonk, et al., 2006; Bonk & Zhang, 2008; Fletcher, Djajalaksana, & Eison, 2012). Their work also incorporated pedagogy from the work of Allen and Hartman (2008), Conger (1992), and Shulman’s framework of signature pedagogies (Shulman, 2005), which provided a recommendation that pedagogies link ideas, practice and values that lead to practical and informed use of the content. Jenkins (2012, 2016) investigations into leadership pedagogy informed the list of instructional methods that were examined for the online leadership pedagogy study. These same lists will be used to examine leadership MOOC pedagogy. This study is one of the few online leadership studies that reveals a comprehensive pedagogy of leadership education from an online perspective.
Glance, Forsey, and Riley (2013) defined signature characteristics for MOOC instruction that still appear to be the standard among MOOC researchers. To validate the claims that MOOCs are based on pedagogical foundations and are comparable to face-to-face instruction offered by universities, the researchers (Glance et al., 2013) conducted a review of the empirical literature. MOOCs are defined by characteristics that include: lectures formatted as short videos combined with formative quizzes, automated assessment and/or peer and self-assessment, and an online forum for peer support and discussion.
It is the combination of leadership pedagogy and definitive MOOC characteristics that serve as a collaborative framework for this study. With these two guiding concepts, this research allows the work to reveal the existing pedagogy being used by leadership MOOCs and provides a better understanding surrounding the current state of leadership MOOCs as a form of distance education.
Methods
An exploratory quantitative descriptive content analysis was the research method chosen to best address the questions of this study. According to Neuendorf (2016), content analysis is a summarizing form of quantitative analysis of messages that follows the standards of the scientific method (including reliability and validity) and are not limited to the types of variables that may be measured or in the way the context of the messages is presented. Prior leadership research has paved the way for content analysis to shed light on avoided areas of segmented knowledge in a manner that provides a clearer representation of what is occurring in various academic areas (Insch, Moore, & Murphy, 1997; Marra, Moore, & Klimczak, 2004; Morris, 1994).
Content Analysis Description. While content analysis may be quantitative or qualitative in nature (Creswell & Creswell, 2017), a descriptive content analysis allows the researcher to examine the content in question while using their findings to provide a descriptive summary of the content that was investigated in a quantitative manner (Neuendorf, 2016). Descriptive analysis provides a sense of clarity to a study as they focus directly on the reporting of data in an almost archival fashion (Neuendorf, 2016). Parry and colleagues (2013) discuss content analysis through a leadership lens as a primarily qualitative method but point out that when used in a summative content analysis approach, the line teeters toward quantitative since investigators are establishing numerical counts to test a theoretical aspect.
This study examines the manifest content of messages existing in MOOC course descriptions, available syllabi, and other descriptive information. The goal is to compile a list of leadership-based pedagogy used by instructors in the delivery of leadership MOOCs, and to understand the key characteristics as they apply to MOOCs. The nature of this survey is cross-sectional (Ary, et al., 2018; Creswell & Creswell, 2017) by using the observational data provided by the leadership MOOC analysis at one point in time. In previous leadership work, Weber (1990) recommends single classification, classifying each item in the category where it best fits, even when carrying weaker attributes of another category. More recent work in leadership has either supported or utilized this same single classification approach (Boréus & Bergström, 2017; Klenke, 2016; Spohn, 2018). This method was chosen to replicate prior leadership studies which investigate instructional methods and related content (Jenkins, 2016).
Data Analysis and Coding. The codebook and codesheet were derived from the list of online leadership course instructional methods and assessments provided by Jenkins (2016) and the MOOC pedagogy provided by Glance and colleagues (2013). The major findings were evaluated from a pilot study and the information available in the descriptive content on the MOOC platforms and on individual MOOC homepages. The pilot study results found better alignment with the strategies used in the online leadership strategies research (Jenkins, 2016), thus the decision was made to incorporate this more recent list of instructional pedagogy into the current research study. The goal of quantitative content analysis is to produce counts of the frequency of measurements or other variables that appear within the bounds of the study (Fink, 2009), thus diminishing the level of bias that can occur under certain content analysis models. Insch, Moore, and Klimczak (1997) also state that content analysis reduces the potential for research bias and demand artifacts.
To field test and provide coding reliability in the pilot study, a coding sample of six MOOCs not used in the pilot analysis were given to another researcher not associated with the project. The researcher was asked to evaluate the coding sample of MOOCs using the codebook and codesheet by evaluating the leadership pedagogical methods, MOOC pedagogical methods, and a determination of cMOOC or xMOOC type. Coding for other information provided in the MOOC descriptions accounted for course title, professor instructing the course, university affiliation, course cost, category/tag words, course level, length of course, estimated effort, final exam or project, certificate granted, certificate price, language (search parameters set to English), and leadership topic course and curricular design. The author and one additional researcher tested a random probability sample of the six leadership-related MOOCs for intercoder reliability. As is customary in similar research situations, all coders received the same cases to code for reliability purposes (Neuendorf, 2016). In order to determine reliability between two coders that have evaluated the same data, Cohen’s Kappa (Cohen, 1960; Fleiss, 1971) was used to evaluate chance agreement (Neundorf, 2016). This process of data comparison is a favorable method when using a checklist in the research data set (Berry & Mielke, 1988).
After completion of the subsample, Cohen’s kappa was calculated based on the coded findings. Three of the five preliminary categories returned with a result between .84 and .97, signifying strong and a near perfect agreement when accounting for chance (McHugh, 2012). The two categories labeled “certificate offered,” which resulted in a .5 (indicating weak agreement), and “pedagogy,” with a result of .23 (indicating minimal agreement), returned percentage agreements below the recommended levels of .70 among coders. In order to improve reliability and validation of the findings from the pilot, steps were taken to develop phrasing of the reconfiguration of the variables with better defined categories (Fink & Gantz, 1996). Enhancements to the codebook explanations were employed to increase reliability of the study in a manner to secure future reliability for this current research.
While it is recommended that multiple coders be used for content analysis, researchers have demonstrated that a single-coder system is appropriate if the coder has undergone a sufficient period of training (Milne & Adler, 1999). The reliability achieved in the pilot sample demonstrates that an acceptable level of reliability has been reached before a single coder begins their analysis (Milne & Adler, 1998). Single-coder systems have been utilized in previous studies and recommendations make single coding possible (Camprubi & Coromina, 2016; Drisko & Maschi, 2015; Krippendorf, 2018; Pennings & Keman, 2002). Based upon these recommendations and examples, the work done in the pilot study to address reliability measures for a single coder system found strong and nearly perfect agreement (McHugh, 2012).
Population and Sample Description. The top six MOOC platforms (as identified by Reviews.com) were Coursera, EdX, FutureLearn, Cognitive Class, Udacity, and iversity. Only Coursera, EdX, and FutureLearn returned MOOC course results by using a “leadership” keyword search in the search bar for each platform. The term “leadership” captured the theme of the study and provided a consistent term to bind the study population together. Udacity, iversity, and Cognitive Class returned zero responses to the “leadership” search and were therefore eliminated from the population. In an effort to gain a broader examination of leadership MOOCs, Canvas.net and Stanford Online were added. These platforms fell within the top ten platforms on the initial Reviews.com list. In addition, the MOOCs housed on these platforms represented U.S and Canadian universities, which was part of the population description from the Jenkins (2016) online leadership pedagogy study. In order to include a broader range of MOOCs for evaluation, the decision was made to add these platforms to the study.
After initial “leadership” searches were conducted on each platform, MOOC courses were eliminated from the population if they were not offered in English, nor taught by a U.S. or Canadian university, a theme replicated from a sample in the Jenkins (2016) study. 96 leadership MOOCs remained. Therefore, to address problems with sampling online content that may change (McMillan, 2000), all available 96 leadership MOOCs that met the criteria were evaluated when the search was conducted. Coursera resulted in the greatest number of leadership MOOCs, as there were more available MOOCs present at the time of the initial search. Using the inclusion criteria for the study, the population of 96 leadership MOOCs were used for data analysis. Table 1 captures the population and matriculation process of the leadership MOOC platforms.
Table 1
Leadership MOOCs Population Description and Matriculation
MOOC Platform |
Initial Search Number |
Final Population |
|
Coursera | 316 | 68 | |
FutureLearn | 89 | 4 | |
EdX | 119 | 19 | |
Canvas.net | 2 | 2 | |
Stanford Online | 19 | 3 | |
n= 96 |
Through the content analysis process, each selected MOOC was evaluated by means of the criterion of established leadership education and MOOC pedagogy. After evaluating the collected data, this coded information was used in a narrative process (Krippendorff, 2018) to establish a list of leadership MOOC pedagogy and instructional methods, to test the MOOC pedagogy offered by Glance and associates (2013), and to create a descriptive understanding of leadership MOOC characteristics.
Results
The data analysis for this section was used to answer the research question, What instructional pedagogies are used in leadership-oriented MOOCs? This section will address the results from instructional strategies and instructional assessment strategies provided by the previous research (Jenkins, 2016).
Instructional Strategies. Overall, MOOC instructors are using a variety of instructional strategies. On average, each evaluated leadership MOOC used 6.5 instructional strategies. Table 2 presents the frequency data for the overall population and each evaluated platform. Over half of the studied course descriptions indicate that instructors are using reading, online lecture, problem-based learning, online formative quizzes, interactive presentations, and self-assessments and instruments. Reading emerged as the top instructional strategy with 89 MOOCs, or 92.7% of the population, utilizing it for instructional pedagogical use. Reading was not included in the online pedagogy instructional strategy from the previous Jenkins (2016) study. Reading emerges as an instructional strategy that was not identified in previous research but was prevalent in a fundamental way in this MOOC-related study. Table 3 reflects emerging instructional pedagogy found in the population.
Table 2
Frequency of Instructional Strategies in Leadership MOOCs
All MOOCs | Coursera | EdX | FutureLearn Canvas.Net Stanford Onl. | |||||
Instructional Strategies | n | % of sample | n | % of sample | n | % of sample | n | % of sample |
Reading** | 89 | 92.7 | 68 | 100.0 | 18 | 94.7 | 3 | 33.3 |
Online Lecture | 87 | 90.6 | 68 | 100.0 | 13 | 68.4 | 6 | 66.7 |
Problem-Based Learning | 66 | 68.8 | 47 | 69.1 | 15 | 78.9 | 4 | 44.4 |
Online Formative
Quizzes |
52 |
54.2 |
47 |
69.1 |
5 |
26.3 |
0 |
0.0 |
Self-Assessments &
Instruments |
52 |
54.2 |
37 |
54.4 |
13 |
68.4 |
2 |
22.2 |
Interactive Presentations | 49 | 51.0 | 47 | 69.1 | 1 | 5.3 | 1 | 11.1 |
Case Study/Studies | 47 | 49.0 | 39 | 57.4 | 7 | 36.8 | 1 | 11.1 |
Interview with a Leader
Video** |
31 |
32.3 |
28 |
41.2 |
0 |
0.0 |
3 |
33.3 |
Media Clips | 28 | 29.2 | 22 | 32.4 | 3 | 15.8 | 3 | 33.3 |
Reflective Journals | 27 | 28.1 | 20 | 29.4 | 5 | 26.3 | 2 | 22.2 |
Group Discussions | 26 | 27.1 | 20 | 29.4 | 3 | 15.8 | 3 | 33.3 |
Creative Mapping** | 22 | 22.9 | 21 | 30.9 | 1 | 5.3 | 0 | 0.0 |
Class Polling and Surveys | 20 | 20.8 | 20 | 29.4 | 0 | 0.0 | 0 | 0.0 |
Social Networking | 16 | 16.7 | 13 | 19.1 | 3 | 15.8 | 0 | 0.0 |
Discussion Boards:
Student-Lead |
15 |
15.6 |
13 |
19.1 |
1 |
5.3 |
1 |
11.1 |
Computer-based Learning | 9 | 9.4 | 5 | 7.4 | 4 | 21.1 | 0 | 0.0 |
Student-Peer Evaluations | 9 | 9.4 | 9 | 13.2 | 0 | 0.0 |
0 |
0.0 |
Debrief** | 7 | 7.3 | 7 | 10.3 | 0 | 0.0 | 0 | 0.0 |
Online Collaborative Projects | 4 | 4.2 | 4 | 5.9 | 0 | 0.0 | 0 | 0.0 |
Discussion Boards:
Instructor-Lead |
2 |
2.1 |
2 |
2.9 |
0 |
0.0 |
0 |
0.0 |
Discussion Boards:
Shared Instructor-Student |
2 |
2.1 |
1 |
1.5 |
1 |
5.3 |
0 |
0.0 |
Discussion with Coach** | 2 | 2.1 | 0 | 0.0 | 2 | 10.5 | 0 | 0.0 |
Scavenger Hunts | 1 | 1.0 | 1 | 1.5 | 0 | 0.0 | 0 | 0.0 |
Student Questions/Activities | 1 | 1.0 | 1 | 1.5 | 0 | 0.0 | 0 | 0.0 |
Live Online Lecture** | 1 | 1.0 | 1 | 1.5 | 0 | 0.0 | 0 | 0.0 |
Participation in Research** | 1 | 1.0 | 1 | 1.5 | 0 | 0.0 | 0 | 0.0 |
Service Learning** | 1 | 1.0 | 1 | 1.5 | 0 | 0.0 | 0 | 0.0 |
Virtual Field Trip** | 1 | 1.0 | 1 | 1.5 | 0 | 0.0 | 0 | 0.0 |
Online Debates | 0 | 1.0 | 0 | 0.0 | 0 | 0.0 | 0 | 0.0 |
Note: All Leadership MOOC Population n=96; ** Denotes emerging strategies initially calculated under “other”
Table 3
Emerging Instructional Strategies in Leadership MOOCs
No. | Instructional Strategy | Description |
1 | Reading | Students read provided articles, course textbooks, eBooks, or suggested resources in order to gain an understanding of content, course material, and application. |
2 | Dialogues with a Leader | Students engage with written or oral recorded interviews/dialogues/videos with a leader |
3 | Concept Mapping | Students create a visual representation of a theory or concept using keywords, relationships, course material, and meaning making. |
4 | Debrief | Students engage in follow-up discussions via a discussion board or in a writing assignment regarding a simulation or to provide follow-up or clarification. |
5 | Discussion with a Coach | Students engage in professional partnership interactions with a leadership coach to apply and refine course concepts and personal development. |
6 | Live Online Lecture | A live lecture in an online/MOOC course is provided by the instructor to address course material/content and improve cognition and engagement. |
7 | Participation in Research | Students are invited to participate in research focused on course content in order to help with the practical view of concepts. |
8 | Service Learning | Students engage in co-curricular opportunities to serve a community, organization, or population in order to gain life preparation and perspective. |
9 | Digital Field Trip | Students interact with a video to explore a destination linked to course content. |
Note: Sources include Storme, Vansieleghem, Devleminck, Masschelin, & Simons, 2016; Bali, 2014; Durkin, 1978; Bloom, 1968; Anderson & Pearson, 1984; Renshaw, 2004; Kvale, 2006; Bass, 2014; Novak, 2010; Breen & Martin, 2018; Sundheim, 2015; Robertson, 2009; Lawrence, Dunn, & Weisfeld-Spolter, 2018; Rosell, Beck, Luther, Goedert, Shore, & Anderson, 2005; Spickard, Alrajeh, Cordray, & Gigante, 2002; Pratton & Hales, 1986; Howard, 1998; Buschlen & Dvorak, 2011; Jacobson, Militello, & Baveye, 2009; Capobianco, Loizzo, & Burgess, 2009.
Assessment Strategies. Assessment strategies are the manner in which instructors evaluate student performance. An average of 4.2 assessment strategies were described for each reviewed MOOC. Quizzes, Creative Individual Projects, and Case Study Analysis were the three most frequently listed assessment pedagogies in the course descriptions and documents. Although Discussion Boards were the top assessment in previous online course studies (Jenkins, 2016), they were observed or described in only 31.3% of the leadership MOOCs in the current study. While all assessment strategies were present in the study analysis, only one leadership MOOC used Group Projects/Presentations. Table 4 presents the frequency of assessment strategies found in leadership MOOCs. Table 5 represents emerging assessment strategies found in leadership MOOCs.
Table 4
Frequency of Assessment Strategies in Leadership MOOCs
Assessment Strategies | All MOOCs | Coursera | EdX | FutureLearn Canvas.Net Stanford Online | ||||
n | % of sample | n | % of sample | n | % of sample | n | % of sample | |
Quizzes | 55 | 57.3 | 48 | 70.6 | 3 | 15.8 | 1 | 11.1 |
Creative Project** | 45 | 46.9 | 37 | 54.4 | 7 | 36.8 | 2 | 22.2 |
Case Study Analysis | 42 | 43.8 | 32 | 47.1 | 7 | 36.8 | 3 | 33.3 |
Self-Evaluation | 36 | 37.5 | 23 | 33.8 | 10 | 52.6 | 3 | 33.3 |
Discussion Boards | 30 | 31.3 | 24 | 35.3 | 1 | 5.3 | 5 | 55.6 |
Individual Leadership
Development Plans |
30 | 31.3 | 21 | 30.9 | 9 | 47.4 | 0 | 0.0 |
Reflective Journals | 30 | 31.3 | 20 | 29.4 | 8 | 42.1 | 2 | 22.2 |
Read & Respond | 27 | 28.1 | 26 | 38.2 | 1 | 5.3 | 0 | 0.0 |
Participation | 20 | 20.8 | 14 | 20.6 | 5 | 26.3 | 1 | 11.1 |
End of Course
Survey** |
17 | 17.7 | 16 | 23.5 | 0 | 0.0 | 1 | 11.1 |
Short Papers | 14 | 14.3 | 11 | 16.2 | 3 | 15.8 | 0 | 0.0 |
Reflective Project** | 12 | 12.5 | 12 | 17.6 | 0 | 0.0 | 0 | 0.0 |
Exams | 10 | 10.4 | 9 | 13.2 | 0 | 0.0 | 1 | 11.1 |
Student- Peer
Assessments |
9 | 9.4 | 9 | 13.2 | 0 | 0.0 | 0 | 0.0 |
Individual Research
Projects/ Presentations |
6 | 6.3 | 4 | 5.9 | 3 | 15.8 | 0 | 0.0 |
Major Writing
Projects |
6 | 6.3 | 6 | 8.8 | 0 | 0.0 | 0 | 0.0 |
Interview of a Leader | 4 | 4.2 | 3 | 4.4 | 1 | 5.3 | 0 | 0.0 |
Mapping** | 4 | 4.2 | 4 | 5.9 | 0 | 0.0 | 0 | 0.0 |
Digital Storytelling | 3 | 3.1 | 2 | 2.9 | 0 | 0.0 | 1 | 11.1 |
Online/ ePortfolio | 2 | 2.1 | 2 | 2.9 | 0 | 0.0 | 0 | 0.0 |
Group Projects /
Presentation |
1 | 1.0 | 0 | 0.0 | 1 | 5.3 | 0 | 0.0 |
Note: All Leadership MOOC Population n=96; ** Denotes an emerging instructional assessment strategy
Table 5
Emerging Assessment Strategies in Leadership MOOCs
No. | Assessment Strategy | Definition |
1 | Creative Projects | Students are graded on a project that targets their creative ability to address a content area problem. |
2 | End of Course Survey | Students complete a course evaluation survey for credit. |
3 | Reflective Project | Students reflect on their experiences and course content in order to evaluate an issue or concept. |
4 | Concept Mapping Analysis | Students create a visual representation of a
theory or concept using keywords, relationships, course material, and meaning making and pair with a short concept paper. |
5 | Participation in Leadership Research | Students must participate in an instructor-led research project, usually focused on the course/course concept. |
Note: Sources include Ellis & Barrs, 2008; Frederiksen & Knudsen, 2017; Buckley & Marion, 2011; McCombs, 1997; Boud & Walker, 1998; Stoyanov, Sloep, De Bie, & Hermans, 2014; McClure, Sonak, & Suen, 1999; Rosell, Beck, Luther, Goedert, Shore, & Anderson, 2005.
Additional Leadership MOOC Characteristics. Additional evaluation of MOOC characteristics produced an understanding of the current state of leadership MOOCs. Traditional xMOOCs focus on the knowledge to be learned, whereas cMOOCs focus on the building of the community of learners and their use of the resources on hand within the bounds of the internet. xMOOCs comprised over 90% of the evaluated leadership MOOCs while cMOOCs made up 3.1% of the evaluated MOOCs. Nearly six percent of the population could not be determined based on the platform’s course descriptions. Over half (55.6%) of the population of the MOOCs from the FutureLearn, Canvas.net, and Stanford Online platform trio could not be determined.
As a summation of the frequency found in analysis, the greatest frequency of leadership MOOCs are offered as beginner courses and occur over the course of 2-4 weeks. Leadership MOOCs suggest 2-4 hours per week of course work. Leadership MOOCs are generally available for free or at a cost if learners are enrolled in the course for credit toward a certificate or specialization. The average leadership MOOC cost listed on eighteen EdX course descriptions was $131.88 per course, with a range of $49-$249 per course listed. The indication of “free” and “course cost” were provided in all nine of the evaluated FutureLearn, Canvas.net, and Stanford Online data, with only one MOOC presenting it had a fee associated with it to take the course but was not listed.
The mean enrollment number for leadership MOOCs listed on the Coursera platform course offerings was 22,749 students. The standard deviation from the Coursera population was 32,764.26. This indicates a wide range of enrollments across the Coursera platform in leadership MOOCs. The use of keywords or skills words also serve as a quick summary of what learners can gain from completing the course at hand. One hundred and forty-four different words were coded during analysis. Table 6 provides a summation of the evaluated characteristics in this study.
Table 6
Characteristics of Leadership MOOCs
Evaluated Characteristics | Summation |
MOOC Structure | xMOOC (87)
cMOOC (3) Cannot be determined (6) |
Course Level | Beginner |
Length of Course | 2-4 weeks |
Expected Student Effort | 2-4 hours per week |
Certificates Offered | Yes, with completion of series (81)
Yes (4) No (10) |
Cost | Free (non-credit)
$131.88 average (for credit) |
Enrollment | 22,749 (Average); SD=32,764.26 |
Keywords/ Topic Areas | 144 words/topical areas |
Note: Total Leadership MOOC Population n=96
MOOC Leadership Content. A total of 144 leadership content areas, skills to be gained, or additional keywords indicating the breadth of content is wide and diverse in nature. An average of 1.5 skill or keywords per MOOC course informs students about the type of leadership topics or skills gained from enrolling in a leadership MOOC. The variety of words show the content covered by leadership MOOCs in terms of personal and professional development. These topical areas, skills, and keywords help in directing learners toward certain MOOCs and serves as a quick summary of what learners can gain from completing the course at hand. One hundred and forty-four different words were coded during analysis. Nearly every MOOC course listed “keywords” or “skills to be gained” in their descriptions. Figure 2 provides a list of words that appeared most frequently (with more than four mentions) across the course descriptions.
Figure 2
Frequency of Leadership Content/ Skills/ Keywords in Leadership MOOCs
Limitations. This study relied heavily on the course syllabus and course descriptions made available across the platforms and were used as an artifact of the educational process. They may not always be the best source to gather evidence of pedagogy. The availability of leadership MOOCs offered through various platforms can seemingly change on a day-to-day basis. Because of the ever-changing nature of MOOCs and online content, this can be a consideration of future study that is outside of the researcher control. An additional limitation for consideration for this research might be focused on instructor intent in the leadership MOOC course descriptions included across platforms. This researcher assumed the use of pedagogies and the descriptions used captured the essence of instruction and content available. The evaluated MOOCs represented 17.6% of the initial population of leadership MOOCs found on the platforms before the results were filtered to provide a list of leadership MOOCs taught in English and by universities in the U.S. or Canada. This shows a greater scope of the MOOCs available worldwide. A limitation for this study might be that the information presented is from a Western perspective on leadership and derivative of the U.S. and Canadian education systems. There is much breadth and depth of knowledge that exists from examining leadership MOOCs on a global scale. A global inclusion research process could reveal new characteristics and present a more well-rounded snapshot of leadership MOOCs.
Discussion/ Implications. The results from the current content analysis study offer implications to the future of leadership MOOCs. Previous work has considered instructor philosophy centered on student engagement in assessment selection for MOOC courses and the potential impact for increased student course involvement (Anderson, et al., 2014). Jenkins’s seminal research in leadership pedagogy (2012, 2016) has done much for the field in providing an understanding of how more traditional leadership courses are delivered through instructional and assessment strategies. The results from this study indicate a wide range of leadership content offered through MOOCs, which employ a variety of instructional strategies and assessments, making them different from traditional classroom and online leadership learning spaces.
Borrowing from previous work (Reich, 2015), it is essential that the advancement of MOOC research go beyond the number of clicks a MOOC gets. MOOC research must examine the pedagogy of instruction by investigating individual courses and comparing contextual information. This research does meet those suggestions through an evaluation of instructional and assessment strategies used across leadership MOOCs, and in providing a more general understanding of the current snapshot of leadership MOOCs. It is necessary that additional work of this nature be conducted.
The appearance of instructional and assessment strategies not discussed in previous leadership studies requires further investigation. For example, the presence of reading as an instructional strategy raises questions regarding its use in other leadership coursework. The importance of reading in MOOCs was clear and stated throughout the course descriptions and syllabi as being crucial to the learning process and course objectives for MOOC learners. The notion of reading as a pedagogy may be taken for granted yet needs to be evaluated further for its impact on higher education and the leadership education field, not just across MOOC platforms. As universities investigate the cost of textbooks and the impact on students, this emergence allows educators to evaluate how reading is being used across the current and future courses. Additional found pedagogy can also be addressed and refined given its role in MOOC education.
The present research serves as a base for future study on a variety of topics related to leadership MOOCs. Scholarship that examines the instructor’s role in developing and managing a leadership MOOC would be valuable in terms of linking pedagogy to instructor intent and providing a deeper understanding of the composition of leadership MOOCs. This scholarship would expand the knowledge regarding leadership MOOCs and leadership education in general.
This research does not provide a prescribed layout for leadership MOOC instruction, but instead offers an understanding of what strategies are currently being utilized across leadership-oriented MOOCs. Leadership education instructors can consider if their selected strategies are connecting to the learning objectives and overall course goals.
An additional study recommendation centers on conducting similar content analysis research on non-University leadership MOOCs. Websites like Udemy offer an expansive list of leadership MOOCs that fall under the popular press distinction and coordinate with leadership books or other personal development sources. It could provide an interesting contrast to see how academic versus non-academic are taught and executed. This research may provide a richer and more robust understanding of instructional pedagogy used to offer leadership MOOCs. It would also further define characteristics and content covered through these types of leadership MOOCs outside of academia and help to develop a more theoretical approach to developing and teaching future leadership MOOCs.
Conclusion. This study provides a unique understanding of the current state of leadership MOOCs in terms of characteristics, content, and the use of pedagogy utilized in instruction. The study revealed a content-focused approach to instruction and a real-world application approach to assessing student learning. The study also revealed the role of traditional MOOC characteristics and brought into question the use of peer-assessment in this leadership education space. This directly connects to the self-paced role students find in an xMOOC format over the connectivist approach, or cMOOC format, found in the literature. The state of leadership MOOCs suggests that the focus remains on the content and not on the connection to a community of learners. Some of the definitive characteristics of leadership MOOCs reveal their role in educating a wide array of students over the course of a few weeks. Due to the varied content and topical areas covered across MOOCs, the leadership education industry is meeting the needs and diversity of thought related to coursework and professional development opportunities.
The investigation of leadership MOOCs broadly could result in theoretical discoveries regarding the leadership field. While we do now have an initial understanding of the pedagogy used for leadership MOOC instruction, further research could help develop how leadership MOOC systems interact with learners and the resulting implications of how this development and interaction is defined.
Leadership educators must raise the level of engagement and research centered on leadership MOOCs to address challenges of the workforce and in the personal leadership development of learners. This study provides a glimpse inside the growing leadership industry’s representation among the MOOC population, particularly for professional development. The results allow us to enter a discussion about the future of leadership education and a heightened awareness of the certificate granting nature of leadership education across a professional development and digital spectrum.
References
Aerts, D., Apostel, L., De Moor, B., Hellemans, S., Maex, E., Van Belle, H., & Van der Veken,
- (1994). World Views: From fragmentation to integration. VUB Press.
Allen, S. J., & Hartman, N. S. (2008). Leadership development: An exploration of sources of
learning. SAM Advanced Management Journal, 73(1), 10.
Anderson, T. (Ed.). (2008). The Theory and Practice of Online Learning. Athabasca University Press, p. 397-399.
Anderson, T., & Dron, J. (2011). Three generations of distance education pedagogy. The
International Review of Research in Open and Distributed Learning, 12(3), 80-97.
http://www.irrodl.org/index.php/irrodl/article/view/890
Anderson, A., Huttenlocher, D., Kleinberg, J., & Leskovec, J. (2014, April). Engaging with
massive online courses. In Proceedings of the 23rd International Conference on World Wide Web (pp. 687-698). ACM.https://doi.org/10.1145/2566486.2568042
Anderson, R. C., & Pearson, P. D. (1984). A schema-theoretic view of basic processes in reading
comprehension. Handbook of Reading Research, 1, 255-291.
Ary, D., Jacobs, L. C., Irvine, C. K. S., & Walker, D. (2018). Introduction to Research in
Education. Cengage Learning.
Bali, M. (2014). MOOC pedagogy: Gleaning good practice from existing MOOCs. Journal of
Online Learning and Teaching, 10(1), 44.
Bannier, B. J. (2016). Global trends in transnational education. International Journal of
Information and Education Technology, 6(1), 80. https://doi.org/10.7763/IJIET.2016.V6.663
Barton, D. (2012). Young, gifted and slack: The skills gap must be bridged if the world is to
avoid dire consequences. The Economist, 21.
Berry, K. J., & Mielke Jr, P. W. (1988). A generalization of Cohen’s kappa agreement measure to interval measurement and multiple raters. Educational and Psychological Measurement, 48(4), 921-933. doi: https://doi.org/10.1177/0013164488484007
Bloom, L. M. (1968). Language development: Form and function in emerging grammars.
Bonk, C. J., & Graham, C. R. (2012). The Handbook of Blended Learning: Global Perspectives,
Local Sesigns. John Wiley & Sons.
Bonk, C. J., & Zhang, K. (2008). Empowering Online Learning: 100+ Activities for Reading,
Reflecting, Displaying, and Doing. John Wiley & Sons.
Breen, J. M., & Martin, J. (2018). Teaching leadership research courses online at the doctoral
level: Why we do it and how it works. In Advancing Doctoral Leadership Education
Through Technology. Edward Elgar Publishing.
Boréus, K., & Bergström, G. (2017). Analyzing Text and Discourse: Eight Approaches for the Social Sciences. Sage Publications
Boud, D., & Walker, D. (1998). Promoting Reflection in Professional Courses: The
Challenge of Context. Studies in Higher Education, 23(2), 191-206. https://doi.org/10.1080/03075079812331380384
Buckley, K., & Marion, S. (2011). A survey of approaches used to evaluate educators in non-
tested grades and subjects. Dover, NH: National Center for the Improvement of Educational Assessment. Retrieved February 21, 2021.
Buschlen, E., & Dvorak, R. (2011). The Social Change Model as Pedagogy:
Examining Undergraduate Leadership Growth. Journal of Leadership Education, 10(2). https://doi.org/10.12806/V10/I2/RF2
Butcher, N., & Wilson-Strydom, M. (2013). A Guide to Quality in Online Learning. Academic
Partnerships. Retrieved on December 12, 2013.
https://www.nba.co.za/sites/default/files/NewBooklet10_single.pdf
Calonge, D. S., & Shah, M. A. (2016). MOOCs, Graduate Skills Gaps, and Employability: A
Qualitative Systematic Review of the Literature. The International Review of Research in Open and Distributed Learning, 17(5). https://doi.org/10.19173/irrodl.v17i5.2675
Camprubí, R., & Coromina, L. (2016). Content Analysis in Tourism Research. Tourism Management Perspectives, 18, 134-140. doi: https://doi.org/10.1016/j.tmp.2016.03.002
Capobianco, B., Loizzo, J., & Burgess, W. (2009, June). Lesson Learned from Integrating
Electronic Fields in the Science Classroom. In EdMedia+ Innovate Learning (pp. 2747-2751). Association for the Advancement of Computing in Education (AACE).
Chunoo, V., & Osteen, L. (2016). Purpose, Mission, and Context: The Call for Educating Future
Leaders. New Directions for Higher Education, 2016(174), 9-20.
https://doi.org/10.1002/he.20185
Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37-46.
Conger, J. A. (1992). Learning to Lead: The Art of Transforming Managers into Leaders.
Jossey-Bass Inc., Pub.
Creswell, J. W., & Creswell, J. D. (2017). Research Design: Qualitative, Quantitative, and
Mixed Methods Approaches. Sage Publications.
Daniel, J. (2012). Making Sense of MOOCs: Musings in a Maze of Myth, Paradox and
Possibility. Journal of Interactive Media in Education, 2012(3). https://doi.org/10.5334/2012-18
Djajalaksana, Y. M. (2011). A National Survey of Instructional Strategies Used to Teach
Information Systems Courses: An Exploratory Investigation. Scholar Commons. University of South Florida. https://scholarcommons.usf.edu/cgi/viewcontent.cgi?article=4269&context=etd
Drisko, J. W., & Maschi, T. (2015). Content Analysis. Pocket Guides to Social Work. doi: https://doi.org/10.1093/acprof:oso/9780190215491.001.0001
Durkin, D. (1978). What Classroom Observations Reveal about Reading Comprehension
Instruction. Reading Research Quarterly, 481-533. https://doi.org/10.1598/RRQ.14.4.2
Egloffstein, M., & Ifenthaler, D. (2017). Employee Perspectives on MOOCs for Workplace
Learning. Tech Trends, 61(1), 65-70. https://doi.org/10.1007/s11528-016-0127-3
Ellis, S., & Barrs, M. (2008). The Assessment of Creative Learning. Creative Learning, 73-89.
Esposito, A. (2012). Research Ethics in Emerging Forms of Online Learning: Issues Arising
from a Hypothetical Study on a MOOC. Electronic Journal of e-Learning, 10(3), 315-325.
Fink, E. L. (2009). The FAQs on Data Transformation. Communication Monographs, 76(4), 379-397. doi: https://doi.org/10.1080/03637750903310352
Fink, E. J., & Gantz, W. (1996). A content analysis of three mass communication research traditions: social science, interpretive studies, and critical analysis. Journalism & Mass Communication Quarterly, 73(1), 114-134. Doi: https://doi.org/10.1177/107769909607300111
Fleiss, J. L. (1971). Measuring nominal scale agreement among many raters. Psychological Bulletin, 76(5), 378. doi: https://doi.org/10.1037/h0031619
Fletcher Jr, E. C., Djajalaksana, Y., & Eison, J. (2012). Instructional strategy uses of faculty in
career and technical education. Journal of Career and Technical Education, 27(2), 69-83.
Frederiksen, M. H., & Knudsen, M. P. (2017). From creative ideas to innovation performance:
The role of assessment criteria. Creativity and Innovation Management, 26(1), 60-74.
https://doi.org/10.1111/caim.12204
Gifford, G. T. (2010). A modern technology in the leadership classroom: Using blogs for critical
thinking development. Journal of Leadership Education, 9(1), 165-172.
https://doi.org/10.12806/V9/I1/AB2
Glance, D. G., Forsey, M., & Riley, M. (2013). The pedagogical foundations of massive open online courses. First Monday, 18(5). doi: https://doi.org/10.5210/fm.v18i5.4350
Guthrie, K. L., & McCracken, H. (2010). Teaching and learning social justice through online
service-learning courses. The International Review of Research in Open and Distributed
Learning, 11(3), 78-94.
Headrick, J. & Luethke, T. (2018, July 11). A Discussion of MOOCs and Fully Online Courses: Recommendations for Facilitating Inclusion and Connection through Web-based Learning. Paper presented at the Association of Leadership Educators: Leadership Innovation and Inclusion in the City of Big Shoulders, Chicago, IL. Retrieved from: tors.org/resources/Documents/ALE%202018%20Conference%20Proceedings%20-%20Chicago.pdf
Hew, K. F., & Cheung, W. S. (2014). Students’ and instructors’ use of massive open online
courses (MOOCs): Motivations and challenges. Educational Research Review, 12, 45-58.
https://doi.org/10.1016/j.edurev.2014.05.001
Howard, J. P. (1998). Academic service learning: a counter normative pedagogy. New Directions
for Teaching and Learning, 73, 21-29. https://doi.org/10.1002/tl.7303
Hoy, M. B. (2014). MOOCs 101: an introduction to massive open online courses. Medical reference services quarterly, 33(1), 85-91.
Insch, G. S., Moore, J. E., & Murphy, L. D. (1997). Content analysis in leadership research:
Examples, procedures, and suggestions for future use. The Leadership Quarterly, 8(1), 1-25. https://doi.org/10.1016/S1048-9843(97)90028-X
Jacobson, A. R., Militello, R., & Baveye, P. C. (2009). Development of computer-assisted virtual
field trips to support multidisciplinary learning. Computers & Education, 52(3), 571-580.
https://doi.org/10.1016/j.compedu.2008.11.007
Jenkins, D. (2012). Exploring signature pedagogies in undergraduate leadership education.
Journal of Leadership Education, 11 (1). https://doi.org/10.12806/V11/I1/RF1
Jenkins, D. (2016). Teaching Leadership Online: An Exploratory Study of Instructional and
Assessment Strategy Use. Journal of Leadership Education, 15(12).
https://doi.org/10.12806/V17/I1/R2
Jenkins, D. M., Endersby, L., & Guthrie, K. L. (2015). Leadership education 2050: Changing the
spaces and faces of experience. Leadership 2050, 127-139.
Kaplan, A. M., & Haenlein, M. (2016). Higher education and the digital revolution: About
MOOCs, SPOCs, social media, and the Cookie Monster. Business Horizons, 59(4), 441-450. https://doi.org/10.1016/j.bushor.2016.03.008
Karnouskos, S. (2017). Massive open online courses (MOOCs) as an enabler for competent
employees and innovation in industry. Computers in Industry, 91, 1-10.
Klenke, K. (2016). Qualitative research in the study of leadership. Emerald Group
Publishing, Limited. doi: https://doi.org/10.1108/9781785606502
Krippendorff, K. (2018). Content analysis: An introduction to its methodology. Sage.
Lakshmi, R. (2013). In India, students’ aspirations are misaligned with job market. The
Washington Post.
Lawrence, E., Dunn, M. W., & Weisfeld-Spolter, S. (2018). Developing leadership potential in
Graduate students with assessment, self-awareness, reflection and coaching. Journal of
Management Development, 37(8), 634-651. https://doi.org/10.1108/JMD-11-2017-0390
Marra, R. M., Moore, J. L., & Klimczak, A. K. (2004). Content analysis of online discussion
forums: A comparative analysis of protocols. Educational Technology Research and
Development, 52(2), 23. https://doi.org/10.1007/BF02504837
Marshall, S. (2013). Evaluating the strategic and leadership challenges of MOOCs. Journal of
Online Learning and Teaching, 9(2), 216.
McClure, J. R., Sonak, B., & Suen, H. K. (1999). Concept map assessment of classroom
learning: Reliability, validity, and logistical practicality. Journal of Research in Science
Teaching: The Official Journal of the National Association for Research in Science
Teaching, 36(4), 475-492. https://doi.org/10.1002/(SICI)1098-2736(199904)36:4<475::AID-TEA5>3.0.CO;2-O
McCombs, B. L. (1997). Self-assessment and reflection: Tools for promoting teacher changes
toward learner-centered practices. NASSP Bulletin, 81(587), 1-14.
https://doi.org/10.1177/019263659708158702
McHugh, M. L. (2012). Interrater reliability: the kappa statistic. Biochemia Medica, 22(3), 276-282.
McMillan, S. J. (2000). The microscope and the moving target: The challenge of applying
content analysis to the World Wide Web. Journalism & Mass Communication Quarterly, 77(1), 80-98. https://doi.org/10.1177/107769900007700107
Meister, J. (2013, August 13). How MOOCs will revolutionize corporate learning and
development. Forbes.
Miller, M. D. (2014). Minds online: Teaching effectively with technology. Harvard University Press. doi: https://doi.org/10.4159/harvard.9780674735996
Miller, T. W., & Miller, J. M. (2001). Educational leadership in the new millennium: a vision for
- International Journal of Leadership in Education, 4(2), 181-189.
https://doi.org/10.1080/13603120120806
Milne, M. J., & Adler, R. W. (1999). Exploring the reliability of social and environmental disclosures content analysis. Accounting, Auditing & Accountability Journal, May 1999.
Mohamed, M. H., & Hammond, M. (2018). MOOCs: a differentiation by pedagogy, content and
assessment. The International Journal of Information and Learning Technology, 35(1), 2-11. https://doi.org/10.1108/IJILT-07-2017-0062
Moldoveanu, M., & Narayandas, D. (2019). The future of leadership development. Harvard
Business Review, 41, 41.
Morris, R. (1994). Computerized content analysis in management research: A demonstration of
advantages & limitations. Journal of Management, 20(4), 903-931.
https://doi.org/10.1016/0149-2063(94)90035-3
Muilenburg, L. Y., & Berge, Z. L. (2016). Digital Badges as a Motivator in MOOCS: The Carpe
Diem MOOC Experience. In Digital Badges in Education (pp. 252- 262). Routledge.
https://doi.org/10.4324/9781315718569
Neuendorf, K. A. (2016). The Content Analysis Guidebook. Sage.
https://doi.org/10.1002/9781118541555.wbiepc065
Novak, J. D. (2010). Learning, creating, and using knowledge: Concept maps as facilitative
tools in schools and corporations. Routledge. https://doi.org/10.4324/9780203862001
Odom, S. F., Jarvis, H. D., Sandlin, M. R. R., & Peek, C. (2013). Social media tools in the
leadership classroom: Students’ perceptions of use. Journal of Leadership Education, 12(1). https://doi.org/10.12806/V12/I1/R3
Parry, K., Mumford, M. D., Bower, I., & Watts, L. L. (2014). Qualitative and historiometric methods in leadership research: A review of the first 25 years of The Leadership Quarterly. The Leadership Quarterly, 25(1), 132-151.
Pennings, P., & Keman, H. (2002). Towards a new methodology of estimating party policy positions. Quality and Quantity, 36(1), 55-79. doi: https://doi.org/10.1023/A:1014380123135
Phelps, K. (2012). Leadership online: expanding the horizon. New Directions for Student
Services, 140, 65-75. https://doi.org/10.1002/ss.20032
Pratton, J., & Hales, L. W. (1986). The effects of active participation on student learning. The
Journal of Educational Research, 79(4), 210-215. https://doi.org/10.1080/00220671.1986.10885679
Quillen, I. (2013). Why do students enroll in (but don’t complete) MOOC courses? Mind/Shift.
Radford, A. W., Robles, J., Cataylo, S., Horn, L., Thornton, J., & Whitfield, K. E. (2014). The
employer potential of MOOCs: A mixed-methods study of human resource professionals’
thinking on MOOCs. The International Review of Research in Open and Distributed
Learning, 15(5). https://doi.org/10.19173/irrodl.v15i5.1842
Regalado, A. (2013). The most important education technology in 200 years. Technology
Review, 116(1), 61-62.
Reich, J. (2015). Rebooting MOOC research. Science, 347(6217), 34-35.
Reich, J., & Ruipérez-Valiente, J. A. (2019). The MOOC pivot. Science, 363(6423), 130-131.
Renshaw, P. D. (2004). Dialogic learning teaching and instruction. In Dialogic learning (pp. 1-
15). Dordrecht: Springer. https://doi.org/10.1007/1-4020-1931-9_1
Robinson, S., & Ritzko, J. (2009). Podcasts in education: What, why and how?. In Allied
Academies International Conference. Academy of Educational Leadership. Proceedings (Vol. 4, No. 1, p. 38). Jordan Whitney Enterprises, Inc.
Robertson, J. (2009). Coaching leadership learning through partnership. School Leadership and
Management, 29(1), 39-49. https://doi.org/10.1080/13632430802646388
Rosell, M. C., Beck, D. M., Luther, K. E., Goedert, K. M., Shore, W. J., & Anderson, D. D.
(2005). The pedagogical value of experimental participation paired with course
content. Teaching of Psychology, 32(2), 95-99. https://doi.org/10.1207/s15328023top3202_3
Roush, D.M., & Anthony, M.D. (2012, Winter). Leadership pedagogy: Putting theory to practice.
In New Directions for Student Services, 2012(140), 37-51. https://doi.org/10.1002/ss.20030
Shah, D. (2018). By the numbers: MOOCs in 2018 [Web log post]. Retrieved from:
Shah, D. (2019). By the numbers: MOOCs in 2019, Class Central MOOC Report, Accessed 08
February, 2019. Retrieved from https://www.classcentral.com/report/mooc-stats-2019/
Shulman, L. S. (2005). Pedagogies. Liberal Education, 91(2), 18-25.
Spickard, A., Alrajeh, N., Cordray, D., & Gigante, J. (2002). Learning about screening using an
online or live lecture. Journal of General Internal Medicine, 17(7), 540-545.
https://doi.org/10.1046/j.1525-1497.2002.10731.x
Storme, T., Vansieleghem, N., Devleminck, S., Masschelein, J., & Simons, M. (2016). The
emerging pedagogy of MOOCs, the educational design of technology and practices of
study. Journal of Computers in Education, 3(3), 309-328.
https://doi.org/10.1007/s40692-016-0070-5
Stoyanov, S., Sloep, P. B., De Bie, M., & Hermans, V. (2014). Teacher-training, ICT, creativity,
MOOC, Moodle-What pedagogy. Proceedings of Edulearn, 14, 5678-5686.
Sundheim, D. (2015). Debriefing: A Simple Tool to Help Your Team Tackle Tough
Problems. Harvard Business Review.
https://hbr.org/2015/07/debriefing-a-simple-tool-tohelp-your-team-tackle-tough-problems
Wood, M. (2013). Opportunities in online education—Staying ahead of the curve: The case of
the MOOC. Council of Independent Colleges, Presidents Institute, January 6.
https://www.oerknowledgecloud.org/record557