Corey Seemiller, David M. Rosch DOI: 10.12806/V22/I1/R6
Historically, higher education has played a significant role in preparing graduates for critical jobs in the workforce (Labaree, 1997), ensuring they are armed with a wide array of competencies and credentials to navigate their chosen professional contexts. Academic programs, in conjunction with discipline-specific accrediting agencies, focus on teaching technical requirements necessary for graduates to be hired for and be successful in their desired occupations. In addition, many programs also incorporate learning outcomes comprised of competencies necessary for effective functioning in the workforce in general (Seemiller, 2021). However, the types of workforce competencies included in learning outcomes and the depth to which they are focused can vary from program to program and degree level to degree level, given the diversity of academic programs and their ultimate goals. This variance may result in vast differences in broad areas of professional preparation for graduates as they enter the workforce. It is imperative to understand the extent that generalized workforce and leadership competencies are embedded into academic programs’ learning outcomes to ascertain a program’s commitment to developing not just a technically competent workforce but also one that has the skills for success in the workplace in general.
In order to best understand the context of the present study, we begin by describing the role of higher education in professional preparation and competency development necessary to build a trained and credentialed workforce.
Workforce Development in Higher Education. Higher education began in the United States with the initial purpose of educating the elite, “primarily with shaping the mind and character of the ruling class” (Trow, 1973, 7). In the mid-20th century, however, the move to the massification of higher education led to the growing expansion of the number of students in college as well as greater diversity in the student population (Trow, 1973). For example, the post-World War II GI Bill resulted in the growth of the number of higher education institutions to accommodate the massive enrollment of veterans going to college; and, a larger system meant there was more room for non-veteran students as well (Adams, 2000). Subsequent pieces of critical legislation designed to expand higher education access emerged in the following years, with one of the most well-known being the Higher Education Act of 1965.
These shifts away from elite liberal education to a higher education system designed to serve society more broadly resulted in colleges and universities offering more occupational-professional programs (Brint, Riddle, Turk-Bicakci, & Levy, 2005). Indeed, the expansion of occupational-professional programs has been more or less consistent through much of the latter half of the 20th century through the 21st century, primarily as a means to provide students with a career pathway to a good job with an ample salary, in part to pay off student loan debt (Brint, Riddle, Turk-Bicakci, & Levy, 2005).
Today, workforce development programs have emerged as a primary focus in legislative discussions across states. For example, in 2018 alone, bills related to postsecondary education in state legislatures addressed workforce development more than any other issue related to education (National Conference of State Legislatures, 2019). Given such public focus, the lack of systemic study of how workforce development is broadly correlated with higher education learning outcomes seems surprising, especially across different levels of degree attainment.
Historical Trends in Workforce Development. Community colleges, as well as junior colleges, have long served as technical training grounds for some occupations even as early as the beginning of the 20th century (O’Banion, 2019). After the passage of the G.I. Bill and the Higher Education Act, more community colleges emerged (Gilbert & Heller, 2013) and thus, more pre-baccalaureate workforce training and credential programs (Jacobs & Worth, 2019).
While many baccalaureate degrees provide professional preparation for a specific field or industry, degrees with less of a clear occupational connection, such as those found in the liberal arts and social sciences, are still situated within a higher education system aimed to develop graduates that are workforce-ready to step into a job in a variety of professional fields (Labaree, 1997).
Graduate-level education has traditionally aimed to feed students into an occupational pipeline of highly-skilled workers in specific professions. For example, the Commission on Accreditation of Healthcare Management Education mandates that “the Program will adopt a set of competencies that align with the mission and types of jobs graduates enter” (Commission on Accreditation of Healthcare Management, 2014, p. 6). Further, in some occupations, a graduate degree is required. For example, to become a school principal, occupational therapist, or urban planner, one needs a master’s degree in their associated field of study (U.S. Bureau of Labor Statistics, n.d.).
Hierarchical Credentialing Across Degree Levels.The existence of differential degree levels and credentials, especially within the same field or industry, has often contributed to institutionalized credentialing hierarchies within many professions. For instance, in the early 20th century, the pathway to nursing involved either earning a baccalaureate degree or engaging in an apprenticeship in a hospital. Few nurses chose to earn a degree, and as a result hospital diploma programs began to wane (Orsolini-Hain & Waters, 2009). However, the passage of the Hill-Burton Act in 1946 authorized the construction of new hospitals, warranting more nurses. Coupled with the growth of community colleges under the Truman administration, two-year Associate degrees in nursing emerged (Orsolini-Hain & Waters, 2009). Today, the Accreditation Commission for Education in Nursing, for example, accredits eight different levels of programs ranging from certificates to clinical doctorates. Regardless of the credential earned, each degree prepares graduates for some type of professional role in the nursing occupation, with higher degrees often prerequisites for more complex jobs with more responsibility (McWhirter, 2021).
Other occupations, historically, have also created sub-occupations, or semi-professions related to the primary profession, to parse out lower-level duties (Witz, 1992). For instance, the role of a radiographer was created to produce X-rays in medical settings, compared to radiologists who were tasked with interpreting those X-rays using their more comprehensive medical expertise (Witz, 1992). Given the distinction in roles, each of those occupations requires different training and credentials. The minimum credential for a radiographer is typically a Certificate or Associate degree, which is often housed in two-year or vocational institutions, whereas a radiologist often holds an M.D. degree from an accredited medical school (Study.com, 2021).
With a spectrum of professional-occupational programs across degree levels, one can assert that each level should serve a different purpose in the occupational field. And, what should be learned at a more basic level in a pre-baccalaureate program may be vastly different than what should be learned at the baccalaureate and graduate levels. As the associated occupations for each level grow more complex and high-level, one could assert that their programs’ academic rigor, focus, and content would as well. However, little study has been conducted to date to assess presumed differences in learning outcomes across degree levels, let alone the study of leadership competencies across degree levels.
Competency-based Education and Accreditation Within Higher Education. Competencies can be defined as “the skills, knowledge, experience, attributes and behavior that an individual needs to perform a job effectively” (Hirsch & Stabler, 1995 in Horton et al., 2002, p. 4). To effectively assess the extent to which degree programs deliver on their educational promises, one of the many roles of higher education accrediting organizations is to put forth a consistent, robust set of learning outcomes comprised of competencies necessary for career success in fields associated with each of their academic programs (Ewell, 2001). Because earning accreditation serves as a legitimizing message directed at students, faculty, and external stakeholders (Congressional Research Service, 2020) while providing a framework for program design, faculty and administrators follow closely the expectations laid out in accreditation manuals so they can document compliance and receive program accreditation. Adhering to these expectations includes aligning curricular goals with the accrediting organization’s published learning outcomes – which are comprised of knowledge and skill-based competencies – and then assessing learning focused on those competencies. For example, the Council on Social Work Education adopted a competency-based framework in 2008, focusing on “identifying and assessing what students demonstrate in practice” (Council on Social Work Education, 2015, p. 6).
The competencies adopted by many accrediting organizations are also designed to benefit graduates as they move through the profession. For example, the American Society of Radiologic Technologists (2016, p. i) aims “to give students a solid foundation of traditional core knowledge while also providing opportunities to develop skills that will serve them beyond the entry-level of the radiologic sciences.”
Given the diversity of professions, obvious differences exist in specific competencies identified in accrediting organization program learning outcomes. However, even in the same profession grouping, subtle differences emerge in outcomes identified across different academic program levels. For instance, the learning outcomes focused on research across the eight different levels of accredited programs in nursing are similar to each other but include slight variations in wording, indicative of the presence of slightly different competencies (National League for Nursing, 2010):
Practical/Vocational: “considering research”
Associate degree/Diploma: “examine evidence”
Baccalaureate: “identifying questions in need of study, critiquing published research”
Master’s: “contribute to the science of research”
Practice Doctorate: “disseminate practice-based knowledge”
Research Doctorate: “designing and implementing research studies”
There can also be consistency with competencies across academic programs, particularly within a specific field. For instance, of the eight nursing programs, all require students to be able to demonstrate “effective communication,” “navigate conflict skillfully,” and “affirm and value diversity” (National League for Nursing, 2010). Further, in a previous study, Seemiller (2013) found that the three most prominent competencies across all academic degree levels were Verbal Communication, Evaluation, and Writing, indicating a trend of similarity rather than difference.
Workforce and Leadership Competencies. In regard to this study, we examine two sets of competencies: Workforce Competencies and Leadership Competencies, both of which are embedded in academic program learning outcomes.
Workforce Competencies. Workforce competencies can be technical, or those that contribute to a specific job function such as the ability to “interpret medical abbreviations or symbols” (American Society of Radiologic Technologists, 2016). Or, they can be non-technical like “speak and listen critically” (American Society of Radiologic Technologists, 2016), which may help a radiographer generally function in the workplace (Seemiller, 2021). While workforce competencies can include both technical and non-technical competencies, the focus of this paper is on non-technical competencies that are universally useful across a variety of occupations.
Leadership Competencies.Leadership competencies, on the other hand, are intended for use when engaging in leadership in an organizational or workforce setting.Contemporary scholars define leadership as a process, not limited to one’s positionality or level in an organization (e.g., Komives, Lucas, & McMahon, 2009). Therefore, any employee can and should be able to engage in leadership behavior at any given time regardless of their level in the organization.
Further, although workforce and leadership competencies may differ to some extent, they share many similarities (Seemiller, 2021). For instance, the National Association of Colleges and Employers publishes a list annually of “Career Readiness Competencies Employers Want,” where, as one example, collaboration – often taught as a leadership skill in higher education leadership programs (Owen, 2012) – is listed as an essential competency college graduates need as they enter the workforce (National Association of Colleges and Employers, 2020). However, the Center for Creative Leadership maintains a list of essential competencies for leaders and also includes collaboration on that list (Center for Creative Leadership, 2015). To make the delineation between employee and leadership competencies even less clear, the Society for Human Resource Management (2008, p. 1) defines competencies as “leadership skills and behaviors that contribute to superior performance.”
Summary.Given the history and literature on workforce development in higher education, three observations can be made.
Developing students’ skills and competencies for occupational effectiveness is a focus of academic programs across all degree levels within higher education, particularly evidenced by the process of accreditation.
While levels of attention paid by accrediting organizations to specific workforce competencies may vary based on degree level or program, there may also be some workforce competencies that are more universally present regardless of degree or program.
There are both differences and similarities between workforce and leadership competencies.
Our Theoretical Framework
For our study, we analyzed the crossover of competencies from both a leadership competency model and a workforce development model. The leadership competency model used was the Student Leadership Competencies framework (Seemiller, 2013). This framework includes 60 unique leadership-focused competencies that were developed and organized by analyzing both contemporary leadership models and relevant higher education literature on workforce development (Seemiller & Murray, 2013). These 60 competencies are grouped into eight categories and are considered fundamental in the practice of leadership in contemporary society: self-awareness and development, personal behavior, interpersonal interactions, group dynamics, communication, civic responsibility, learning and reasoning, and strategic planning (Seemiller, 2013).
The Student Leadership Competencies framework was created as a result of a comprehensive study of accrediting organizations examining their listed competencies and their overlap with leadership models specifically crafted for developing the leadership capacities of students in a higher education setting (Seemiller & Murray, 2013). The frameworkhas been widely referenced in higher education since its release and has undergone extensive testing to validate a measurement instrument and develop a taxonomy for content sequencing (e.g., Rosch & Seemiller, 2018; Seemiller & Whitney, 2019). Lastly, this framework has been employed in previous studies to analyze learning outcomes in higher education accrediting manuals (Seemiller & Murray, 2013; Seemiller, 2013); we borrow past analytic procedures from these efforts for this current study.
The workforce competency model used was the “Common Employability Skills” published by the National Network of Business and Industry Associations (2014). This universal workforce competency model, absent specific technical competencies geared toward any one profession, was undertaken using an industry-vetting process to create a “model [that] allows employees to understand the skills that all industries believe prepare individuals to succeed” (National Network of Business and Industry Associations, 2014, p. 2). Some competencies listed in this model include teamwork, initiative, listening, positive attitude, problem solving, and planning, which according to their industry experts would be valuable across any professional field (National Network of Business and Industry Associations, 2014).
Figure 1 includes a list of competencies solely present in the Student Leadership Competencies framework (Seemiller, 2013), competencies present in both the Student Leadership Competencies framework (Seemiller, 2013) and “Common Employability Skills” (National Network of Business and Industry Associations, 2014) translated to language specific to the Student Leadership Competencies framework, and competencies only listed in “Common Employability Skills” (National Network of Business and Industry Associations, 2014).
Of the 60 competencies from the Student Leadership Competencies framework (Seemiller, 2013) and 83 competencies from the “Common Employability Skills” (National Network of Business and Industry Associations, 2014), there are 34 competencies are represented across both models.
Figure 1. List of Leadership Competencies, Shared Leadership and Workforce Competencies, and Workforce Competencies
This study involved an examination of leadership competencies rooted within published learning outcomes in academic accrediting manuals across all degree levels and inclusive of the significant accrediting organizations associated with degree programs in postsecondary institutions in the United States. For this study, we asked the following research questions:
To what extent do academic programs within post-secondary accrediting organizations differentially attend to certain leadership and workforce competencies in their learning outcomes?
Are the leadership competencies that receive more attention also aligned with general workforce competencies?
Do differences exist in the attention paid to leadership competencies within learning outcomes based on differing academic degree levels (i.e., Associate, Bachelor’s, Graduate), and if so, are there patterns to those differences?
Data for this study was gathered through an analysis of learning outcomes of accredited academic programs in higher education from a comprehensive data set that has also been utilized in a previous publication (e.g., Seemiller, 2021).
Data Collection. The first step of data collection involved creating a list of accredited academic programs by narrowing options first from an accrediting body and then to an accrediting organization. To better understand this process, three terms need to be defined.
Accrediting Body: An entity that accredits accrediting organizations and their respective programs (e.g., Council for Higher Education Accreditation, or CHEA). Accrediting bodies can also accredit institutions as a whole. Only accrediting bodies that accredit at the program level were included in this study.
Accrediting Organization: A council, organization, or association aimed to accredit specific academic programs within a particular field or industry (e.g., National League for Nursing).
Academic Program: A postsecondary academic program focused on a specific subject matter which results in a credential or degree (e.g., Bachelor’s Degree in Nursing).
To create a list of accredited academic programs, we began by consulting the Council for Higher Education Accreditation (CHEA), the U.S. Department of Education (DOE), and the Association of Specialized Accreditors (ASPA), which are prominent accrediting bodies in the U.S. In total, the three bodies accredit 83 accrediting organizations which oversee the accreditation of 605 academic programs in total. From this review, a spreadsheet with a page for each of the 83 accrediting organizations and their associated academic programs was created. To learn the details of the programs each organization accredits, their accrediting manuals were obtained, either freely on their websites or by purchase from the organization directly. Organizations accredit anywhere from one program (e.g. American Library Association) to 124 programs (e.g. Accreditation Board for Engineering and Technology).
For each program, we identified a degree level (Level 1, 2, or 3). Level 1 included all programs that provide certification below the baccalaureate level: Diplomas, Certificates, Associate degrees, and Post-Associate certificates. Overall, 130 Level 1 programs were included in this study. Level 2 included all programs at the baccalaureate level: Bachelor’s degrees and Post-Bachelor’s certificates. In this study, we identified 206 Level 2 programs. Level 3 included all graduate programs: Master’s degrees, Post-Master’s certificates, Doctorate degrees, Post-Doctorate certificates, and Post-Doctorate residencies, where we identified 269 programs. We examined every associated learning outcome published in each accrediting manual on the relevant organization spreadsheet page. In total, 36,327 learning outcomes were listed across all 605 programs.
Data Organization and Analysis. For each program, the text of all associated learning outcomes was entered individually into the rows on the organization’s spreadsheet. Across the top of the spreadsheet, each of the 60 Student Leadership Competencies (Seemiller, 2013), which served as the a priori codes in this study, were added, one per column. Figure 1 includes a list of all leadership competencies and therefore, the a priori codes we utilized, organized by category.
Identifying the Competency.The first step of coding involved analyzing the language within each stated learning outcome to look for alignment with one of the a priori codes. In most cases, an outcome could be easily associated with a particular competency because the language was identical. For example, an outcome from the AASCB Master’s degree in Accounting was written as “Group and individual behaviors in organizations and society.” This outcome was coded with the a priori code of Group Dynamics. For outcomes lacking language directly reflective of a competency definition, a synonyms list initially created by Seemiller and Murray (2013) and Seemiller (2013) in previous related analyses was used. For example, learning outcomes that referred to a concept such as “teamwork” were coded as “Collaboration.” Using a synonyms list helped ensure inclusivity of terms beyond direct verbiage from competency definitions as well as created consistency with analysis across outcomes.
Coding Process.If an a priori code was present in the text of the learning outcome, the spreadsheet column for that a priori code was coded with a number 1. In some cases, there was more than one a priori code present in the learning outcome text, warranting a number 1 in more than one column. For example, one learning outcome from the Planning Accreditation Board is worded as, “Leadership: tools for attention, formation, strategic decision-making, team building, and organizational/community motivation” (Planning Accreditation Board, 2017, p. 10). This outcome was coded with a number 1 for each of the following competencies: Decision-Making, Motivation, and Group Development.
This process of coding is a form of quantitative text analysis, which involves reducing text into statistically measurable values (Shapiro & Markoff, 1997), which helps in summarizing themes in a measurable way (Hannah & Lautsch, 2011) and making meaning of what was originally non-numerical data (Maxwell, 2010). Not all outcomes were associated with an a priori code; many were technical and specific to the discipline. For example, one of the competencies for the AACSB Master’s degree in Accounting is “Tax policy, strategy, and compliance for individuals and enterprises” (AACSB, 2018, p. 24). These outcomes were coded with a “0,” indicating no presence of an a priori code. Once all learning outcomes were coded, totals for each a priori code were calculated. This total reflects the extent to which attention is paid to that particular code; in effect, how often and therefore deeply the competency is included across different learning outcomes.
Data Analysis.We conducted three types of analyses: a simple frequency count of leadership competencies identified across learning outcomes; a Chi-square analysis to investigate differences between expected and measured frequencies; and an ANOVA analysis to more deeply look at patterns of differences within degree levels and categories of leadership competencies.
Frequency Count and Chi-square Analyses. Our first investigation gauging the extent to which postsecondary accrediting organizations focus on leadership competencies began simply with conducting a frequency analysis of competencies across learning outcomes, particularly to determine the degree of difference in attention given to leadership competencies by accrediting organizations. To address the existence of broad differences in attention to specific leadership competencies across accrediting organizations, we conducted a Chi-square analysis which compared the measured frequencies of each leadership competency in the framework with what would be expected if attention to each competency were evenly distributed across competencies. The frequency count and Chi-Square analysis allowed us to determine the overall shape of attention to specific leadership competencies and whether statistically significant differences in attention emerged across all competencies.
ANOVA Analyses.We separated the list of competencies into their eight respective categories discussed earlier, as well as by degree level (i.e., Associate, Bachelor’s, and Graduate), and conducted a series of ANOVA analyses to determine the degree to which differences emerging from the Chi-square analysis were represented in certain competency categories and degree levels. This modification allowed us to continue to focus on patterns of potential differences in attention from accrediting organizations while avoiding the need to employ a model with 60 separate individual items. Because the number of programs that grant Associate, Bachelor’s, and Graduate degrees differ in the United States, we created a standardized statistic representing the “per-program” frequency of identified leadership competency categories across degree levels. We then analyzed differences in attention to each competency category within each degree level by conducting separate ANOVAs comparing these standardized frequencies.
We were also interested in investigating the degree to which variability existed across academic degree levels. Here, our interest was not to examine differences in the attention given to certain leadership competencies compared to others, but rather to examine the degree to which differences might exist across degree levels with the attention accrediting organizations pay to specific leadership competencies in their learning outcomes. For example, potentially, Associate degree programs might focus on some competencies more than graduate-level programs, while those awarding graduate-level degrees might focus on others more than Associate degree programs). To address this issue, we conducted a series of eight ANOVAs that analyzed these frequencies across program degree levels within each leadership competency category.
Through our analyses, we sought to answer three research questions to better understand if, and to what extent, patterns emerged in regard to the presence of leadership competencies within accrediting organization programmatic learning outcomes.
Varying Emphasis Across Leadership Competencies. The first set of analyses aimed to answer research question 1: Do academic programs within post-secondary accrediting organizations differentially emphasize leadership competencies? Table 2 includes the competencies in order of highest observed frequencies to lowest observed frequencies. There is variation in these frequencies; for example, the competency of Evaluation shows up 2,094 times across all learning outcomes compared to three occurrences of the Positive Attitude competency. From the findings from this frequency analysis, it is apparent that academic programs emphasize some competencies much more than others.
To investigate these differences further, we conducted a 2×60 Chi-square analysis, comparing the measured frequencies of each competency to what we would expect to see if each competency was identified at a rate equal to the others. For example, the leadership competency, Diversity, which is defined as “Interacting with people who have different backgrounds, beliefs, and/or experiences” (Seemiller, 2020), was identified a total of 146 times across all learning outcomes. However, were each competency identified to an equal degree, Diversity would have been identified 260 times (i.e., the mean frequency calculated by dividing the total number of competencies by the total number of times they were cumulatively identified). In total, 17 of the 60 competencies yielded higher than the mean frequency and are listed in Table 1, whereas 43 of the 60 competencies yielded lower than the mean frequency in Table 2. This analysis across all leadership competencies resulted in a statistically significant Chi-square result (χ2(59)=34,002.26, p<.0001), and indicates that the degree of differences in attention paid to each of the 60 leadership competencies is not likely due to random variation.
Further, when looking by category, Communication (71.4%) and Learning and Reasoning (60%) were the only two in which more than 50 percent of the competencies in the category yielded higher than expected frequencies. All other categories were considerably lower: Strategic Planning (20%), Interpersonal Interaction (20%), Civic Responsibility (16.7%), Self-Awareness and Development (16.7%), Personal Behavior (9.1%), and Group Dynamics (0%).
Frequencies by Competency: Higher-Than-Expected Results
Learning and Reasoning
Learning and Reasoning
Learning and Reasoning
Learning and Reasoning
Learning and Reasoning
Learning and Reasoning
Advocating for a Point of View
NOTE: +: A competency that is also identified in the list “Common Employability Skills.”
Frequencies by Competency: Lower-Than-Expected Results
+Scope of Competence
Learning and Reasoning
Reflection and Application
Learning and Reasoning
Learning and Reasoning
+Responsibility for Personal Behavior
Learning and Reasoning
Responding to Ambiguity
+Responding to Change
NOTE: +: A competency that is also identified in the list “Common Employability Skills.”
Emphasis of Leadership and Workforce Competencies. We also aimed to answer research question 2: Are the leadership competencies that are aligned with general workforce competencies have higher frequency counts than those not aligned? Sixteen of the 17 leadership competencies (94%) with higher than expected frequency counts (260 or more) across learning outcomes are also workforce competencies.
Workforce Competencies by Category and Frequency
Number of leadership competencies associated with workforce competencies
Number of leadership competencies associated with workforce competencies with higher than expected frequencies (260 or higher)
2 of 6 (33.3%)
1 of 2 (50%)
5 of 7 (71.4%)
5 of 5 (100%)
1 of 4 (25%)
0 of 1 (0%)
4 of 11 (36.4%)
2 of 4 (50%)
8 of 10 (80%)
6 of 8 (75%)
7 of 11 (63.6%)
1 of 7 (14.3%)
Self-Awareness & Development
2 of 6 (33.3%)
1 of 2 (50%)
3 of 5 (60%)
1 of 3 (33.3%)
Table 3 highlights the number of leadership competencies associated with general workforce competencies. Some categories include several competencies that are listed at a rate of 260 or more, such as five of seven competencies in the Communication category. Other categories, however, were identified fewer times than the mean of 260, such as those in the Group Dynamics category, where only one of four competencies included within both lists were identified 260 times or more. Notable is the pattern of frequency representation for the competencies that are both leadership and workforce competencies in the accrediting organization learning outcomes. For example, while seven of the 11 competencies in the Personal Behavior category are associated with both leadership and the workforce, only 1 of those 7 have a higher than average frequency count.
Emphasis of Competencies By Degree Level. Finally, we sought to investigate research question 3: Do differences exist in the attention paid to leadership competencies within learning outcomes based on differing academic degree levels (i.e., Associate, Bachelor’s, Graduate), and if so, are there patterns to those differences?
Our analysis to investigate similarities and differences between degree levels involved collapsing leadership competencies within their respective overall categories and specific academic degree levels. After creating standardized per-program frequencies to control for the different number of degree programs across each of the three degree levels, we conducted 24 separate ANOVA analyses, three within each of the eight competency categories, one for each academic degree program level. Our results are summarized in Table 4, which show statistically significant results (each p<.0001) with most η2 effect sizes ranging from moderate to large (Cohen, 1973). These findings indicate the moderate to large disparity of attention given to individual competencies even within categories and specific degree levels.
Self-Aw. & Dev.
ANOVAs by Category Within Academic Levels
Our final analysis focused on investigating the degree to which differences existed across degree levels within each competency category. Employing the standardized per-program frequency of identifications of each individual competency, we grouped these frequencies into their respective categories and conducted a series of eight ANOVAs to investigate differences across degree levels. Table 5 lists the results of these analyses, which show non-statistically significant results (p<.05) within each category, suggesting that no systemic differences exist between academic degree levels to the extent that they focus on specific competencies within categories.
Across-Academic-Level ANOVAs Within Categories
Self-Awareness & Development
Several findings emerged from this study. Our frequency and Chi-square analysis results suggested a moderate to large degree of statistical variation, using Cohen’s (1988) terminology, across particular leadership competencies within academic program learning outcomes, generally. These patterns persisted even for leadership competencies that have also been identified as workforce competencies. Notably, several leadership and workforce competencies have been largely omitted in the learning outcomes that are published by most postsecondary accrediting organizations. Lastly, we found a lack of statistical difference in the extent degree levels vary in focus on categories of competencies. Taken together, these findings suggest that some competencies related to postsecondary student leadership development receive far more attention in academic programs than others and that existing under- and overrepresentation of outcomes may be consistent across degree levels.
Varied Emphasis on Competencies Across Learning Outcomes. It is clear that there is a differentiation in the representation of the competencies within the learning outcomes analyzed. For example, the competency of Evaluation appears more than 2000 times, whereas Positive Attitude only appears three times. The variance in attention paid to competencies may be influenced by both higher education and workforce contexts driving the curriculum today. As such, it might be expected that Evaluation, which includes critical thinking, would emerge as a prominent competency, given the Association of American Colleges & Universities (n.d.) highlights critical thinking as an essential skill that students should learn in college to prepare them for addressing contemporary challenges. Further, 99 percent of employers want employees who can critically think (National Association of Colleges and Employers, 2020). On the other end of the spectrum of attention, supporting students in developing a positive attitude to their work was identified in only three learning outcomes across the 605 programs we included in this study. It may or may not be related that intrapersonal competencies like this have received much less attention from organizations like the Association of American Colleges & Universities (n.d.).
Of the 17 leadership competencies with frequency counts of 260 or more, 16 are also workforce competencies (94%). These 16 were primarily clustered into only two leadership competency categories: Communication as well as Learning and Reasoning, which may seem logical, as these clusters of competencies also contribute to generalized academic success in a higher education learning environment.
Unaddressed Competency Needs for Today’s Workforce. Even though the competencies that were identified more often were frequently both leadership and workforce competencies, 16 competencies included on both lists yielded frequencies lower than the mean. Accrediting organizations may need to focus on integrating these potentially underrepresented workforce competencies to a greater extent into their learning outcomes so their graduates are equipped with the skills they need to be successful in their future occupations. In particular, competencies such as Diversity, Conflict Negotiation, and Responding to Change, all emerged less often than the mean, which might be particularly relevant for student learning given our society’s increased emphasis on social justice and equity since the 2020 anti-racism protests and need for agility and flexibility during the COVID-19 pandemic. If mastering these competencies does not take place within higher education, where else can today’s graduates systemically address their learning deficits with these competencies?
Further, there were 26 leadership competencies not listed in the workforce competency model that had lower-than-expected frequencies. A logical reason might be that these leadership competencies do not serve essential workforce needs. However, a deeper consideration raises cause for potential concern. For example, two of the three least frequently identified competencies in the learning outcomes include Resiliency and Confidence, both of which are also not listed as workforce competencies. The lack of presence of these competencies raises questions about how graduates become equipped to effectively meet the mental and emotional demands of working in a post-pandemic society. Even prior to the pandemic, work was shifting, requiring employees to manage ambiguity, flat organizational structures, consistent change, and shifting priorities (Burke, Warner, & Noumair, 2015). The global pandemic and its accompanying upheaval have highlighted the degree to which individuals must leverage these and related competencies in a workforce context for both individual professional performance as well as organizational stability.
Lack of Differentiation Between Degree Levels. No statistically significant differences emerged within competency categories across degree levels, suggesting that academic programs seem to pay the same degree of attention to similar leadership competencies regardless of academic level. Such a finding might imply that students in Associate degree programs are exposed to the same leadership-focused curriculum as students in Graduate degree programs, for example, and that students who graduate from one level of curriculum to another might be expected to master the same competencies. Presumably, differentiation might exist within a specific leadership competency, which may not emerge from a frequency analysis. For example, students in an Associate degree-granting program might learn basic skills related to conflict management and more advanced skills within a Bachelor’s degree program. Still, it is noteworthy that no statistically significant differences across degree levels emerged within any of the eight categories of leadership competencies. At the least, our findings suggest strong topical overlap with regard to leadership as well as workforce development across all levels of academic programs, which may signal strength in continually reinforcing important learning outcomes or weakness in repeating similar outcomes regardless of academic level.
Our findings suggest there is a disparity of attention paid to specific leadership (and workforce) competencies within postsecondary education and that this pattern of disparity is consistent across degree levels.
Implications for Postsecondary Educators. The leadership competencies most often included in accrediting organizations’ stated learning outcomes might be divided into cognitive skills (e.g., using information and research techniques to make rigorous evaluation-based decisions; analyzing data; and understanding ethics), and relational skills (e.g., communicating verbally and collaborating with others; engaging in appropriate interactions; understanding the circumstances of people beyond yourself). However, many categories associated with relational skills (e.g., Interpersonal Interaction or Group Dynamics), yielded frequencies of competencies lower than average. Relational skills, to be sure, possess critical importance for success in the contemporary workforce given the degree to which educated employees should be able to manage complex information streams while working in collaborative relationships (Cherniss et. al, 1998).
Other leadership competencies that seem situated on the margins of program learning outcomes across the disciplines almost exclusively focus on intrapersonal skills like possessing a positive attitude; demonstrating resilience and confidence in the face of challenge; and building a sense of mission and vision in one’s work, demonstrated by frequency identification much lower than the mean frequency count. The importance of these intrapersonal competencies cannot be understated in the contemporary workforce.
Further, given the attention currently being paid to issues related to social justice, competencies such as Social Justice, Inclusion, Other Perspectives, Empathy, and Helping Others, despite not being labeled as essential workforce competencies, seem crucial for students to be exposed to in order to navigate the complexities of both societal and organizational dynamics regarding equity and equality.
Given graduates’ need for acquiring proficiency in critical workforce competencies, faculty will want to consider how to implement into the curriculum those workforce competencies with frequencies lower than the average. These competencies can be woven into the framework for course design, assignments, selected readings, and even examples provided through instruction.
Further, perhaps there is also a natural collaboration opportunity with student affairs professionals whose programs often address many of the competencies around intrapersonal development and social justice. For example, student organization training, programs in residential housing, and leadership, multicultural, and career-oriented events and initiatives could provide excellent opportunities for student development and complement the credit-bearing curriculum.
These results also beg the question, to what degree should all competencies receive equal attention among learning outcomes in postsecondary education? Peter Drucker (e.g., 2020) is often attributed to having written the quote, “What gets measured, gets managed,” which implies that it is only those items that are included on lists of organizational goals and assessment measurements that receive attention in the form of intentional treatment. Many of the more intrapersonal competencies are more difficult to measure – such as how students develop resiliency attitudes and behaviors – can still be assessed without intensive resources. For example, when students are assigned difficult assignments or volunteer for challenging co-curricular projects, they develop the skills for resilience simply by persisting through the experience. However, because accrediting organizations pay little attention to these competencies, postsecondary educators put comparatively less effort in assessing such development and possess a lack of empirical data supporting development commensurate with their effort.
Many colleges and universities house their own offices of student outcomes assessment and enact their measurement efforts independent of, and occasionally in conjunction with, relevant accrediting organizations. To be clear, we are not suggesting that these important campus units are not already engaging in the assessment of leadership competency development, especially in categories that might be less emphasized in accreditation processes. However, accrediting organizations rose in significance in postsecondary education as a means to provide consistency across colleges and universities offering similar degree programs. The extent that these organizations might be marginalizing important leadership outcomes within postsecondary education is likely comparable to the extent of variability of attention paid across similar degree programs to these leadership outcomes. Some college or university programs might provide exemplary education in helping students gain and sustain self-confidence, resiliency, a positive attitude, and an attention to detail in follow-through on their goals. Other institutions or programs, however, might not.
Implications for Postsecondary Researchers. This current study was designed to provide a wide lens snapshot of the state of postsecondary education across disciplines and degree levels regarding the depth and patterns of attention given to a broad set of leadership development competencies relevant to students. Our broad sweep of programs and degrees allowed us to draw some potential conclusions about the state of postsecondary education overall in the United States regarding its treatment of leadership development, and subsequently, workforce development. However, our analysis represented more of a blunt instrument than a scalpel in this assessment. Further steps to drill down to more specificity is crucial for a more fine-tuned understanding of whether and how academic programs are being held accountable for educating their students in the skills of leadership development, and in many cases, general workforce development, in any particular discipline or degree level.
This study was also designed to serve as an indirect indicator of student learning, focusing on what is being assessed rather than what is actually being learned. Our results imply that some types of leadership development – such as that related to cognitive reasoning and relational skills – are largely supported across postsecondary education in the United States, while other areas – such as intrapersonal development – receive less attention. Research attempting to connect these findings with assessments of empirical growth might shed light on the significance of these particular findings and bolster (or challenge) the need to validate these results through future research efforts and potentially dig deeper to understand the practical implication for leaving some aspects of student leadership development up to chance in certain academic disciplines.
We conducted a multi-disciplinary, multi-degree study of 605 post-secondary academic programs that included all relevant higher education accrediting organizations in the United States to uncover patterns in the presence of leadership-related competencies within the stated learning outcomes employed by these organizations. The findings from the study suggested a large degree of variability in attention paid to particular areas of leadership competence development. Competencies identified far above the mean frequency count related to Learning and Reasoning as well as Communication. However, Personal Behavior competencies such as Resiliency and Positive Attitude were almost completely absent from the learning outcomes identified across accrediting organizations.
Moreover, while we found a disparity in attention to certain competencies over others, we found similarities in the patterns of attention to these competencies across academic degree levels (Associate, Bachelor’s, and Graduate). While this study sought to examine the whole of postsecondary education in scope in the United States and therefore did not focus on specific disciplines, degrees, or institutions, these findings are concerning in the degree to which academic programs within postsecondary institutions, as a whole, support their students in developing the leadership and workforce skills required for their success in their workplaces upon graduation.
AACSB. (2018). 2018 eligibility procedures and accreditation standards for accounting accreditation. https://www.aacsb.edu/-/media/documents/accreditation/2018-business-standards.pdf?rev=87acd91b6dcd4ca987f61fd63d6dac4b&hash=02A7DC298A392FF34BBA3FDBA27E1B01
Adams, J. A. (2000, November 16-19). The G.I. Bill and the changing place of U.S. higher education after World War II [Paper presentation]. Association for the Study of Higher Education Annual Meeting, Sacramento, CA. https://files.eric.ed.gov/fulltext/ED449721.pdf
American Society of Radiologic Technologists. (2016). Radiography curriculum. https://www.asrt.org/docs/default-source/educators/curriculum/acad_curr_radrvsn2016draft_20160321.pdf?sfvrsn=2
Association of American Colleges & Universities. (n.d.). Essential learning outcomes. https://www.aacu.org/essential-learning-outcomes
Brint, S., Riddle, M., Turk-Bicakci, L., & Levy, C. S. (2005). From the liberal to the practical arts in American colleges and universities: Organizational analysis and curricular change. The Journal of Higher Education, 76(2), 151-180. https://doi.org/10.1080/00221546.2005.11778909
Burke, W. W., & Noumair, D. A. (2015). Organization development: A process of learning and changing. FT Press.
Center for Creative Leadership. (2015). The leadership gap: What you need, and still don’t have, when it comes to leadership talent.http://cclinnovation.org/wp-content/uploads/2020/03/leadership-gap-what-you-need.pdf
Cherniss, C., Goleman, D., Emmerling, R., Cowan, K., & Adler, M. (1998). Bringing emotional intelligence to the workplace. https://www.eiconsortium.org/pdf/technical_report.pdf
Cohen, J. (1988). Statistical power for the behavioral sciences (2nd ed.). Routledge.
Cohen, J. (1973). Eta-squared and partial eta-squared in fixed factor ANOVA designs. Educational and psychological measurement, 33(1), 107-112. https://doi.org/10.1177/001316447303300111
Commission on Accreditation of Healthcare Management Education. (2014). Eligibility requirements. cahme.org.
Congressional Research Service. (2020). An overview of accreditation of higher education in the United States. https://fas.org/sgp/crs/misc/R43826.pdf
Council on Social Work Education. (2015). Educational policy and accreditation standards. https://www.cswe.org/Accreditation/Standards-and-Policies/2015-EPAS
Drucker, P. F. (2020). The essential Drucker. Routledge.
Ewell, P. T. (2001). Accreditation and student learning outcomes: A proposed point of departure. Council for Higher Education Accreditation, Washington, DC. https://files.eric.ed.gov/fulltext/ED469482.pdf
Gilbert, C. K., & Heller, D. E. (2013). Access, equity, and community colleges: The Truman Commission and federal higher education policy from 1947 to 2011. The Journal of Higher Education, 84(3), 417-443. https://eric.ed.gov/?id=EJ1011675
Hannah, D. R., & Lautsch, B. A. (2011). Counting in qualitative research: Why to conduct it, when to avoid it, and when to closet it. Journal of Management Inquiry, 20(1), 14–22. https://doi. org/10.1177/1056492610375988
Horton, S., Hondeghen, A., & Farnham, D. (2002). Competency management in the public sector: European variations on a theme. IOP Press.
Jacobs, J., & Worth, J. (2019). The evolving mission of workforce development in the community college. Community College Research Center Working Paper No. 107. https://files.eric.ed.gov/fulltext/ED594218.pdf
Komives, S. R., Lucas, N., & McMahon, T. R. (2009). Exploring leadership: For college students who want to make a difference. John Wiley & Sons.
Labaree, D. F. (1997). Public goods, private goods: The American struggle over educational goals. American Educational Research Journal, 34(1), 39-81. https://doi.org/10.3102/00028312034001039
Maxwell, J. A. (2010). Using numbers in qualitative research. Qualitative Inquiry, 16(6), 475–482. https://doi. org/10.1177/1077800410364740
McWhirter, K. (2021). Types of nursing degrees and levels. Nurse Journal. https://nursejournal.org/degrees/types-of-nursing-degrees/
National Association of Colleges and Employers. (2020). Job outlook for the class of 2020. https://www.stjohns.edu/sites/default/files/2020-05/nace_job_outlook_0.pdf
National Conference of State Legislatures. (2019). Higher education legislation in 2018. https://www.ncsl.org/research/education/higher-education-legislation-in-2018.aspx
National League for Nursing. (2010). Outcomes and competencies for graduates of practical/vocational, diploma, associate degree, baccalaureate, master’s practice doctorate, and research doctorate programs in nursing. National League for Nursing.
National Network of Business and Industry Associations. (2014). Common employability skills. https://s3.amazonaws. com/brt.org/archive/Common%20Employability_asingle_fm.pdf
O’Banion, T. U. (2019). A brief history of workforce education in community colleges. Community College Journal of Research and Practice, 43(3), 216-223.https://doi.org/10.1080/10668926.2018.1547668
Orsolini-Hain, L., & Waters, V. (2009). Education evolution: A historical perspective of associate degree nursing. Journal of Nursing Education, 48(5), 266-271. DOI: 10.3928/01484834-20090416-05
Owen, J. (2012). Examining the design and delivery of collegiate student leadership development programs: Findings from the Multi-Institutional Study of Leadership (MSL-IS), a national report. Washington, DC: Council for the Advancement of Standards in Higher Education.
Planning Accreditation Board. (2017). PAB accreditation standards and criteria. https://www.planningaccreditationboard.org/pab-accreditation-standards-and-criteria/
Rosch, D. M., & Seemiller, C. (2018). A psychometric analysis of the Student Leadership Competencies Inventory. Journal of Leadership Education, 17(4), 146-168. DOI:10.12806/V17/I4/R9
Seemiller, C. (2021). Preparing leaders of tomorrow: An analysis of leadership competencies within accredited academic program learning outcomes. Journal of Leadership Studies, 15(1), 6-27. https://doi.org/10.1002/jls.21737
Seemiller, C. (2020). The Student Leadership Competencies resource guide. n.p.: Independently published.
Seemiller, C. (2013). The student leadership competencies guidebook: Designing intentional leadership learning and development. Jossey-Bass.
Seemiller, C., & Murray, T. (2013). The common language of leadership. Journal of Leadership Studies, 7(1), 33-45. https://doi.org/10.1002/jls.21277
Seemiller, C., & Whitney, R. (2020). Creating a taxonomy of leadership competency development. Journal of Leadership Education, 19(1), 119-132. DOI: 10.12806/V19/I1/R5
Shapiro, G., & Markoff, J. (1997). A matter of definition. In C.W. Roberts (Ed.), Text Analysis for the Social Sciences: Methods for Drawing Statistical Inferences from Texts and Transcripts. Routledge.
Society for Human Resource Management. (2008). Leadership competencies. https://www.shrm.org/resourcesandtools/hr-topics/behavioral-competencies/leadership-and-navigation/pages/leadershipcompetencies.aspx
Study.com. (2021). How to become a radiologist: Schooling & certification requirements. https://study.com/radiologist_education.html
Trow, M. (1973). Problems in the transition from the elite to mass higher education. Carnegie Commission on Higher Education. https://files.eric.ed.gov/fulltext/ED091983.pdf
U.S. Bureau of Labor Statistics. (n.d.). Occupational outlook handbook. https://www.bls.gov/ooh/
Witz, A. (1992). Professions and patriarchy. Routledge.